Imbalance in classes is one of the critical issues in machine learning practice, especially in the context of healthcare and organizational applications, where the class of interest, representing subjects at high risk of some adverse event, holds significant clinical and organizational value. In this paper, a comparative machine learning framework is presented to assess the influence of Synthetic Minority Over-sampling Technique (SMOTE) and stratified validation strategies on burnout risk prediction under extreme class imbalance scenarios. There was an apparent class imbalance ratio of 7.16:1 between the two classes of Not at Risk and At Risk (Moderate or High). There were sixteen possible experimental setups that were used, where each algorithm was evaluated using two approaches for addressing the class imbalance problem, with and without using SMOTE, along with hold-out and 5-Fold Stratified Cross-Validation. Based on the results, SMOTE provided a significant improvement in the performance of minority class detection in all the models evaluated without having any negative impact on the global model accuracy. Additionally, the application of SMOTE along with Stratified Cross-Validation was found to be the most effective in terms of achieving balance in classification performance. Overall, the use of XGBoost along with SMOTE and 5-Fold Stratified Cross-Validation provided the best results in this respect, with the highest minority class F1-Score and ROC-AUC performance. In particular, the SHAP-based analysis showed that SMOTE contributed to redistributing the attributions among a wide set of clinically relevant features such as stress level, hours of work overtime, and anxiety. Overall, the use of SMOTE and Stratified Cross-Validation was proven to significantly improve the performance of burnout risk classification in imbalanced data samples.