Originally developed as an effective feature selection method in healthcare predictive analytics, Recursive Feature Elimination (RFE) has gained increasing popularity in Educational Data Mining (EDM) due to its ability to handle high-dimensional data and support interpretable modeling. Over time, various RFE variants have emerged, each introducing methodological enhancements. To help researchers better understand and apply RFE more effectively, this study organizes existing variants into four methodological categories: (1) integration with different machine learning models, (2) combinations of multiple feature importance metrics, (3) modifications to the original RFE process, and (4) hybridization with other feature selection or dimensionality reduction techniques. Rather than conducting a systematic review, we present a narrative synthesis supported by illustrative studies from EDM to demonstrate how different variants have been applied in practice. We also conduct an empirical evaluation of five representative RFE variants across two domains: a regression task using a large-scale educational dataset and a classification task using a clinical dataset on chronic heart failure. Our evaluation benchmarks predictive accuracy, feature selection stability, and runtime efficiency. Results show that the evaluation metrics vary significantly across RFE variants. For example, while RFE wrapped with tree-based models such as Random Forest and Extreme Gradient Boosting (XGBoost) yields strong predictive performance, these methods tend to retain large feature sets and incur high computational costs. In contrast, a variant known as Enhanced RFE achieves substantial feature reduction with only marginal accuracy loss, offering a favorable balance between efficiency and performance. These findings underscore the trade-offs among accuracy, interpretability, and computational cost across RFE variants, providing practical guidance for selecting the most appropriate algorithm based on domain-specific needs and constraints.

This study systematically explores Recursive Feature Elimination (RFE) and its variants as effective methods for feature selection within educational data mining (EDM) and healthcare analytics. Feature selection is crucial in predictive modeling due to challenges arising from high dimensionality and data complexity. The paper first reviews foundational principles of the RFE algorithm, highlighting its iterative mechanism for ranking and removing less informative features. Next, it provides a comprehensive narrative review of various RFE modifications, including integration with different machine learning models, utilization of multiple feature importance metrics, algorithmic enhancements through cross-validation and local search techniques, and hybrid approaches combining RFE with other dimensionality reduction methods. Subsequently, an empirical evaluation compares four RFE variants—standard RFE, RF-RFE, Enhanced RFE, and RFE with local search—across two distinct predictive tasks: regression for predicting students’ mathematics achievement using a digital assessment dataset, and classification for predicting chronic heart failure in a clinical dataset. Results indicate that while RF-RFE captured complex feature interactions and improved predictive performance slightly, Enhanced RFE provided substantial dimensionality reduction with minimal loss in accuracy, offering a balanced approach beneficial for practical applications. The study concludes by discussing implications, limitations, and recommendations for effectively applying RFE in educational and clinical contexts.