Classification of High-dimensional data, such as gene expression data having far more variables (genes) than observations, is challenging. Classifiers aggregation known as ensemble method has proven improved classification accuracy in a wide range of applications. In this study, we propose an Ensemble of Penalized Logistic models (EPL) which utilizes Ridge regression, Lasso and Elastic net as base learners for ensemble generation. Classification is done on the basis of majority votes of the models. The EPL is assessed on both simulated data and bench mark microarray data sets. Its performance in terms of classification accuracy is compared with state-of-the-art classifiers, i.e., Support Vector Machines (SVM), K-Nearest Neighbors (KNN) and Random Forest (RF). The experimental comparisons show that the EPL has high classification accuracy as compared to the other classifiers considered here.
3 March 2022 in Communications in Statistics - Simulation and Computation
Classification accuracy of any classifier can be enhanced by performing the classification on selected informative features. Features selection methods are generally approached for the purpose. Combining multiple models known as ensemble method emerged as prominent method for achieving classification accuracy. These techniques can be considered for variable selection for accuracy gain and interpretability of a classifier. we propose an ensemble of penalized logistic models (EPLM) for feature selection. EPLM employs Lasso, adaptive Lasso and elastic net for feature selection. EPLM is applied to high dimensional microarray data sets and simulated data sets. State-of-the-art classifiers, Lasso, Random Forest (RF), Support Vector Machine (SVM) and K- Nearest Neighbors (KNN) are employed for assessment of EPLM. Experimental comparisons reveal that significant improvement in classification accuracy is achieved for all the classifiers considered. The proposed method is also compared to the Ensemble of subset of KNN classifiers (ESKNN). In comparison to ESKNN EPLM achieves better performance accuracy for all the classifiers on simulated and microarray data sets. Moreover, it is also observed that EPLM has selected significantly smaller number of features as compared to the full feature set.