Maximum Relevance Minimum Redundancy (mRMR) is a widely used feature selection method that is applied in a wide range of applications in various fields. mRMR adds to the optimal subset the features that have high relevance to the target variable while having minimum redundancy with each other. Mutual information is a key component of mRMR as it measures the degree of dependence between two variables. However, the real value of mutual information is not known and needs to be estimated. The aim of this study is to examine whether the choice of mutual information estimator affects the performance of mRMR. To this end, three variations of mRMR are compared. The first one uses Parzen window estimation to assess mutual information between continuous variables. The second is based on equidistant partitioning using the cells method, while the third incorporates a bias-corrected version of the same estimator. All methods are tested with and without a regularization term in the mRMR denominator, introduced to improve numerical stability. The evaluation is conducted on synthetic datasets where the target variable is defined as a combination of continuous features, simulating both linear and nonlinear dependencies. To demonstrate the applicability of the proposed methods, we also include a case study in real-world classification tasks. The study carried out showed that the choice of mutual information estimator can affect the performance of mRMR and it must be carefully selected depending on the dataset and the parameters of the examined problem. The application of the corrected mutual information estimator improves the performance of mRMR in the examined setup.

In feature selection, it is crucial to identify features that are not only relevant to the target variable but also non-redundant. Conditional Mutual Information Nearest-Neighbor (CMINN) is an algorithm developed to address this challenge by using Conditional Mutual Information (CMI) to assess the relevance of individual features to the target variable, while identifying redundancy among similar features. Although effective, the original CMINN algorithm can be computationally intensive, particularly with large and high-dimensional datasets. In this study, we extend the CMINN algorithm by parallelizing it for execution on Graphics Processing Units (GPUs), significantly enhancing its efficiency and scalability for high-dimensional datasets. The parallelized CMINN (PCMINN) leverages the massive parallelism of modern GPUs to handle the computational complexity inherent in sequential feature selection, particularly when dealing with large-scale data. To evaluate the performance of PCMINN across various scenarios, we conduct both an extensive simulation study using datasets with combined feature effects and a case study using financial data. Our results show that PCMINN not only maintains the effectiveness of the original CMINN in selecting the optimal feature subset, but also achieves faster execution times. The parallelized approach allows for the efficient processing of large datasets, making PCMINN a valuable tool for high-dimensional feature selection tasks. We also provide a package that includes two Python implementations to support integration into future research workflows: a sequential version of CMINN and a parallel GPU-based version of PCMINN.