Numerous investigations have been conducted to enhance the motor imagery-based brain–computer interface (BCI) classification performance on various aspects. However, there are limited studies comparing their proposed feature selection framework performance on both objective and subjective datasets. Therefore, this study aims to provide a novel framework that combines spatial filters at various frequency bands with double-layered feature selection and evaluates it on published and self-acquired datasets. Electroencephalography (EEG) data are preprocessed and decomposed into multiple frequency sub-bands, whose features are then extracted, calculated, and ranked based on Fisher’s ratio and minimum-redundancy-maximum-relevance (mRmR) algorithm. Informative filter banks are chosen for optimal classification by linear discriminative analysis (LDA). The results of the study, firstly, show that the proposed method is comparable to other conventional methods through accuracy and F1-score. The study also found that hand vs. feet classification is more discriminable than left vs. right hand (4–10% difference). Lastly, the performance of the filter banks common spatial pattern (FBCSP, without feature selection) algorithm is found to be significantly lower (p = 0.0029, p = 0.0015, and p = 0.0008) compared to that of the proposed method when applied to small-sized data.