Deep learning (DL) models have been widely studied in the field of micro-expression recognition (MER). However, micro-expressions (MEs) suffer from small number of samples and difficulty in extracting subtle and transient features, resulting in limited improvement in the recognition performance. In addition, DL models are prone to overfitting problems and are difficult to extract discriminative features of facial actions from ME images or sequences. To address these issues, we propose a MER method that combines a nonlinear deep subspace network and optical flow features. Firstly, facial motion features are captured using optical flow computation, and then the optical flow features are input into a kernel principal component analysis network (KPCANet) to further learn deeper spatio-temporal features. Finally, a linear support vector machine (SVM) is used for ME classification. Experiments conducted on four public spontaneous ME datasets including SMIC, CASME, CASME II and SAMM validate the effectiveness of the proposed method. The experimental results demonstrate that the proposed method achieves even better recognition performance compared to existing state-of-the-art MER methods.

A principal component analysis network (PCANet), as one of the representative deep subspace learning networks, utilizes principal component analysis (PCA) to learn filters that represent the dominant structural features of objects. However, the filters used in PCANet are linear combinations of all the original variables and contain complex and redundant principal components, which hinders the interpretability of the results. To address this problem, we introduce sparse constraints into a subspace learning network and propose three sparse bi-directional two-dimensional PCANet algorithms, including sparse row 2D${}^2$PCANet (SR2D${}^2$PCANet), sparse column 2D${}^2$PCANet (SC2D${}^2$PCANet), and sparse row–column 2D${}^2$PCANet (SRC2D${}^2$PCANet). These algorithms perform sparse operations on the projection matrices in the row, column, and row–column direction, respectively. Sparsity is achieved by utilizing the elastic net to shrink the loads of the non-primary elements in the principal components to zero and to reduce the redundancy in the projection matrices, thus improving the learning efficiency of the networks. Finally, a variety of experimental results on ORL, COIL-100, NEC, and AR datasets demonstrate that the proposed algorithms learn filters with more discriminative information and outperform other subspace learning networks and traditional deep learning networks in terms of classification and run-time performance, especially for less sample learning.