Facial micro-expression is a brief involuntary facial movement and can reveal the genuine emotion that people try to conceal. Traditional methods of spontaneous micro-expression recognition rely excessively on sophisticated hand-crafted feature design and the recognition rate is not high enough for its practical application. In this paper, we proposed a Dual Temporal Scale Convolutional Neural Network (DTSCNN) for spontaneous micro-expressions recognition. The DTSCNN is a two-stream network. Different of stream of DTSCNN is used to adapt to different frame rate of micro-expression video clips. Each stream of DTSCNN consists of an independent shallow network for avoiding the overfitting problem. Meanwhile, we fed the networks with optical-flow sequences to ensure that the shallow networks can further acquire higher-level features. Experimental results on spontaneous micro-expression databases (CASME I/II) showed that our method can achieve a recognition rate almost 10% higher than what some state-of-the-art method can achieve.

This paper presents the methods used in our submission to 2018 Facial Micro-Expression Grand Challenge (MEGC). The object of the challenge is to recognize micro-expression in two provided databases, including holdout-database recognition and composite database recognition. Considering the small size of the databases, we follow a rout of transfer learning to implement convolutional neural network to recognize the micro-expression. ResNet10 pre-trained on ImageNet dataset was fine-tuned on macro-expression datasets with large size and then on the provided micro-expression datasets. Experimental results show that the method can achieve weighted average recall (WAR) of 0.561 and unweighted average recall (UAR) of 0.389 in Holdout-database Evaluation Task, and F1 Score of 0.64 in Composite Database Evaluation Task, which are much higher than what baseline methods (LBP-TOP, HOOF, HOG3D) can achieve.

Micro-expression is a subtle and involuntary facial expression that may reveal the hidden emotion of human beings. Spotting micro-expression means to locate the moment when the micro-expression happens, which is a primary step for micro-expression recognition. Previous work in micro-expression spotting focus on spotting micro-expression from short video, and with hand-crafted features. In this paper, we present a methodology for spotting micro-expression from long videos. Specifically, a new convolutional neural network named spotting micro-expression convolutional network was designed for extracting features from video clips, which is the first time that deep learning is used in micro-expression spotting. Then, a feature matrix processing method was proposed for spotting the apex frame from long video, which uses a sliding window and takes the characteristics of micro-expression into account to search the apex frame. Experimental results demonstrate that the proposed method can achieve a better performance than the existing state-of-art methods.

The detection of stress at early stages is beneficial to both individuals and communities. However, traditional stress detection methods that use physiological signals are contact-based and require sensors to be in contact with test subjects for measurement. In this paper, we present a method to detect psychological stress in a non-contact manner using a human physiological response. In particular, we utilize a hyperspectral imaging (HSI) technique to extract the tissue oxygen saturation (StO2) value as a physiological feature for stress detection. Our experimental results indicate that this new feature may be independent from perspiration and ambient temperature. Trier Social Stress Tests (TSSTs) on 21 volunteers demonstrated a significant difference $p\< 0.005$ and a large practical discrimination (d 1/4 1.37) between normalized baseline and stress StO2 levels. The accuracy for stress recognition from baseline using a binary classifier was 76.19 and 88.1 percent for the automatic and manual selections of the classifier threshold, respectively. These results suggest that the StO2 level could serve as a new modality to recognize stress at standoff distances.

A micro-expression is defined as an uncontrollable muscular movement shown on the face of humans when one is trying to conceal or repress his true emotions. Many researchers have applied the deep learning framework to micro-expression recognition in recent years. However, few have introduced the human visual attention mechanism to micro-expression recognition. In this study, we propose a three-dimensional (3D) spatiotemporal convolutional neural network with the convolutional block attention module (CBAM) for micro-expression recognition. First image sequences were input to a medium-sized convolutional neural network (CNN) to extract visual features. Afterwards, it learned to allocate the feature weights in an adaptive manner with the help of a convolutional block attention module. The method was testified in spontaneous micro-expression databases (Chinese Academy of Sciences Micro-expression II (CASME II), Spontaneous Micro-expression Database (SMIC)). The experimental results show that the 3D CNN with convolutional block attention module outperformed other algorithms in micro-expression recognition.

Near-infrared (NIR) face recognition has attracted increasing attention because of its advantage of illumination invariance. However, traditional face recognition methods based on NIR are designed for and tested in cooperative-user applications. In this paper, we present a convolutional neural network (CNN) for NIR face recognition (specifically face identification) in non-cooperative-user applications. The proposed NIRFaceNet is modified from GoogLeNet, but has a more compact structure designed specifically for the Chinese Academy of Sciences Institute of Automation (CASIA) NIR database and can achieve higher identification rates with less training time and less processing time. The experimental results demonstrate that NIRFaceNet has an overall advantage compared to other methods in the NIR face recognition domain when image blur and noise are present. The performance suggests that the proposed NIRFaceNet method may be more suitable for non-cooperative-user applications.