Micro-expressions may occur in high-stake situations when people attempt to conceal or suppress their true feelings. Nowadays, intelligent micro-expression analysis has long been focused on videos captured under constrained laboratory conditions. This is due to the relatively small number of publicly available datasets. Moreover, micro-expression characteristics are subtle and brief, and thus very susceptible to interference from external factors and difficult to capture. In particular, head movement is unavoidable in unconstrained scenarios, making micro-expression spotting highly challenging. This paper proposes a simple yet effective method for avoiding the interference of head movement on micro-expression spotting in natural scenarios by considering three-dimensional space. In particular, based on the head pose, which can be mapped to two-dimensional vectors (translations and rotations) for representation, long and complex videos could be divided into short video segments that basically exclude head movement interference. Following that, segmented micro-expression spotting is realized based on an effective short-segment-based micro-expression spotting algorithm. Experimental results on in-the-wild databases demonstrate the effectiveness of our proposed method in avoiding head movement interference. Additionally, due to the simplicity of this method, it creates opportunities for spotting micro-expressions in real-world scenarios, possibly even in real-time. Furthermore, it helps alleviate the small sample size problem in micro-expression analysis by boosting the spotting performance in massive unlabeled videos.
Facial expression recognition plays an increasingly important role in daily life, and it is used in several areas of human–computer interaction, such as robotics, assisted driving, and intelligent tutoring systems. However, the current mainstream methods are based on the whole face, and do not consider the existence of expression asymmetry between the left and right half-face. Hence, the accuracy of facial expression recognition needs to be improved. In this paper, we propose a half-face self-attention weighted approach called HSAW. Using statistical analysis and computer vision techniques, we found that the left half-face contains richer expression features than the right half-face. Specifically, we employed a self-attention mechanism to assign different weights to the left and right halves of the face. These weights are combined with convolutional neural network features for improved facial expression recognition. Furthermore, to attack the presence of uncertain categories in the dataset, we introduce adaptive re-labeling module, which can improve the recognition accuracy. Extensive experiments conducted on the FER2013 and RAF datasets have verified the effectiveness of the proposed method, which utilizes fewer parameters.