Psychological studies have shown that Micro Gestures (MG) are closely linked to human emotions. MG-based emotion understanding has attracted much attention because it allows for emotion understanding through nonverbal body gestures without relying on identity information (e.g., facial and electrocardiogram data). Therefore, it is essential to recognize MG effectively for advanced emotion understanding. However, existing Micro Gesture Recognition (MGR) methods utilize only a single modality (e.g., RGB or skeleton) while overlooking crucial textual information. In this letter, we propose a simple but effective visual-text contrastive learning solution that utilizes text information for MGR. In addition, instead of using handcrafted prompts for visual-text contrastive learning, we propose a novel module called Adaptive prompting to generate context-aware prompts. The experimental results show that the proposed method achieves state-of-the-art performance on two public datasets. Furthermore, based on an empirical study utilizing the results of MGR for emotion understanding, we demonstrate that using the textual results of MGR significantly improves performance by 6%+ compared to directly using video as input.
In this work, a cascade classifier is trained to detect multi-view face samples. Comparing with most of face detection system which use different classifier to classify frontal face and profile face, our system has advantage in detection speed. The proposed face detector extracts the Haar-like feature from the training samples and train a cascade classifier by using Adaboost learing algorithm. Different from the existing algorithms, our detection system only contains a cascade classifier model. Our preliminary experiments demonstrate that our cascade classifier can achieve similiar accuracy and 60% higher speed detection than the multi-view face detection system which consist of two sparate cascade classifiers.
Micro-gesture is an imperceptible non-verbal behaviour characterised by low-intensity movement. However, its low-intensity and short-duration nature pose challenges for traditional action recognition models. To address this, we propose micro-gesture Mamba-inspired linear attention (MGMILA), a motion-aware framework integrating Mamba-inspired linear attention (MILA), a linear complexity model optimized for video-based micro-gesture recognition. Additionally, we design motion extraction module variants, motion as layer (MAL), motion as content (MAC), and motion as gate (MAG) to enhance spatiotemporal motion localization. Furthermore, we introduce human segmentation mask prediction as an auxiliary task to guide the network in attending to human-related regions, thereby improving its motion perception and recognition capability. Experiments on iMiGUE, spontaneous micro gesture (SMG), and MA-52 demonstrate state-of-the-art (SOTA) performance, validating the effectiveness of our approach.