In the Human-Machine Interactions (HMI) landscape, understanding user emotions is pivotal for elevating user experiences. This paper explores Facial Expression Recognition (FER) within HMI, employing a distinctive multimodal approach that integrates visual and auditory information. Recognizing the dynamic nature of HMI, where situations evolve, this study emphasizes continuous emotion analysis. This work assesses various fusion strategies that involve the addition to the main network of different architectures, such as autoencoders (AE) or an Embracement module, to combine the information of multiple biometric cues. In addition to the multimodal approach, this paper introduces a new architecture that prioritizes temporal dynamics by incorporating Long Short-Term Memory (LSTM) networks. The final proposal, which integrates different multimodal approaches with the temporal focus capabilities of the LSTM architecture, was tested across three public datasets: RAVDESS, SAVEE, and CREMA-D. It showcased state-of-the-art accuracy of 88.11%, 86.75%, and 80.27%, respectively, and outperformed other existing approaches.
Soft bimetrics has become a trending research topic over the past decade. In last years, the increase of new technologies such as the wearable camera devices has introduced a new challenge into the gender classification problem. In this sense, the ability to classify the gender not by an image but by the 2D estimated skeleton points is considered in this paper. Our experiments show that the human gender can be classified just considering the pose information provided by the body pose information. The proposed method have shown a remarkable performance on a dataset where subjects and camera are in movement.