Facial expressions of emotion provide relevant cues for understanding social interactions and the affective processes involved in emotion perception. Virtual human faces are useful for conducting controlled experiments. However, little is known regarding the possible differences between physiological responses elicited by virtual versus real human facial expressions. The aim of the current study was to determine if virtual and real emotional faces elicit the same rapid facial reactions for the perception of facial expressions of joy, anger, and sadness. Facial electromyography (corrugator supercilii, zygomaticus major, and depressor anguli) was recorded in 30 participants during the presentation of dynamic or static and virtual or real faces. For the perception of dynamic facial expressions of joy and anger, analyses of electromyography data revealed that rapid facial reactions were stronger when participants were presented with real faces compared with virtual faces. These results suggest that the processes underlying the perception of virtual versus real emotional faces might differ.
30 September 2014 in International Journal of Human–Computer Interaction
Behavior change is one of the most important goals in psychotherapy. This study focuses on Motivational Interviewing (MI), which is collaborative communication aimed at eliciting the client’s own reasons for behavior change. To investigate the effectiveness of facial information in modeling MI, we collected an MI encounter corpus with speech and video data in the nutrition and fitness domains and annotated client utterances using the Manual for the Motivational Interviewing Skill Code (MISC). By analyzing client answers to the questions after the session, we found that clients who expressed more Change Talk were more motivated to change their behavior than those who expressed less Change Talk. We then proposed RNN-based multimodal models to detect Change Talk by setting a 2-class classification task: "Change Talk" and "not Change Talk." Our experiment showed that the best performing model was a multimodal BiLSTM model that fused language and client facial information. We also found that fusing language and facial information as context achieved better performance than the unimodal and no-context models. Moreover, we discuss the label imbalance problem and conduct an additional analysis using turns as a unit of analysis. As a result, our best model reached F1-score of 0.65 for Change Talk detection.
In this paper, we describe the elaboration and the validation of a body and face database 1 , of 96 videos of 1 to 2 seconds of duration, expressing 4 emotions (i.e., anger, happiness, fear, and sadness) elicited through 4 platforms of increased visual complexity and level of embodiment. The final aim of this database is to develop an individualized training program designed for individuals suffering of autism in order to help them recognize various emotions on different test platforms: two robots, a virtual agent, and a human. Before assessing the recognition capabilities of individuals with ASD, we validated our video database on typically developed individuals (TD). Moreover, we also looked at the relationship between the recognition rate and their personality traits (extroverted (EX) vs. introverted (IN)). We found that the personality of our TD participants did not lead to a different recognition behavior. However, introverted individuals better recognized emotions from less visually complex characters than extroverted individuals.