Humanoid social robots have an increasingly prominent place in today’s world. Their acceptance in social and emotional human–robot interaction (HRI) scenarios depends on their ability to convey well recognized and believable emotional expressions to their human users. In this article, we incorporate recent findings from psychology, neuroscience, human–computer interaction, and HRI, to examine how people recognize and respond to emotions displayed by the body and voice of humanoid robots, with a particular emphasis on the effects of incongruence. In a social HRI laboratory experiment, we investigated contextual incongruence (i.e., the conflict situation where a robot’s reaction is incongrous with the socio-emotional context of the interaction) and cross-modal incongruence (i.e., the conflict situation where an observer receives incongruous emotional information across the auditory (vocal prosody) and visual (whole-body expressions) modalities). Results showed that both contextual incongruence and cross-modal incongruence confused observers and decreased the likelihood that they accurately recognized the emotional expressions of the robot. This, in turn, gives the impression that the robot is unintelligent or unable to express “empathic” behaviour and leads to profoundly harmful effects on likability and believability. Our findings reinforce the need of proper design of emotional expressions for robots that use several channels to communicate their emotional states in a clear and effective way. We offer recommendations regarding design choices and discuss future research areas in the direction of multimodal HRI.
Advances in miniaturized computing, storage and communication resources for personal wearable electronics devices, as well as the availability of diverse sensors for state assessment enable the development of a wide variety of wearable body area network (BAN) systems for psychophysiological measurements. These systems pave the way for acquisition of quality data relevant for research studies, amongst others, on the autonomic nervous system (ANS) activation in emotion. We present a high-level overview of BAN and its features, and we review 173 publications that report research studies on emotion activation and particularly 15 BAN systems employed in these studies. We discuss each BAN in terms of its capacity for ambulatory, i.e., out of the laboratory, assessment of the ANS activation in emotion. Finally, we highlight the design challenges to be addressed to make BAN systems effective for a wide range of applications to support users wellbeing and overall Quality of Life improvement. This paper provides knowledge to those interested in (ambulatory) assessment of the ANS activation on the set of systems currently used in research, and it aims to highlight opportunities for scientists and practitioners in, amongst others, the affective computing domain, enabling them to reflect upon their BAN requirements and study designs.