Over recent years, robots are increasingly being employed in several aspects of modern society. Among others, social robots have the potential to benefit education, healthcare, and tourism. To achieve this purpose, robots should be able to engage humans, recognize users’ emotions, and to some extent properly react and "behave" in a natural interaction. Most robotics applications primarily use visual information for emotion recognition, which is often based on facial expressions. However, the display of emotional states through facial expression is inherently a voluntary controlled process that is typical of human–human interaction. In fact, humans have not yet learned to use this channel when communicating with a robotic technology. Hence, there is an urgent need to exploit emotion information channels not directly controlled by humans, such as those that can be ascribed to physiological modulations. Thermal infrared imaging-based affective computing has the potential to be the solution to such an issue. It is a validated technology that allows the non-obtrusive monitoring of physiological parameters and from which it might be possible to infer affective states. This review is aimed to outline the advantages and the current research challenges of thermal imaging-based affective computing for human–robot interaction.

Infrared thermal imaging (IRI) is a contact-less technology able to monitor human skin temperature for biomedical applications and in real-life contexts. Its capacity to detect fever was exploited for mass screening during past epidemic emergencies as well as for the current COVID-19 pandemic. However, the only assessment of fever may not be selective for the Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) infection. Hence, novel approaches for IRI data analysis have been investigated. The present review aims to describe how IRI have been employed during the last epidemics, highlighting the potentialities and the limitations of this technology to contain the contagions. Specifically, the methods employed for automatic face recognition and fever assessment and IRI’s performances in mass screening at airports and hospitals are reviewed. Moreover, an overview of novel machine learning methods for IRI data analysis, aimed to identify respiratory diseases, is provided. In addition, IRI-based smart technologies developed to support the healthcare during the COVID-19 pandemic are described. Finally, relevant guidelines to fully exploit IRI for COVID-19 identification are defined, to improve the effectiveness of IRI in the detection of the SARS-CoV-2 infection.

Traffic accidents determine a large number of injuries, sometimes fatal, every year. Among other factors affecting a driver’s performance, an important role is played by stress which can decrease decision-making capabilities and situational awareness. In this perspective, it would be beneficial to develop a non-invasive driver stress monitoring system able to recognize the driver’s altered state. In this study, a contactless procedure for drivers’ stress state assessment by means of thermal infrared imaging was investigated. Thermal imaging was acquired during an experiment on a driving simulator, and thermal features of stress were investigated with comparison to a gold-standard metric (i.e., the stress index, SI) extracted from contact electrocardiography (ECG). A data-driven multivariate machine learning approach based on a non-linear support vector regression (SVR) was employed to estimate the SI through thermal features extracted from facial regions of interest (i.e., nose tip, nostrils, glabella). The predicted SI showed a good correlation with the real SI (r = 0.61, p = ~0). A two-level classification of the stress state (STRESS, SI ≥ 150, versus NO STRESS, SI < 150) was then performed based on the predicted SI. The ROC analysis showed a good classification performance with an AUC of 0.80, a sensitivity of 77%, and a specificity of 78%.

Mental workload (MW) represents the amount of brain resources required to perform concurrent tasks. The evaluation of MW is of paramount importance for Advanced Driver-Assistance Systems, given its correlation with traffic accidents risk. In the present research, two cognitive tests (Digit Span Test—DST and Ray Auditory Verbal Learning Test—RAVLT) were administered to participants while driving in a simulated environment. The tests were chosen to investigate the drivers’ response to predefined levels of cognitive load to categorize the classes of MW. Infrared (IR) thermal imaging concurrently with heart rate variability (HRV) were used to obtain features related to the psychophysiology of the subjects, in order to feed machine learning (ML) classifiers. Six categories of models have been compared basing on unimodal IR/unimodal HRV/multimodal IR + HRV features. The best classifier performances were reached by the multimodal IR + HRV features-based classifiers (DST: accuracy = 73.1%, sensitivity = 0.71, specificity = 0.69; RAVLT: accuracy = 75.0%, average sensitivity = 0.75, average specificity = 0.87). The unimodal IR features based classifiers revealed high performances as well (DST: accuracy = 73.1%, sensitivity = 0.73, specificity = 0.73; RAVLT: accuracy = 71.1%, average sensitivity = 0.71, average specificity = 0.85). These results demonstrated the possibility to assess drivers’ MW levels with high accuracy, also using a completely non-contact and non-invasive technique alone, representing a key advancement with respect to the state of the art in traffic accident prevention.

Cerebral palsy (CP) is a non-progressive neurologic condition that causes gait limitations, spasticity, and impaired balance and coordination. Robotic-assisted gait training (RAGT) has become a common rehabilitation tool employed to improve the gait pattern of people with neurological impairments. However, few studies have demonstrated the effectiveness of RAGT in children with CP and its neurological effects through portable neuroimaging techniques, such as functional near-infrared spectroscopy (fNIRS). The aim of the study is to evaluate the neurophysiological processes elicited by RAGT in children with CP through fNIRS, which was acquired during three sessions in one month. The repeated measure ANOVA was applied to the β-values delivered by the General Linear Model (GLM) analysis used for fNIRS data analysis, showing significant differences in the activation of both prefrontal cortex (F (1.652, 6.606) = 7.638; p = 0.022), and sensorimotor cortex (F (1.294, 5.175) = 11.92; p = 0.014) during the different RAGT sessions. In addition, a cross-validated Machine Learning (ML) framework was implemented to estimate the gross motor function measure (GMFM-88) from the GLM β-values, obtaining an estimation with a correlation coefficient r = 0.78. This approach can be used to tailor clinical treatment to each child, improving the effectiveness of rehabilitation for children with CP.

Social Robots (SRs) are substantially becoming part of modern society, given their frequent use in many areas of application including education, communication, assistance, and entertainment. The main challenge in human–robot interaction is in achieving human-like and affective interaction between the two groups. This study is aimed at endowing SRs with the capability of assessing the emotional state of the interlocutor, by analyzing his/her psychophysiological signals. The methodology is focused on remote evaluations of the subject’s peripheral neuro-vegetative activity by means of thermal infrared imaging. The approach was developed and tested for a particularly challenging use case: the interaction between children and a commercial educational robot, Mio Amico Robot, produced by LiscianiGiochi©. The emotional state classified from the thermal signal analysis was compared to the emotional state recognized by a facial action coding system. The proposed approach was reliable and accurate and favored a personalized and improved interaction of children with SRs.