Over recent years, robots are increasingly being employed in several aspects of modern society. Among others, social robots have the potential to benefit education, healthcare, and tourism. To achieve this purpose, robots should be able to engage humans, recognize users’ emotions, and to some extent properly react and "behave" in a natural interaction. Most robotics applications primarily use visual information for emotion recognition, which is often based on facial expressions. However, the display of emotional states through facial expression is inherently a voluntary controlled process that is typical of human–human interaction. In fact, humans have not yet learned to use this channel when communicating with a robotic technology. Hence, there is an urgent need to exploit emotion information channels not directly controlled by humans, such as those that can be ascribed to physiological modulations. Thermal infrared imaging-based affective computing has the potential to be the solution to such an issue. It is a validated technology that allows the non-obtrusive monitoring of physiological parameters and from which it might be possible to infer affective states. This review is aimed to outline the advantages and the current research challenges of thermal imaging-based affective computing for human–robot interaction.

Infrared thermal imaging (IRI) is a contact-less technology able to monitor human skin temperature for biomedical applications and in real-life contexts. Its capacity to detect fever was exploited for mass screening during past epidemic emergencies as well as for the current COVID-19 pandemic. However, the only assessment of fever may not be selective for the Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) infection. Hence, novel approaches for IRI data analysis have been investigated. The present review aims to describe how IRI have been employed during the last epidemics, highlighting the potentialities and the limitations of this technology to contain the contagions. Specifically, the methods employed for automatic face recognition and fever assessment and IRI’s performances in mass screening at airports and hospitals are reviewed. Moreover, an overview of novel machine learning methods for IRI data analysis, aimed to identify respiratory diseases, is provided. In addition, IRI-based smart technologies developed to support the healthcare during the COVID-19 pandemic are described. Finally, relevant guidelines to fully exploit IRI for COVID-19 identification are defined, to improve the effectiveness of IRI in the detection of the SARS-CoV-2 infection.

Traffic accidents determine a large number of injuries, sometimes fatal, every year. Among other factors affecting a driver’s performance, an important role is played by stress which can decrease decision-making capabilities and situational awareness. In this perspective, it would be beneficial to develop a non-invasive driver stress monitoring system able to recognize the driver’s altered state. In this study, a contactless procedure for drivers’ stress state assessment by means of thermal infrared imaging was investigated. Thermal imaging was acquired during an experiment on a driving simulator, and thermal features of stress were investigated with comparison to a gold-standard metric (i.e., the stress index, SI) extracted from contact electrocardiography (ECG). A data-driven multivariate machine learning approach based on a non-linear support vector regression (SVR) was employed to estimate the SI through thermal features extracted from facial regions of interest (i.e., nose tip, nostrils, glabella). The predicted SI showed a good correlation with the real SI (r = 0.61, p = ~0). A two-level classification of the stress state (STRESS, SI ≥ 150, versus NO STRESS, SI < 150) was then performed based on the predicted SI. The ROC analysis showed a good classification performance with an AUC of 0.80, a sensitivity of 77%, and a specificity of 78%.

Mental workload (MW) represents the amount of brain resources required to perform concurrent tasks. The evaluation of MW is of paramount importance for Advanced Driver-Assistance Systems, given its correlation with traffic accidents risk. In the present research, two cognitive tests (Digit Span Test—DST and Ray Auditory Verbal Learning Test—RAVLT) were administered to participants while driving in a simulated environment. The tests were chosen to investigate the drivers’ response to predefined levels of cognitive load to categorize the classes of MW. Infrared (IR) thermal imaging concurrently with heart rate variability (HRV) were used to obtain features related to the psychophysiology of the subjects, in order to feed machine learning (ML) classifiers. Six categories of models have been compared basing on unimodal IR/unimodal HRV/multimodal IR + HRV features. The best classifier performances were reached by the multimodal IR + HRV features-based classifiers (DST: accuracy = 73.1%, sensitivity = 0.71, specificity = 0.69; RAVLT: accuracy = 75.0%, average sensitivity = 0.75, average specificity = 0.87). The unimodal IR features based classifiers revealed high performances as well (DST: accuracy = 73.1%, sensitivity = 0.73, specificity = 0.73; RAVLT: accuracy = 71.1%, average sensitivity = 0.71, average specificity = 0.85). These results demonstrated the possibility to assess drivers’ MW levels with high accuracy, also using a completely non-contact and non-invasive technique alone, representing a key advancement with respect to the state of the art in traffic accident prevention.

Parkinson’s disease (PD) is a neurodegenerative disorder characterized by a range of motor and non-motor symptoms. One of the notable non-motor symptoms of PD is the presence of vocal disorders, attributed to the underlying pathophysiological changes in the neural control of the laryngeal and vocal tract musculature. From this perspective, the integration of machine learning (ML) techniques in the analysis of speech signals has significantly contributed to the detection and diagnosis of PD. Particularly, MEL Frequency Cepstral Coefficients (MFCCs) and Gammatone Frequency Cepstral Coefficients (GTCCs) are both feature extraction techniques commonly used in the field of speech and audio signal processing that could exhibit great potential for vocal disorder identification. This study presents a novel approach to the early detection of PD through ML applied to speech analysis, leveraging both MFCCs and GTCCs. The recordings contained in the Mobile Device Voice Recordings at King’s College London (MDVR-KCL) dataset were used. These recordings were collected from healthy individuals and PD patients while they read a passage and during a spontaneous conversation on the phone. Particularly, the speech data regarding the spontaneous dialogue task were processed through speaker diarization, a technique that partitions an audio stream into homogeneous segments according to speaker identity. The ML applied to MFCCS and GTCCs allowed us to classify PD patients with a test accuracy of 92.3%. This research further demonstrates the potential to employ mobile phones as a non-invasive, cost-effective tool for the early detection of PD, significantly improving patient prognosis and quality of life.

Cerebral palsy (CP) is a non-progressive neurologic condition that causes gait limitations, spasticity, and impaired balance and coordination. Robotic-assisted gait training (RAGT) has become a common rehabilitation tool employed to improve the gait pattern of people with neurological impairments. However, few studies have demonstrated the effectiveness of RAGT in children with CP and its neurological effects through portable neuroimaging techniques, such as functional near-infrared spectroscopy (fNIRS). The aim of the study is to evaluate the neurophysiological processes elicited by RAGT in children with CP through fNIRS, which was acquired during three sessions in one month. The repeated measure ANOVA was applied to the β-values delivered by the General Linear Model (GLM) analysis used for fNIRS data analysis, showing significant differences in the activation of both prefrontal cortex (F (1.652, 6.606) = 7.638; p = 0.022), and sensorimotor cortex (F (1.294, 5.175) = 11.92; p = 0.014) during the different RAGT sessions. In addition, a cross-validated Machine Learning (ML) framework was implemented to estimate the gross motor function measure (GMFM-88) from the GLM β-values, obtaining an estimation with a correlation coefficient r = 0.78. This approach can be used to tailor clinical treatment to each child, improving the effectiveness of rehabilitation for children with CP.