Children with autism face challenges in various skills (e.g., communication and social) and they exhibit challenging behaviours. These challenging behaviours represent a challenge to their families, therapists, and caregivers, especially during therapy sessions. In this study, we have investigated several machine learning techniques and data modalities acquired using wearable sensors from children with autism during their interactions with social robots and toys in their potential to detect challenging behaviours. Each child wore a wearable device that collected data. Video annotations of the sessions were used to identify the occurrence of challenging behaviours. Extracted time features (i.e., mean, standard deviation, min, and max) in conjunction with four machine learning techniques were considered to detect challenging behaviors. The heart rate variability (HRV) changes have also been investigated in this study. The XGBoost algorithm has achieved the best performance (i.e., an accuracy of 99%). Additionally, physiological features outperformed the kinetic ones, with the heart rate being the main contributing feature in the prediction performance. One HRV parameter (i.e., RMSSD) was found to correlate with the occurrence of challenging behaviours. This work highlights the importance of developing the tools and methods to detect challenging behaviors among children with autism during aided sessions with social robots.

Social robots have gained a lot of attention recently as they have been reported to be effective in supporting therapeutic services for children with autism. However, children with autism may exhibit a multitude of challenging behaviors that could be harmful to themselves and to others around them. Furthermore, social robots are meant to be companions and to elicit certain social behaviors. Hence, the presence of a social robot during the occurrence of challenging behaviors might increase any potential harm. In this paper, we identified harmful scenarios that might emanate between a child and a social robot due to the manifestation of challenging behaviors. We then quantified the harm levels based on severity indices for one of the challenging behaviors (i.e. throwing of objects). Our results showed that the overall harm levels based on the selected severity indices are relatively low compared to their respective thresholds. However, our investigation of harm due to throwing of a small social robot to the head revealed that it could potentially cause tissue injuries, subconcussive or even concussive events in extreme cases. The existence of such behaviors must be accounted for and considered when developing interactive social robots to be deployed for children with autism.

The quality of a companion robot’s reaction is important to make it acceptable to the users and to sustain interactions. Furthermore, the robot’s reaction can be used to train socially acceptable behaviors and to develop certain skills in both normally developing children and children with cognitive disabilities. In this study, we investigate the influence of reaction time in the emotional response of a robot when children display aggressive interactions toward it. Different interactions were considered, namely, pickup, shake, drop and throw. The robot produced responses as audible sounds, which were activated at three different reaction times, namely, 0.5 s, 1.0 s, and 1.5 s. The results for one of the tasks that involved shaking the robotic toys produced a significant difference between the timings tested. This could imply that producing a late response to an action (i.e. greater than 1.0 s) could negatively affect the children’s comprehension of the intended message. Furthermore, the response should be comprehensible to provide a clear message to the user. The results imply that the designers of companion robotic toys need to consider an appropriate timing and clear modality for their robots’ responses.

Feedback from sensory modalities is crucial for the precise grasping of tissues during minimally invasive robotic surgery. The aims of the study are to determine the influence of visual and haptic feedback on the detection of threshold forces and to evaluate the applicability of the sensory integration model to a surgical grasping task. A sensorized surgical grasper and a fingertip haptic force feedback device were used. Three types of stimuli were presented (i.e. visual-alone, haptic-alone, and bimodal visual and haptic stimuli). Threshold forces of 100 mN and 87.5 mN were detected for visual and haptic feedback, respectively. When bimodal feedback was provided, the participants detected a threshold force of 75 mN. The threshold force for the bimodal condition was 28.6% lower than the visual-alone feedback and 15.4% lower than the haptic-alone feedback stimuli. Our bimodal condition results showed that there was a 13.1% difference between the experimental result and the predicted value from the sensory integration model. The threshold force discrimination was strongly influenced by the haptic force feedback. It is likely that the tissue stiffness can be more intuitively perceived through the direct force stimulation of the fingertip than just by visual observation alone. Cues like small deformations or changes in the grasping angles of the surgical tool are more difficult to interpret visually as compared to the haptic modality.