Autonomous vehicle technology increasingly allows drivers to turn their primary attention to secondary tasks (e.g., eating or working). This dramatic behavior change thus requires new input modalities to support driver–vehicle interaction, which must match the driver’s in-vehicle activities and the interaction situation. Prior studies that addressed this question did not consider how acceptance for inputs was affected by the physical and cognitive levels experienced by drivers engaged in Non-driving Related Tasks (NDRTs) or how their acceptance varies according to the interaction situation. This study investigates naturalistic interactions with a fully autonomous vehicle system in different intervention scenarios while drivers perform NDRTs. We presented an online methodology to 360 participants showing four NDRTs with different physical and cognitive engagement levels, and tested the six most common intervention scenarios (24 cases). Participants evaluated our proposed seven natural input interactions for each case: touch, voice, hand gesture, and their combinations. Results show that NDRTs influence the driver’s input interaction more than intervention scenario categories. In contrast, variation of physical load has more influence on input selection than variation of cognitive load. We also present a decision-making model of driver preferences to determine the most natural inputs and help User Experience designers better meet drivers’ needs.

Engagement plays a key role in improving the cognitive and motor development of children with autism spectrum disorder (ASD). Sensing and recognizing their engagement is crucial before sustaining and improving the engagement. Engaging technologies involving interactive and multi-sensory stimuli have improved engagement and alleviated hyperactive and stereotyped behaviors. However, due to the scarcity of data on engagement recognition for children with ASD, limited access to and small pools of participants, and the prohibitive application requirements such as robots, high cost, and expertise, implementation in real world is challenging. However, serious games have the potential to overcome those drawbacks and are suitable for practical use in the field. This study proposes Engagnition, a dataset for engagement recognition of children with ASD (N = 57) using a serious game, “Defeat the Monster,” based on enhancing recognition and classification skills. The dataset consists of physiological and behavioral responses, annotated by experts. For technical validation, we report the distributions of engagement and intervention, and the signal-to-noise ratio of physiological signals.

Autonomous vehicles (AVs) enable drivers to devote their primary attention to non-driving-related tasks (NDRTs). Consequently, AVs must provide intelligibility services appropriate to drivers’ in-situ states and in-car activities to ensure driver safety, and accounting for the type of NDRT being performed can result in higher intelligibility. We discovered that sleeping is drivers’ most preferred NDRT, and this could also result in a critical scenario when a take-over request (TOR) occurs. In this study, we designed TOR situations where drivers are woken from sleep in a high-fidelity AV simulator with motion systems, aiming to examine how drivers react to a TOR provided with our experimental conditions. We investigated how driving performance, perceived task workload, AV acceptance, and physiological responses in a TOR vary according to two factors: (1) feedforward timings and (2) presentation modalities. The results showed that when awakened by a TOR alert delivered >10 s prior to an event, drivers were more focused on the driving context and were unlikely to be influenced by TOR modality, whereas TOR alerts delivered <5 s prior needed a visual accompaniment to quickly inform drivers of on-road situations. This study furthers understanding of how a driver’s cognitive and physical demands interact with TOR situations at the moment of waking from sleep and designs effective interventions for intelligibility services to best comply with safety and driver experience in AVs.

Autonomous vehicles (AV) increasingly allow drivers to engage in secondary tasks such as eating or working on a laptop and thus require easy and reliable interaction inputs to facilitate communication between the driver and the vehicle. However, drivers report feeling less in control when driving is no longer the primary task, which suggests that novel approaches for assessing satisfaction regarding AV decision-making are needed. Therefore, we propose an immediate experience sampling method (IESM) that learns driver preferences for AV actions. We also suggest gaze-head input (G-HI) as a novel input in an AV. G-HI provides a hands-free, remote, and intuitive input modality that allows drivers to interact with the AV while continuing to engage in non-driving related tasks. We compare G-HI with voice and touch inputs via IESM for two simulated driving scenarios. Our results report the differences among the three inputs in terms of system usability, reaction time, and perceived workload. It also reveals that G-HI is a promising candidate for AV input interaction, which could replace voice or touch inputs where those inputs could not be utilized. Variation in driver satisfaction and expectations for AV actions confirms the effectiveness of using IESM to increase drivers’ sense of control.

Active engagement where children with autism spectrum disorder (ASD) are involved (e.g., educational and social activities) plays a crucial role in enhancing their cognitive, motor, and social development. This offers opportunities to enhance overall development, including learning abilities, physical coordination, and social interactions. Indirect methods, leveraging sensors and artificial intelligence (AI), have exhibited potential for enhancing engagement predictions but have been primarily focused within specific fields, resulting in a gap that leads to limited generalizability of ASD studies. This gap, due to small ASD sample sizes, presents a significant challenge as the annual ASD population increases, highlighting the need for practical and applicable research solutions, especially for general learning. In this work, we conducted expert interviews to explore the potential application areas of AI-infused systems that provide three levels of engagement status for children with ASD, ranging from "not engaged and out of control" to "highly engaged." Interviews with special educators revealed five key application areas for AI-driven engagement recognition: social skills training, stereotyped behavior modification, support for leisure activities, effective tutoring, and independent daily living skills. These findings highlight the potential of adaptive AI interventions in improving educational and daily outcomes, advocating for expanded applications for children with ASD.