Semi-autonomous vehicles (AVs) enable drivers to engage in non-driving tasks but require them to be ready to take control during critical situations. This “out-of-the-loop” problem demands a quick transition to active information processing, raising safety concerns and anxiety. Multimodal signals in AVs aim to deliver take-over requests and facilitate driver–vehicle cooperation. However, the effectiveness of auditory, visual, or combined signals in improving situational awareness and reaction time for safe maneuvering remains unclear. This study investigates how signal modalities affect drivers’ behavior using virtual reality (VR). We measured drivers’ reaction times from signal onset to take-over response and gaze dwell time for situational awareness across twelve critical events. Furthermore, we assessed self-reported anxiety and trust levels using the Autonomous Vehicle Acceptance Model questionnaire. The results showed that visual signals significantly reduced reaction times, whereas auditory signals did not. Additionally, any warning signal, together with seeing driving hazards, increased successful maneuvering. The analysis of gaze dwell time on driving hazards revealed that audio and visual signals improved situational awareness. Lastly, warning signals reduced anxiety and increased trust. These results highlight the distinct effectiveness of signal modalities in improving driver reaction times, situational awareness, and perceived safety, mitigating the “out-of-the-loop” problem and fostering human–vehicle cooperation.

Autonomous vehicles represent a significant development in our society, and their acceptance will largely depend on trust. This study investigates strategies to increase trust and acceptance by making the cars’ decisions. For this purpose, we created a virtual reality (VR) experiment with a self-explaining autonomous car, providing participants with verbal cues about crucial traffic decisions. First, we investigated attitudes toward self-driving cars among 7850 participants using a simplified version of the Technology Acceptance Model (TAM) questionnaire. Results revealed that female participants are less accepting than male participants, and that there is a general decline among all genders. Otherwise in general, a self-explaining car has a positive impact on trust and perceived usefulness. Surprisingly, it adversely affected the intention to use and perceived ease of use. This entails dissociation of trust from the other items of the questionnaire. Second, we analyzed behavioral of 26 750 participants to investigate the effect of self-explaining systems on head movements during the VR drive. We observed significant differences in head movements during the critical events and the baseline periods of the drive between the three driving conditions. Additionally, we demonstrated positive correlations between head movement parameters and the TAM scores, where trust showed the lowest correlation. This provides further evidence of the dissociation of trust from other TAM items. These findings illustrate the benefits of combining subjective questionnaire data with objective behavioral data. Overall, the outcomes indicate a partial dissociation of self-reported trust from intention to use and objective behavioral data.

The brain’s ability to transform complex, high-dimensional sensory and motor inputs into coordinated, goal-directed behavior remains a central challenge in neuroscience research. Current research suggests that behavior is generated through patterns within a low-dimensional structure. What visuomotor patterns underlie complex, unconstrained behaviors such as driving, and how stereotypical they are, remain poorly understood. Driving involves complex perception-action interactions, which we hypothesize are specific to the driver’s role and adapt to task demands. Here, we unfold these dynamics using an immersive virtual drive that recreates ten hazardous on-road events. We collected eye, head, and vehicle movements from participants assigned to either a manual or an automated driving condition. From the behavioral movements of 284 drivers, we constructed a common time-resolved behavioral state space based on collective variance, computed via principal component analysis across participants. The obtained low-dimensional representation thus captures generalized visuomotor coordination strategies. We then characterized the evolving low-dimensional structure with cosine similarity and various measures of effective dimensionality. We tested whether leading components contained condition-specific information using discriminant analysis. Throughout the entire drive, few components accounted for most of the variance in behavior, and effective dimensionality decreased reliably around critical events, indicating tighter coordination between perceptual and motor variables. During periods of low effective dimensionality, the contributions of eye, head, steering, and vehicle heading reorganized toward leading dimensions in a task-dependent manner. Across events, between-subject similarity was partially stable and stronger among manual drivers. This reorganization also made behaviors more distinct across driving conditions, enabling better classification of driving modes using only the first two components. These results show that naturalistic driving is supported by shared low-dimensional visuomotor coordination strategies that are flexibly reshaped by contextual demands, and provide a principled behavioral framework for linking neural manifolds, human driver models, and the design of adaptive automated vehicles.

The brain’s ability to transform complex, high-dimensional sensory and motor inputs into coordinated, goal-directed behavior remains a central challenge in neuroscience research. Current research suggests that behavior is generated through patterns within a low-dimensional structure. What visuomotor patterns underlie complex, unconstrained behaviors such as driving, and how stereotypical they are, remain poorly understood. Driving involves complex perception-action interactions, which we hypothesize are specific to the driver’s role and adapt to task demands. Here, we unfold these dynamics using an immersive virtual drive that recreates ten hazardous on-road events. We collected eye, head, and vehicle movements from participants assigned to either a manual or an autonomous driving condition. From the behavioral movements of 284 drivers, we constructed a common time-resolved behavioral state space based on collective variance, computed via principal component analysis across participants. The obtained low-dimensional manifold thus represents generalized visuomotor coordination strategies. We then characterized the evolving low-dimensional structure with cosine similarity and various entropy-based measures of effective dimensionality. We tested whether early components contained condition-specific information using discriminant analysis. Across the entire drive, a few components accounted for most of the variance in behavior, and effective dimensionality decreased reliably around critical events, indicating tighter coordination between perceptual and motor variables. During periods of low effective dimensionality, the contributions of eye, head, steering, and vehicle heading reorganized toward early dimensions in a task-dependent manner. As a result, behaviors became more distinct across driving conditions, enabling better classification of driving modes using only the first two components. These results show that naturalistic driving is supported by shared low-dimensional visuomotor coordination strategies that are flexibly reshaped by contextual demands, and provide a principled behavioral framework for linking neural manifolds, human driver models, and the design of adaptive autonomous vehicles. Author summary: Human behavior is complex. Driving, for instance, means constantly turning what we see into the right movements of our head, hands, and feet. Research using constrained repeated actions suggests that these seemingly complex behaviors are highly coordinated and can be described by a relatively small number of variables. We want to know whether these aspects also hold during natural, unconstrained tasks. We asked people to complete a virtual drive comprising ten sudden hazards, such as animals and pedestrians on the road. Some participants drove the car, while others sat in a self-driving car. We recorded where people looked, how they moved their heads, and steered the vehicle. Then, we used statistical methods to reduce the data’s dimensionality and looked for patterns that could explain all these signals together. We found that a small number of patterns were enough to describe behavior. These patterns became even simpler and more linked around hazards, differentiating autonomous from manual driving behavior. Our results suggest that even complex driving behavior follows a few basic coordination strategies that become more pronounced around specific situations. These basic dimensions contain important details that are unique to certain groups, ultimately helping us better understand and compare human behavior.

Semi-autonomous vehicles (AVs) enable drivers to engage in non-driving tasks but require them to be ready to take control during critical situations. This "out-of-the-loop" problem demands a quick transition to active information processing, raising safety concerns and anxiety. Multimodal signals in AVs aim to deliver take-over requests and facilitate driver-vehicle cooperation. However, the effectiveness of auditory, visual, or combined signals in improving situational awareness and reaction time for safe maneuvering remains unclear. This study investigates how signal modalities affect driver’s behavior using virtual reality (VR). We measured drivers’ reaction times from signal onset to take-over response and gaze dwell time for situational awareness across twelve critical events. Furthermore, we assessed self-reported anxiety and trust levels using the Autonomous Vehicle Acceptance Model questionnaire. Results showed visual signals significantly reduced reaction times, whereas auditory signals did not. Additionally, any warning signal, together with seeing driving hazards, increased successful maneuvering. Analysis of gaze dwell time on driving hazards revealed that audio and visual signals improved situational awareness. Lastly, warning signals reduced anxiety and increased trust. These results highlight the distinct effectiveness of signal modalities in improving driver reaction times, situational awareness, and perceived safety, mitigating the “out-of-the-loop” problem and fostering human-vehicle cooperation.