In-vehicle human-machine interface (HMI) plays a significant role in accomplishing effective interactions between driving automation systems and drivers, especially during the transition of control. For this reason, different in-vehicle HMIs have been designed to convey additional semantic information from the driving automation systems to the drivers to realize safer, smoother, and better control transitions. This review summarizes and analyses 86 previously published studies that researched the effects of additional semantic information delivered through in-vehicle HMIs just before, during and right after takeover request (TOR). The additional semantic information mentioned in this review refer to the information beyond simple alerts to not only gain drivers’ attention but also additionally communicate contextual content and explanation to the drivers regarding its own purpose. In this review, the additional semantic information are categorized according to their purposes and effects into three aspects: mode awareness enhancement, situation awareness enhancement, and takeover maneuver assistance. The specificities and the corresponding concerns when applying additional semantic information to in-vehicle HMIs have been detailed analyzed throughout the entire article. Further suggestions are proposed for what should be carefully considered when adding additional information for better takeover. Prospects into future in-vehicle HMI possibilities are also raised that could be applied in both academic research and industry.

For highly automated vehicles, effective take-over performance measures are essential for establishing quantitative take-over models and exploring approaches to improve take-over performance. However, there is a lack of comprehensive take-over performance measures that suitably combine multiple objective metrics based on an average evaluation from human drivers. In this study, we proposed a human-centered comprehensive measure of take-over performance (HCMTP). There are four main building blocks for the HCMTP. First, we adopted sparse principal component analysis to identify the main aspects of take-over performance based on multiple original objective take-over performance metrics. Second, we developed a scale of take-over performance assessment to obtain drivers’ original subjective self-assessments of take-over performance. Third, we established nonlinear individual mapping functions to acquire different drivers’ evaluation criteria for take-over performance. Fourth, we proposed a relabeling algorithm to obtain drivers’ average evaluation of take-over performance. To verify the effectiveness of the HCMTP, we conducted a verification experiment involving 68 participants. The results indicate that the HCMTP is effective and able to reduce the interference of individual differences, stochasticity, and data imbalance. This study contributes to identifying the main aspects of take-over performance, systematically understanding how human drivers subjectively evaluate take-over performance, and evaluating drivers’ take-over performance comprehensively.

Traffic accidents due to fatigue account for a large proportion of road fatalities. Based on simulated driving experiments with drivers recruited from college students, this paper investigates the use of heart rate variability (HRV) features to detect driver fatigue while considering sex differences. Sex-independent and sex-specific differences in HRV features between alert and fatigued states derived from 2 min electrocardiogram (ECG) signals were determined. Then, decision trees were used for driver fatigue detection using the HRV features of either all subjects or those of only males or females. Nineteen, eighteen, and thirteen HRV features were significantly different (Mann–Whitney U test, p < 0.01) between the two mental states for all subjects, males, and females, respectively. The fatigue detection models for all subjects, males, and females achieved classification accuracies of 86.3%, 94.8%, and 92.0%, respectively. In conclusion, sex differences in HRV features between drivers’ mental states were found according to both the statistical analysis and classification results. By considering sex differences, precise HRV feature-based driver fatigue detection systems can be developed. Moreover, in contrast to conventional methods using HRV features from 5 min ECG signals, our method uses HRV features from 2 min ECG signals, thus enabling more rapid driver fatigue detection.

As human-machine collaborative driving systems, highly automated driving vehicles require human drivers to take over when take-over requests are triggered. Extensive studies have shown that drivers’ take-over performance is affected by their fatigue state, traffic conditions, and the take-over time budget (TB). However, there is still a paucity of a systematic understanding of how these factors affect take-over performance, which prevents the implementation of adaptive take-over systems. This study establishes a highly accurate take-over performance prediction model to systematically explore the effects of these factors on take-over performance and to propose an adaptive TB adjustment strategy for highly automated driving vehicles. First, we propose metrics to evaluate drivers’ fatigue states and the relative positions of surrounding traffic. Second, a generalized additive model is established to predict take-over performance and accurately evaluate the influence of the aforementioned factors on take-over performance. Based on the model, we propose an adaptive adjustment strategy of the TB for take-over systems and demonstrate its effectiveness by a verification experiment. This study contributes to understanding the influence of drivers’ passive fatigue states, the relative positions of surrounding traffic, and the TB on drivers’ take-over performance as well as to the development of adaptive take-over systems for highly automated vehicles.

Although latent hazard notification for highly automated driving is expected to enhance traffic safety, its practical effects have yet to be verified. This study systemically investigated the expected safety benefits and driver behavioral adaptation based on structural equation modeling. First, we developed a notification system to inform drivers of latent hazards with auditory alerts and conducted a driving simulation experiment involving eyes-off-road situations. To test the system, we adopted two types of events (i.e., the collision avoidance function working or failure) in which latent hazards transform into immediate risks. Then, a measurement model was developed to evaluate driver trust, driver attention, and traffic safety. Subsequently, we examined the corresponding causal relationships. On the one hand, latent hazard notification significantly improves driver attention (i.e., more fixations on latent hazards, less engagement in non-driving-related tasks, and faster notice of immediate risks), which significantly enhances traffic safety. On the other hand, latent hazard notification significantly increases driver trust, which lowers driver attention and consequently impairs traffic safety. This causality reveals driver behavioral adaptation, although driver trust does not directly affect traffic safety. Overall, we find that latent hazard notification for highly automated driving can improve traffic safety, but the consequent driver behavioral adaptation impairs 15.12% of the expected safety benefits.

Take-over performance plays a significant role in evaluating drivers' state, and serves as a crucial reference for enhancing control transitions in the context of conditionally automated driving. In this study, we aim to predict minimum anticipated collision time (min ACT), an indicator of drivers' take-over performance, in expectation of promoting safer take-overs via deep learning, so that drivers' state detriment of take-over safety could be adjusted accordingly with intelligent human-machine interaction algorithms predictably. By incorporating multi-source information including drivers' state, drivers' demographics, surrounding traffic features as well as driver-vehicle interaction characteristics, network model “ACTNet” was proposed to facilitate continuous estimation. Depthwise separable convolution and non-local self-attention were utilized to prevent overfitting and establish spatial dependency over fixation heatmap, respectively. To overcome data distribution imbalance, class balanced loss was used in conjunction with regression loss to realize more accurate predictions. Driving simulator experiment was conducted with dataset collected for the subsequent verification of the proposed algorithm. Potentialities of deep learning methods were highlighted for take-over studies, contributing to the design of intelligent human-machine interaction systems in conditional automation. Our findings present a valid method of deep learning in predicting drivers' take-over performance and meanwhile have implications for the development of intelligent adaptive take-over time budget regulation and dynamic drivers' state adjustment algorithms.