Autonomous driving will change the role of the driver. From being the main actor in driving, the driver will now have a supervisory role during the autonomous driving phases. However, if the driver has to take over control of the vehicle, he must be aware of the situation around him. This is why it is important to develop interfaces to keep him in the loop. This article proposes a systematic review of Human Vehicle Interaction (HVI) providing situation awareness in the context of autonomous driving. 37 articles presenting such interactions are analyzed in terms of design of the interaction (modalities, location, conveyed information) but also in term of evaluation and experimental conditions. We present an overview of previous studies in order to highlight the work already done or in progress. Current studies present mainly monomodal interfaces although the evaluation of multimodal interactions present promising results in this field.

Advanced driver assistances are becoming increasingly common in commercial cars, not only to assist but also to free drivers from manual driving whenever possible. Soon, drivers should be allowed to engage in non-driving-related tasks. The fact that responsibility for driving is shifting from humans to machines must be considered in the development of these assistances in order to guarantee safety and trust. In this article, we introduce AdVitam (for Advanced Driver-Vehicle Interaction to Make future driving safer), an autonomous system aiming at maintaining driver’s situation awareness and optimizing takeover quality during conditionally automated driving (L3-SAE). The information conveyed to drivers is dynamically adapted to achieve these goals, depending on the driving environment and the driver’s physiological state. This system consists of three connected modules. The first module ( Driver State ) predicts the driver’s state with machine learning and physiological signals as inputs. The second module ( Supervision ) uses different interfaces (a haptic seat, a personal device, and ambient lights) to maintain the drivers’ situation awareness during the autonomous driving phases. The third module ( Intervention ) is a machine learning model that chooses the most appropriate combination among haptic, auditory, and visual modalities to request the driver to take over control and thus optimize takeover quality. To evaluate the system and each module independently, a preliminary user study with 35 drivers was conducted in a fixed-base driving simulator. All participants drove in two different environments (rural and urban). In addition, the activation of the Supervision and Intervention modules were manipulated as two between-subject factors. Results show that conveying information on the driving environment status through multimodal interfaces increases drivers’ situation awareness (i.e., better identification of potential problems in the environment) and trust in the automated vehicle. However, the system does not show positive outcomes on takeover quality. Besides, the Driver State module provided consistent predictions with the experimental manipulation. The system proposed in this paper could lead to better acceptance and safety when L3-SAE vehicles will be released by increasing drivers’ trust during phases of automated driving.

This dataset contains data of 346 drivers collected during six experiments conducted in a fixed-base driving simulator. Five studies simulated conditionally automated driving (L3-SAE), and the other one simulated manual driving (L0-SAE). The dataset includes physiological data (electrocardiogram (ECG), electrodermal activity (EDA), and respiration (RESP)), driving and behavioral data (reaction time, steering wheel angle, …), performance data of non-driving-related tasks, and questionnaire responses. Among them, measures from standardized questionnaires were collected, either to control the experimental manipulation of the driver's state, or to measure constructs related to human factors and driving safety (drowsiness, mental workload, affective state, situation awareness, situational trust, user experience). In the provided dataset, some raw data have been processed, notably physiological data from which physiological indicators (or features) have been calculated. The latter can be used as input for machine learning models to predict various states (sleep deprivation, high mental workload, ...) that may be critical for driver safety. Subjective self-reported measures can also be used as ground truth to apply regression techniques. Besides that, statistical analyses can be performed using the dataset, in particular to analyze the situational awareness or the takeover quality of drivers, in different states and different driving scenarios. Overall, this dataset contributes to better understanding and consideration of the driver's state and behavior in conditionally automated driving. In addition, this dataset stimulates and inspires research in the fields of physiological/affective computing and human factors in transportation, and allows companies from the automotive industry to better design adapted human-vehicle interfaces for safe use of automated vehicles on the roads.

People with intellectual disability (ID) should routinely train themselves to carry out a variety of daily challenging tasks while being supervised by one or more supervisors. Virtual reality (VR) technology enables the simulation of certain learning scenarios that would be risky or difficult to set up or repeatedly replicate in the actual world. This paper introduces a VR simulator created for this aim with the assistance of social educators. The purpose is to use VR to conduct learning exercises with teenagers with ID and assess the extent to which the abilities learned in VR can be transferred to the real world. This project focuses mainly on urban mobility with three types of exercise. A study was conducted with 18 students in five institutions for 7 months. Post-tests were also carried out after 1 month with 11 teenagers. In the end, four teenagers are completely autonomous in their travels: two travel on foot and two travel by public transport. Regarding the impact analysis, the results are stable over time regarding self-reported ease, satisfaction, and fatigability. Finally, the system received valuable feedback from the educators.

ContextConditionally autonomous vehicles are studied for numerous reasons, due to the dual responsibility in the driving task, switching from the car to the driver depending on the situation. This transition of control, namely the takeover, is usually not a concern when initiated by the driver. However, when the car issues a takeover request (TOR), it does it without consideration for the driver state, or anything else. In such a situation, the driver can be out of the driving loop, unaware of the current environment or focusing on a non-driving-related task (NDRT), impacting the takeover quality.>ObjectiveIn this study, the goal is to find out if a Machine Learning can take advantages of many different factors and how they interact together, to create a model able to predict the takeover quality. Factors highlighted by the literature were considered: the driver physiological state, the external environment, the driver current activity and the modality of the TOR. Some studies tried to predict takeover quality, but they usually study the TOR modality impact, the NDRT or the driver state. In this paper, we attempt to study the bigger picture and grasps the impact of the interaction between those multiple factors. >Methodology>>Experiment15 drivers’ physiological signals (EDA, ECG and respiration) were recorded during a 50 minutes driving session on a fixed-base driving simulator. They encountered 9 takeover situations each, caused by a stationary obstacle appearing on a road with a time-to-collision of around 7 seconds. Physiological data were processed on the last 90 seconds before each TOR. The possible TOR modalities were combinations of visual (icon on the dashboard), auditory (short beep sound) and haptic (vibrating seat). Combinations tested were visual-haptic, visual-auditory and visual-auditory-haptic. The drivers had 3 task for each third of the driving session: Visual n-back task, auditory n-back task or monitoring the car (no task).Half the participants (8) had an adverse weather during the driving session, with low luminosity and heavy rain, while the other half (7) had a sunny weather. Reaction time between the TOR and the takeover, and the maximum steering wheel angle attained during the takeover process were recorded as takeover quality metrics. >>Machine LearningTakeover quality metrics were combined to create a unique label to predict. State-of-the-art Feature Selection techniques were applied to keep only to more relevant features, and after outliers suppressions and data processing, 80 takeovers were kept for the Machine Learning models training. Data Augmentation methods, such as SMOGN and Random Noise were implemented and tested to boost the training dataset. KNeighbors Regressor, Support Vector Regressor (SVR), Random Forest Regressors (RFR) and Neural Networks (NN) were trained using a grid search approach and cross validation. Evaluation was done using MSE, MAE and R2.>ResultsThe NN gave the best results, with a MSE of 0.0538, a R2 of 0.1040 and a MAE of 0.1614. The results are similar to the most recent literature. Most important physiological features included indicators of the Respiratory Sinus Arrhythmia, Heart Rate Variability and Skin Conductance Response

In conditionally automated driving, several factors can affect the driver’s situation awareness and ability to take over control. To better understand the influence of some of these factors, 88 participants spent 20 minutes in a conditionally automated driving simulator. They had to react to four obstacles that varied in danger and movement. Half of the participants were required to engage in a verbal cognitive non-driving-related task. Situation awareness, takeover performance and physiological responses were measured for each situation. The results suggest that obstacle movement influences obstacle danger perception, situation awareness, and response time, while the latter is also influenced by obstacle danger. The cognitive verbal task also had an effect on the takeover response time. These results imply that the driver’s cognitive state and the driving situation (e.g. the movement/danger of entities present around the vehicle) must be considered when conveying information to drivers through in-vehicle interfaces.