Paving the way to future mobility, teleoperation of vehicles promises a reachable solution to effectively use the benefits of automated driving as long as fully automated vehicles (SAE 5) are not entirely feasible. Safety and reliability are assured by a human operator who remotely observes the vehicle and takes over control in cases of disturbances that exceed the vehicle automation’s skills. In order to integrate the vehicle’s automation and human remote-operation, we developed a novel user-centered human-machine interface (HMI) for teleoperation. It is tailored to the remote-operation of a highly automated shuttle (SAE 4) by a public transport control center and based on a systematic analysis of scenarios, of which detailed requirements were derived. Subsequently, a paper-pencil prototype was generated and refined until a click-dummy emerged. This click-dummy was evaluated by twelve control center professionals. The experts were presented the prototype in regular mode and were then asked to solve three scenarios with disturbances in the system. Using structured interview and questionnaire methodology, the prototype was evaluated regarding its usability, situation awareness, acceptance, and perceived workload. Results support our HMI design for teleoperation of a highly automated shuttle, especially regarding usability, acceptance, and workload. Participant ratings and comments indicated particularly high satisfaction with the interaction design to resolve disturbances and the presentation of camera images. Participants’ feedbacks provide valuable information for a refined HMI design as well as for further research.

Remotely operating vehicles utilize the benefits of vehicle automation when fully automated driving is not yet possible. A human operator ensures safety and availability from afar and supports the vehicle automation when its capabilities are exceeded. The remote operator, conceptualized as remote assistant, fulfills the legal requirements in Germany as a Technical Supervisor to operate highly automated vehicles at Society of Automotive Engineers 4. To integrate the remote operator into the automated driving system, a novel user-centered human–machine interface (HMI) for a remote assistant’s workplace was developed and initially evaluated. The insights gained in this process were incorporated into the design of a workplace prototype for remote assistance. This prototype was tested in the study reported here by 34 participants meeting the professional background criteria for the role of Technical Supervisor according to the German law. Typical scenarios that may occur in highly automated driving and require remote assistance were created in a simulation environment. Even under elevated cognitive load induced by simultaneously engaging in a secondary task, participants were able to obtain sufficient situation awareness and quickly resolve the scenarios. The HMI also yielded favorable usability and acceptance ratings. The results of this study inform the iterative workplace development and further research on the remote assistance of highly automated vehicles.

Remote operation bears the potential to roll out highly automated vehicles (AVs, SAE Level 4) more safely and quickly. Moreover, legal regulations on highly automated driving, e.g., the current law on highly automated driving (SAE Level 4) in Germany, permit a remote supervisor to monitor and intervene in driving operations remotely in lieu of a safety operator on board AVs. In order to derive requirements for safe and effective remote driving and remote assistance of AVs and to create suitable human-centered design solutions for human-machine interfaces (HMIs) that serve this purpose, a set of 74 core scenarios that are likely to occur in public transport AVs under remote operation was compiled. The scenarios were collected in several projects on the remote operation of AVs across a variety of contexts including interviews with and observations of control center staff, video analyses from naturalistic road events, and interviews with safety operators of AVs. A hierarchical system that is based on interactions of central actors was used to structure the scenarios. The set explicates relevant cases in remote operation, which may help improve workplaces for remote operation both by combatting human factors issues such as distraction and fatigue, and by boosting usability, user experience, trust, and acceptance. As the catalogue of scenarios is not exhaustive, scenarios may be added as knowledge of the remote operation of AVs progresses. Further research is needed to validate and adapt the scenarios to specific conceptualizations of remote operations.

Remote assistance for highly automated vehicles (HAVs), i.e., third-party assistance from support staff outside the vehicle in times of the need for assistance, presents a solution to extend the capabilities of HAVs by integrating a third party for decision making in uncertain situations. Similar to other control center positions, we expect the remote assistance tasks to exert high mental demands on the human operators. Therefore, we assessed impact of elevated mental workload during HAV remote assistance in a controlled environment in a user study (N = 37) with the goal of identifying cues to differentiate workload levels based on eye-tracking-related, skin conductance, and cardiovascular indicators. The results provide evidence that (A) elevated workload induced via a secondary task depreciates performance, and (B) we can identify workload levels person-independently as differences in tonic skin conductance (F(2,72) = 24.538, p < 0.001, partial η² = 0.405) and pupil dilation (F(2,72) = 13.872, p < 0.001, partial η² = 0.278), resulting in a classification accuracy of 58% in a three-class classification task. The results provide evidence that we are able to differentiate operator workload during remote assistance in a time-resolved way with the ultimate goal to provide adaptations to counteract task deficiencies.