Engaging in non-driving related tasks (NDRTs) while driving can be considered distracting and safety detrimental. However, with the introduction of highly automated driving systems that relieve drivers from driving, more NDRTs will be feasible. In fact, many car manufacturers emphasize that one of the main advantages with automated cars is that it “frees up time” for other activities while on the move. This paper investigates how well drivers are able to engage in an NDRT while in automated driving mode (i.e., SAE Level 4) in real traffic, via a Wizard of Oz platform. The NDRT was designed to be visually and cognitively demanding and require manual interaction. The results show that the drivers’ attention to a great extent shifted from the road ahead towards the NDRT. Participants could perform the NDRT equally well as when in an office (e.g. correct answers, time to completion), showing that the performance did not deteriorate when in the automated vehicle. Yet, many participants indicated that they noted and reacted to environmental changes and sudden changes in vehicle motion. Participants were also surprised by their own ability to, with ease, disconnect from driving. The presented study extends previous research by identifying that drivers to a high extent are able to engage in an NDTR while in automated mode in real traffic. This is promising for future of automated cars ability to “free up time” and enable drivers to engage in non-driving related activities.

Fully automated driving has posed more challenges than expected, and remote operation of heavy vehicles is increasingly getting attention. Therefore, human remote operators may have an essential role in compensating for the technological shortcomings in vehicle automation. This poses challenges in designing the work of human remote operators of automated heavy vehicles. This paper present findings from a research project performed in collaboration between the RISE Research Institutes of Sweden and Scania. In the project, human-automation interaction requirements and challenges for remote operator work were explored through a simulator study. Before the study, three main operator tasks were defined: assessment, assistance, and remote driving. The simulation occurred in a transportation scenario where operators handled ten trucks driving on a public road and in confined areas (transportation hub). Fifteen participants completed the study. The results provide examples and insights into classical automation-related challenges in a new context—the remote operation of heavy vehicles. Instances of challenges with situational awareness, out-of-the-loop, trust, and attention management were found and are discussed in relation to HMI design and requirements. In addition, it was found that transitions between relatively passive monitoring and more active assistance and driving were performed more fluently than expected. In general, supervisory control of ten vehicles in parallel was seen as a feasible task given the conditions in the simulated environment.

Thanks to the advent of telepresence applications, we can remotely take control and operate industrial machinery. Teleoperation removes operators from hazardous workplaces such as mining and plays an essential role in the safety of workers. In addition, augmented telepresence can introduce information that helps the user understand the remote scene. However, remote operation presents challenges since the information received is more limited than what could be perceived by being physically present, such as accurately judging depth. This study investigates how well operators interact with an augmented remote operation scaling system (AROSS) in a mining context when different computer-generated visual interfaces are provided. The system can achieve five visual interfaces: Disocclusion Augmentation view using selective content removal, Novel Perspective view generation, Lidar view, Right (Original) view, and Left (Original) view. We performed two experiments in a mine-like laboratory to analyze human interaction with the designed prototype by applying a mixed research methodology that used questionnaires, interviews, and observations. This mixed methodology consisted of quality of experience methods to discover the users’ requirements from a technological standpoint and user experience methods (i.e., user-centric approaches). We investigated 10 and 11 users’ interactions in two experimental studies. The first experiment focused on identifying small patterns (e.g., cracks in the mine wall), and the second focused on depth and three-dimensional understanding. We considered the first experiment a feasibility test to understand how to conduct the second experiment. Therefore, we designed the second test to assess the technical readiness of AROSS from the users’ perspective. The overall conclusion yields a comprehensive understanding of users’ perceptions and experiences. The quality of experience results favored Left and Right (Original) views for remote control, indicating a preference among remote operators using natural (Original) views due to their facilitation of environmental comprehension. User experience analysis revealed the reason why other views were less favored and what their potential benefits are. Specifically, Novel Perspective and Lidar views were found helpful for depth perception, and Disocclusion Augmentation view functionality could be enhanced if robot arm position tracking is enabled. These insights inform design recommendations, emphasizing the value of incorporating Disocclusion Augmentation and Novel Perspective views and suggesting improvements to enhance system usability.

To explore driver behavior in highly automated vehicles (HAVs), independent researchers are mainly conducting short experiments. This limits the ability to explore drivers’ behavioral changes over time, which is crucial when research has the intention to reveal human behavior beyond the first-time use. The current paper shows the methodological importance of repeated testing in experience and behavior related studies of HAVs. The study combined quantitative and qualitative data to capture effects of repeated interaction between drivers and HAVs. Each driver ( n = 8 n=8 ) participated in the experiment on two different occasions (∼90 minutes) with one-week interval. On both occasions, the drivers traveled approximately 40 km on a rural road at AstaZero proving grounds in Sweden and encountered various traffic situations. The participants could use automated driving (SAE level 4) or choose to drive manually. Examples of data collected include gaze behavior, perceived safety, as well as interviews and questionnaires capturing general impressions, trust and acceptance. The analysis shows that habituation effects were attenuated over time. The drivers went from being exhilarated on the first occasion, to a more neutral behavior on the second occasion. Furthermore, there were smaller variations in drivers’ self-assessed perceived safety on the second occasion, and drivers were faster to engage in non-driving related activities and become relaxed (e. g., they spent more time glancing off road and could focus more on non-driving related activities such as reading). These findings suggest that exposing drivers to HAVs on two (or more) successive occasions may provide more informative and realistic insights into driver behavior and experience as compared to only one occasion. Repeating an experiment on several occasions is of course a balance between the cost and added value, and future research should investigate in more detail which studies need to be repeated on several occasions and to what extent.