Previous studies in robotic-assisted surgery (RAS) have studied cognitive workload by modulating surgical task difficulty, and many of these studies have relied on self-reported workload measurements. However, contributors to and their effects on cognitive workload are complex and may not be sufficiently summarized by changes in task difficulty alone. This study aims to understand how multi-task requirement contributes to the prediction of cognitive load in RAS under different task difficulties. Multimodal physiological signals (EEG, eye-tracking, HRV) were collected as university students performed simulated RAS tasks consisting of two types of surgical task difficulty under three different multi-task requirement levels. EEG spectral analysis was sensitive enough to distinguish the degree of cognitive workload under both surgical conditions (surgical task difficulty/multi-task requirement). In addition, eye-tracking measurements showed differences under both conditions, but significant differences of HRV were observed in only multi-task requirement conditions. Multimodal-based neural network models have achieved up to 79% accuracy for both surgical conditions.
Drivers' situation awareness (SA) is often assessed using indirect behavioural indicators such as takeover performance, despite known limitations. Many studies have also focused heavily on visual attention, primarily capturing the perception level of SA. To address this, we developed two physiological SA prediction models. Model 1 links physiological responses to full-level SA (perception, comprehension, and projection), while Model 2 targets comprehension-level SA to examine mechanisms beyond perception. A simulated driving study was conducted with 39 participants, during which eye-tracking and EEG data were collected. In Model 1, both modalities distinguished the degree of full-level SA. In Model 2, eye-tracking metrics differentiated successful from unsuccessful comprehension. In both models, the eye-tracking-based model outperformed the EEG-based model. However, combined model showed better performance, particularly in predicting comprehension-level SA. These findings demonstrate the benefit of integrating cognitive state monitoring beyond perception alone, especially for assessing higher-level SA in automated driving.
Conditionally automated vehicles require drivers to take over control occasionally. To date, takeover performance has been mostly evaluated using only re-engagement time and quality metrics. However, the appropriateness of takeover decisions, which has not been considered by previous research, should also be included as a performance indicator as it reflects one’s situation awareness of the takeover scenario. The goal of this study was to use eye-tracking, demographic factors, workload, and non-driving-related task (NDRT) conditions to predict takeover decisions. Forty-three participants drove a simulated conditionally automated vehicle while performing visual NDRTs and needed to decide the most appropriate maneuver around a roadway obstacle. Six classifiers were used to predict takeover decisions. The Random Forest model achieved the best performance, and driving experience and perceived workload were the most influential features. Findings may be used to assist in the design of adaptive algorithms that support drivers taking over from automated vehicles.
1 September 2023 in Proceedings of the Human Factors and Ergonomics Society Annual Meeting
The immediate effect of olfactory stimuli on drivers’ cognitive states with brain activity monitoring techniques was investigated. Time-variant cognitive states induced by external stimuli in driving are not well understood. We applied active stimuli in-cabin to understand the temporal characteristics of the stimuli on drivers’ cognitive states. Nine subjects participated in simulated driving task. Neural signals were acquired by a wireless electroencephalogram (EEG) headset. The olfactory stimuli were applied by providing scent. Time-frequency analysis was conducted for measuring time-variant EEG signal. Finally, event-related synchronization/desynchronization spectral power were calculated to evaluate drivers’ cognitive states. We found ERSs in theta (1-2s, 3-4s)/alpha (3-4s)/beta (3-5s) band.