Automation has been extensively used in flight operations, so pilots are less involved in actual flight control. With the long idle time during cruising, pilots may have their vigilance level reduced and eventually become out-of-the-loop. This research proposes a two-stage explainable adaptive automation approach to keep pilots in the loop based on Convolutional Neural Networks, Long Short-Term Memory, and EEG data collected from 24 participants in a one-hour simulator-based flight task in each level of automation. Our proposed spatiotemporal model yields test accuracy of 0.9918 and 0.9907 in the first and second stages, respectively, outperforming other benchmarking models by 30.79% and 10.73%, respectively. Furthermore, the Shapley additive explanations are adopted to strengthen the model interpretability and trustworthiness for safety-critical applications. Our model successfully identified that high delta and theta waves with low beta and gamma waves contribute positively to the out-of-the-loop state. It indicates that the classification aligns with the theoretical background and is trustworthy. The trustworthy adaptive deep learning model supports the dynamical automation configuration for improving human-automation collaboration in cruising flights.
Identifying the absence of situation awareness (SA) in air traffic controllers is critical since it directly affects their hazard perception. This study aims to introduce and validate a multimodal methodology employing electroencephalogram (EEG) and eye-tracking to investigate SA variation within specific air traffic control contexts. Data from 28 participants executing the experiment involving three different SA-probe tests illustrated the conceptual relationship between EEG and eye-tracking indicators and SA variations, using behavioural data as a proxy. The results indicated that both EEG and eye-tracking metrics correlated positively with the SA levels required, that is, the frequency spectrum in the β (13–30 Hz) and γ (30–50 Hz) bands, alongside the fixation/saccade-based indicators and pupil dilation increased in response to higher SA levels. This research has substantial implications for investigating SA using a human-centric approach via psychophysiological indicators, revealing the intrinsic interactions between the human capability envelope and SA, contributing to the development of a real-time monitoring system of SA variations for air transportation safety research.
3 July 2023 in The International Journal of Aerospace Psychology