Conflicts between the pilot and the automation, when pilots detect but do not understand them, cause "automation surprise" situations and jeopardize flight safety. We conducted an experiment in a 3-axis motion flight simulator with 16 pilots equipped with an eye-tracker to analyze their behavior and eye movements during the occurrence of such a situation. The results revealed that this conflict engages participant's attentional abilities resulting in excessive and inefficient visual search patterns. This experiment confirmed the crucial need to design solutions for detecting the occurrence of conflictual situations and to assist the pilots. We therefore proposed an approach to formally identify the occurrence of "automation surprise" conflicts based on the analysis of "silent mode changes" of the autopilot. A demonstrator was implemented and allowed for the automatic trigger of messages in the cockpit that explains the autopilot behavior. We implemented a real-time demonstrator that was tested as a proof-of-concept with 7 subjects facing 3 different conflicts with automation. The results shown the efficacy of this approach which could be implemented in existing cockpits.

Electro-oculography (EOG)-based brain–computer interface (BCI) is a relevant technology influencing physical medicine, daily life, gaming and even the aeronautics field. EOG-based BCI systems record activity related to users’ intention, perception and motor decisions. It converts the bio-physiological signals into commands for external hardware, and it executes the operation expected by the user through the output device. EOG signal is used for identifying and classifying eye movements through active or passive interaction. Both types of interaction have the potential for controlling the output device by performing the user’s communication with the environment. In the aeronautical field, investigations of EOG-BCI systems are being explored as a relevant tool to replace the manual command and as a communicative tool dedicated to accelerating the user’s intention. This paper reviews the last two decades of EOG-based BCI studies and provides a structured design space with a large set of representative papers. Our purpose is to introduce the existing BCI systems based on EOG signals and to inspire the design of new ones. First, we highlight the basic components of EOG-based BCI studies, including EOG signal acquisition, EOG device particularity, extracted features, translation algorithms, and interaction commands. Second, we provide an overview of EOG-based BCI applications in the real and virtual environment along with the aeronautical application. We conclude with a discussion of the actual limits of EOG devices regarding existing systems. Finally, we provide suggestions to gain insight for future design inquiries.

Commercial aviation is currently one of the safest modes of transportation; however, human error is still one major contributing cause of aeronautical accidents and incidents. One promising avenue to further enhance flight safety is Neuroergonomics, an approach at the intersection of neuroscience, cognitive engineering and human factors, which aims to create better human–system interaction. Eye-tracking technology allows users to “monitor the monitoring” by providing insights into both pilots’ attentional distribution and underlying decisional processes. In this position paper, we identify and define a framework of four stages of step-by-step integration of eye-tracking systems in modern cockpits. Stage I concerns Pilot Training and Flight Performance Analysis on-ground; stage II proposes On-board Gaze Recordings as extra data for the “black box” recorders; stage III describes Gaze-Based Flight Deck Adaptation including warning and alerting systems, and, eventually, stage IV prophesies Gaze-Based Aircraft Adaptation including authority taking by the aircraft. We illustrate the potential of these four steps with a description of incidents or accidents that we could certainly have avoided thanks to eye-tracking. Estimated milestones for the integration of each stage are also proposed together with a list of some implementation limitations. We believe that the research institutions and industrial actors of the domain will all benefit from the integration of the framework of the eye-tracking systems into cockpits.