Increasing the level of automation in air traffic management is seen as a measure to increase the performance of the service to satisfy the predicted future demand. This is expected to result in new roles for the human operator: he will mainly monitor highly automated systems and seldom intervene. Therefore, air traffic controllers (ATCos) would often work in a supervisory or control mode rather than in a direct operating mode. However, it has been demonstrated how human operators in such a role are affected by human performance issues, known as Out-Of-The-Loop (OOTL) phenomenon, consisting in lack of attention, loss of situational awareness and de-skilling. A countermeasure to this phenomenon has been identified in the adaptive automation (AA), i.e., a system able to allocate the operative tasks to the machine or to the operator depending on their needs. In this context, psychophysiological measures have been highlighted as powerful tool to provide a reliable, unobtrusive and real-time assessment of the ATCo’s mental state to be used as control logic for AA-based systems. In this paper, it is presented the so-called “Vigilance and Attention Controller”, a system based on electroencephalography (EEG) and eye-tracking (ET) techniques, aimed to assess in real time the vigilance level of an ATCo dealing with a highly automated human–machine interface and to use this measure to adapt the level of automation of the interface itself. The system has been tested on 14 professional ATCos performing two highly realistic scenarios, one with the system disabled and one with the system enabled. The results confirmed that (i) long high automated tasks induce vigilance decreasing and OOTL-related phenomena; (ii) EEG measures are sensitive to these kinds of mental impairments; and (iii) AA was able to counteract this negative effect by keeping the ATCo more involved within the operative task. The results were confirmed by EEG and ET measures as well as by performance and subjective ones, providing a clear example of potential applications and related benefits of AA.

Recent work on self-supervised pre-training focus on leveraging large-scale unlabeled speech data to build robust end-to-end (E2E) acoustic models (AM) that can be later fine-tuned on downstream tasks e.g., automatic speech recognition (ASR). Yet, few works investigated the impact on performance when the data properties substantially differ between the pre-training and fine-tuning phases, termed domain shift. We target this scenario by analyzing the robustness of Wav2Vec 2.0 and XLS-R models on downstream ASR for a completely unseen domain, air traffic control (ATC) communications. We benchmark these two models on several open-source and challenging ATC databases with signal-to-noise ratio between 5 to 20 dB. Relative word error rate (WER) reductions between 20% to 40% are obtained in comparison to hybrid-based ASR baselines by only fine-tuning E2E acoustic models with a smaller fraction of labeled data. We analyze WERs on the low-resource scenario and gender bias carried by one ATC dataset.

Air traffic controllers normally manage all aircraft information with flight strips. These strips contain static information about each flight such as call sign or weight category. Additionally, all clearances regarding altitude, speed, and direction are noted by the controller. Historically paper flight strips were in operation, but modern controller working positions use electronic flight strips or electronic aircraft labels. However, independent from the type, considerable controller effort is needed to manually maintain strip information consistent with commands given to the aircraft. Automatic Speech Recognition (ASR) is a solution which requires no additional work from the controller to maintain radar label information. The Assistant Based Speech Recognizer developed by DLR and Saarland University enables command error rates below 2%. Validation trials with controllers from Germany and Austria showed that workload reduction by a factor of three for label maintenance is possible.

Objective: The objective of this study was to examine the potential benefits and impact on pilot behavior from the use of portable weather applications. Method: Seventy general aviation (GA) pilots participated in the study. Each pilot was randomly assigned to an experimental or a control group and flew a simulated single-engine GA aircraft, initially under visual meteorological conditions (VMC). The experimental group was equipped with a portable weather application during flight. We recorded measures for weather situation awareness (WSA), decision making, cognitive engagement, and distance from the aircraft to hazardous weather. Results: We found positive effects from the use of the portable weather application, with an increased WSA for the experimental group, which resulted in credibly larger route deviations and credibly greater distances to hazardous weather (≥30 dBZ cells) compared with the control group. Nevertheless, both groups flew less than 20 statute miles from hazardous weather cells, thus failing to follow current weather-avoidance guidelines. We also found a credibly higher cognitive engagement (prefrontal oxygenation levels) for the experimental group, possibly reflecting increased flight planning and decision making on the part of the pilots. Conclusion: Overall, the study outcome supports our hypothesis that portable weather displays can be used without degrading pilot performance on safety-related flight tasks, actions, and decisions as measured within the constraints of the present study. However, it also shows that an increased WSA does not automatically translate to enhanced flight behavior. Application: The study outcome contributes to our knowledge of the effect of portable weather applications on pilot behavior and decision making.

Automatic speech recognition (ASR) allows transcribing the communications between air traffic controllers (ATCOs) and aircraft pilots. The transcriptions are used later to extract ATC named entities, e.g., aircraft callsigns. One common challenge is speech activity detection (SAD) and speaker diarization (SD). In the failure condition, two or more segments remain in the same recording, jeopardizing the overall performance. We propose a system that combines SAD and a BERT model to perform speaker change detection and speaker role detection (SRD) by chunking ASR transcripts, i.e., SD with a defined number of speakers together with SRD. The proposed model is evaluated on real-life public ATC databases. Our BERT SD model baseline reaches up to 10% and 20% token-based Jaccard error rate (JER) in public and private ATC databases. We also achieved relative improvements of 32% and 7.7% in JERs and SD error rate (DER), respectively, compared to VBx, a well-known SD system. ^{1 1} Our code is stored in the following public GitHub repository: https://github.com/idiap/bert-text-diarization-atc