Excessive mental workload will reduce work efficiency, but low mental workload will cause a waste of human resources. It is very significant to study the mental workload status of operators. The existing mental workload classification method is based on electroencephalogram (EEG) features, and its classification accuracy is often low because the channel signals recorded by the EEG electrodes are a group of mixed brain signals, which are similar to multi-source mixed speech signals. It is not wise to directly analyze the mixed signals in order to distinguish the feature of EEG signals. In this study, we propose a mental workload classification method based on EEG independent components (ICs) features, which borrows from the blind source separation (BSS) idea of mixed speech signals. This presented method uses independent component analysis (ICA) to obtain pure signals, i.e., ICs. The energy features of ICs are directly extracted for classifying the mental workload, since this method directly uses ICs energy features for feature extraction. Compared with the existing solution, the proposed method can obtain better classification results. The presented method might provide a way to realize a fast, accurate, and automatic mental workload classification.

Behavioral performance, subjective assessment based on NASA Task Load Index (NASA-TLX), as well as physiological measures indexed by electrocardiograph (ECG), event-related potential (ERP), and eye tracking data were used to assess the mental workload (MW) related to flight tasks. Flight simulation tasks were carried out by 12 healthy participants under different MW conditions. The MW conditions were manipulated by setting the quantity of flight indicators presented on the head-up display (HUD) in the cruise phase. In this experiment, the behavioral performance and NASA-TLX could reflect the changes of MW ideally. For physiological measures, the indices of heart rate variability (HRV), P3a, pupil diameter and eyelid opening were verified to be sensitive to MW changes. Our findings can be applied to the comprehensive evaluation of MW during flight tasks and the further quantitative classification.

Situation awareness (SA) is a crucial factor affecting flight safety for pilots, yet few studies have focused specifically on modeling SA for pilots, resulting in limited success. In this paper, we propose a novel multimodal deep learning approach to monitor pilots’ SA. The approach combines handcrafted and deep features obtained from eye movement and flight control data collected from 27 novice pilots across different training phases using a flight simulator. Ground truth SA measurements were obtained using the Situation Awareness Global Assessment Technique (SAGAT). The handcrafted features included 13 eye movements and 22 flight control features, while deep features were extracted from time-series of gaze positions using a deep extractor based on Transformer. By fusing the handcrafted features of eye movement and flight control, along with one deep feature of eye movement, we predicted the final SA level. Through leave-one-flight-out cross-validation, our model achieved a higher accuracy of 92.04%. The results indicate that the multimodal model outperforms the unimodal models, with the eye movement modality demonstrating superiority over the flight control modality in predicting SA. This suggests our method provides an objective means of predicting pilot’s SA and offers new insights for SA assessment in aviation and other fields. Overall, our multimodal deep learning approach holds promise for enhancing pilot training and flight safety by facilitating a more comprehensive understanding of pilots’ SA during critical flight scenarios.

BACKGROUND: The prediction and evaluation of pilot workload is a key problem in human factor airworthiness of cockpit. OBJECTIVE: A pilot traffic pattern task was designed in a flight simulation environment in order to carry out the pilot workload prediction and improve the evaluation method. METHODS: The prediction of typical flight subtasks and dynamic workloads (cruise, approach, and landing) were built up based on multiple resource theory, and a favorable validity was achieved by the correlation analysis verification between sensitive physiological data and the predicted value. RESULTS: Statistical analysis indicated that eye movement indices (fixation frequency, mean fixation time, saccade frequency, mean saccade time, and mean pupil diameter), Electrocardiogram indices (mean normal-to-normal interval and the ratio between low frequency and sum of low frequency and high frequency), and Electrodermal Activity indices (mean tonic and mean phasic) were all sensitive to typical workloads of subjects. CONCLUSION: A multinominal logistic regression model based on combination of physiological indices (fixation frequency, mean normal-to-normal interval, the ratio between low frequency and sum of low frequency and high frequency, and mean tonic) was constructed, and the discriminate accuracy was comparatively ideal with a rate of 84.85%.

Pilot situation awareness (SA) regulates flight safety, and inexperience may impair novice pilot reliability in SA. This study aims to determine the key influencing factors of novice pilot SA and to analyze the interrelationship and interaction mechanism of the factors. We investigated 55 novice pilots trained at aviation schools and identified the influencing factor index system by the Delphi survey. The method of Decision Making Trial and Evaluation (DEMATEL) combined with Adversarial Interpretive Structure Modeling (AISM) was adopted. The results show that: (1) The influencing factor index system includes 18 factors, divided into four categories: individual factors, team factors, task and human-machine system factors, and cockpit environment factors. (2) Team communication, team cooperation, basic cognitive ability, interface design, occupational age and experience, and authority gradient are the six crucial influencing factors. The former three have the greatest association with other factors, while the latter three are most likely to affect other factors. (3) Team communication, basic cognitive ability, and interface design are root-cause factors, of which team communication is the most fundamental. (4) The results of DEMATEL and AISM are consistent, both disclosing team communication as the fundamental factor with the highest priority, and cockpit environmental factors as the direct influencing factors but most susceptible to other factors. The present study can be viewed as a conducive attempt to extract vital influencing factors of novice pilot SA, and to provide ergonomic insights for determining the priorities to improve novice pilot SA in training and aircraft design for flight safety.

According to a previously built situation-awareness (SA) model based on attention allocation, with the ACT-R (Adaptive Control of Thought, Rational) theory for analyzing pilot cognitive processes for situation elements, an SA mathematical model was improved to predict pilot SA during exposure to different display interfaces and missions. An indicator-display monitoring task was performed under different experimental conditions for SA model verification, while the SA global assessment technique (SAGAT), performance measures, 10-dimensional SA rating technique (10-D SART), and eye movement measures were adopted to comprehensively assess the operator's SA. The experimental results revealed that theoretical prediction values calculated using the improved SA model were strongly correlated with the operation performance, and thus confirming the model validation. The SAGAT was shown to be a more effective method than SART in this research, and the overall SAGAT accuracy rate, as well as the accuracy response time, are effective indices for SA measurement. Eye-movement indices, such as the fixation/saccade ratio, which corresponds to the mode of information perception and extraction, was examined to be sensitive to operator's SA changes. The Advances of the improved SA model have been achieved in predicting and indicating SA by using human behaviors, including operation performance, SAGAT response behaviors, and visual behaviors. Thereby, it provides a new auxiliary tool for quantitative characterization of pilot's SA during cockpit interface design optimization and ergonomic evaluation.