Accurate and interpretable delay predictions are vital for decision-making in the aviation industry. However, effectively incorporating spatiotemporal dependencies and external factors related to delay propagation remains a challenge. To address this challenge, we propose the SpatioTemporal Propagation Network (STPN), a novel space-time separable graph convolutional network that models delay propagation by considering both spatial and temporal factors. STPN uses a multi-graph convolution model that considers both geographic proximity and airline schedules from a spatial perspective, while employing a multi-head self-attention mechanism that can be learned end-to-end and explicitly accounts for various types of temporal dependencies in delay time series from a temporal perspective. Experiments on two real-world delay datasets show that STPN outperforms state-of-the-art methods for multi-step ahead arrival and departure delay prediction in large-scale airport networks. Additionally, the counterfactuals generated by STPN provide evidence of its ability to learn explainable delay propagation patterns. Comprehensive experiments also demonstrate that STPN sets a robust benchmark for general spatiotemporal forecasting. The code for STPN is available at https://github.com/Kaimaoge/STPN .

Illumination normalization for face recognition is very important when a face is captured under harsh lighting conditions. Instead of designing hand-crafted features, in this paper we formulate face illumination normalization as an image-to-image translation task. A great challenge of face normalization is that human facial structures are particularly sensitive to image structure distortion, which frequently occurs in traditional image-to-image translation tasks. Unfortunately, sometimes even slight facial structure distortions may prohibit human eyes and machine face recognition methods from identifying face identities. To address this issue, a novel GAN-based network architecture called the asymmetric joint generative adversarial network (AJGAN) is developed to normalize face images under arbitrary illumination conditions, without known face geometry and albedo information. In addition, an illumination normalization GAN $G_1$ and an asymmetric relighting GAN $G_2$ that maps a frontal-illuminated image to images with various lighting conditions are incorporated in AJGAN to maintain personalized facial structures. To avoid image blurring caused by underconstrained relighting mapping, we introduce a scheme of one-hot lighting labels into $G_2$ and enforce label classification loss. Furthermore, the number of training images starting from a very limited number of labels is dynamically extended by the combination of different lighting labels. Qualitative and quantitative experiments on three databases validate that AJGAN significantly outperforms the state-of-the-art methods.

Airport traffic flow prediction is a fundamental research topic in the field of air traffic flow management. Most existing works focus on the single airport traffic flow prediction with temporal dynamics but fail to consider the influence of the topological airport network. In this paper, a novel deep learning-based framework, called airport traffic flow prediction network (ATFPNet), is proposed to capture spatial-temporal dependencies of the historical airport traffic flow (departure and arrival) for the multiple-step situational (network-level) arrival flow prediction. Firstly, considering the nature of the airport distribution and the context of air transportation, a special semantic graph built on the flight schedule is applied to represent the airport network, which is the key to encoding the situational airport traffic flow into a single representation. Then, the graph convolution operator and the gated recurrent unit are combined to extract high-level transition patterns of airport traffic flow in the spatial and temporal dimensions. Finally, a real-world airport traffic flow dataset is applied to validate the effectiveness of the proposed model, and the experimental results demonstrate that the ATFPNet outperforms other baselines on different prediction horizons. Specifically, the proposed method achieves up to 17% MAE improvement compared to baselines. Based on the proposed approach, efficient traffic planning is expected to be achieved for airport management.

Due to increased air traffic flow, air traffic controllers (ATCs) operate in a state of high load or even overload for long periods of time, which can seriously affect the reliability and efficiency of controllers’ commands. Thus, the early identification of ATCs who are overworked is crucial to the maintenance of flight safety while increasing overall flight efficiency. This study uses a comprehensive comparison of existing cognitive load assessment methods combined with the characteristics of the ATC as a basis from which a method for the utilization of speech parameters to assess cognitive load is proposed. This method is ultimately selected due to the minimal interference of the collection equipment and the abundance of speech signals. The speech signal is pre-processed to generate a Mel spectrogram, which contains temporal information in addition to energy, tone, and other spatial information. Therefore, a speech cognitive load evaluation model based on a stacked convolutional neural network (CNN) and the Transformer encoder (SCNN-TransE) is proposed. The use of a CNN and the Transformer encoder allows us to extract spatial features and temporal features, respectively, from contextual information from speech data and facilitates the fusion of spatial features and temporal features into spatio-temporal features, which improves our method’s ability to capture the depth features of speech. We conduct experiments on air traffic control communication data, which show that the detection accuracy and F1 score of SCNN-TransE are better than the results from the support-vector machine (SVM), k-nearest neighbors (KNN), random forest (RF), adaptive boosting (AdaBoost), and stacked CNN parallel long short-term memory with attention (SCNN-LSTM-Attention) models, reaching values of 97.48% and 97.07%, respectively. Thus, our proposed model can realize the effective evaluation of cognitive load levels.

The reconstruction of 3D face data is widely used in the fields of biometric recognition and virtual reality. However, the rapid acquisition of 3D data is plagued by reconstruction accuracy, slow speed, excessive scenes and contemporary reconstruction-technology. To solve this problem, an accurate 3D face-imaging implementation framework based on coarse-to-fine spatiotemporal correlation is designed, improving the spatiotemporal correlation stereo matching process and accelerating the processing using a spatiotemporal box filter. The reliability of the reconstruction parameters is further verified in order to resolve the contention between the measurement accuracy and time cost. A binocular 3D data acquisition device with a rotary speckle projector is used to continuously and synchronously acquire an infrared speckle stereo image sequence for reconstructing an accurate 3D face model. Based on the face mask data obtained by the high-precision industrial 3D scanner, the relationship between the number of projected speckle patterns, the matching window size, the reconstruction accuracy and the time cost is quantitatively analysed. An optimal combination of parameters is used to achieve a balance between reconstruction speed and accuracy. Thus, to overcome the problem of a long acquisition time caused by the switching of the rotary speckle pattern, a compact 3D face acquisition device using a fixed three-speckle projector is designed. Using the optimal combination parameters of the three speckles, the parallel pipeline strategy is adopted in each core processing unit to maximise system resource utilisation and data throughput. The most time-consuming spatiotemporal correlation stereo matching activity was accelerated by the graphical processing unit. The results show that the system achieves real-time image acquisition, as well as 3D face reconstruction, while maintaining acceptable systematic precision.