Neurosymbolic artificial intelligence (AI) is an emerging branch of AI that combines the strengths of symbolic AI and sub-symbolic AI. Symbolic AI is based on the idea that intelligence can be represented using semantically meaningful symbolic rules and representations, while deep learning (DL), or sometimes called sub-symbolic AI, is based on the idea that intelligence emerges from the collective behavior of artificial neurons that are connected to each other. A major drawback of DL is that it acts as a “black box”, meaning that predictions are difficult to explain, making the testing & evaluation (T&E) and validation & verification (V&V) processes of a system that uses sub-symbolic AI a challenge. Since neurosymbolic AI combines the advantages of both symbolic and sub-symbolic AI, this survey explores how neurosymbolic applications can ease the V&V process. This survey considers two taxonomies of neurosymbolic AI, evaluates them, and analyzes which algorithms are commonly used as the symbolic and sub-symbolic components in current applications. Additionally, an overview of current techniques for the T&E and V&V processes of these components is provided. Furthermore, it is investigated how the symbolic part is used for T&E and V&V purposes in current neurosymbolic applications. Our research shows that neurosymbolic AI has great potential to ease the T&E and V&V processes of sub-symbolic AI by leveraging the possibilities of symbolic AI. Additionally, the applicability of current T&E and V&V methods to neurosymbolic AI is assessed, and how different neurosymbolic architectures can impact these methods is explored. It is found that current T&E and V&V techniques are partly sufficient to test, evaluate, verify, or validate the symbolic and sub-symbolic part of neurosymbolic applications independently, while some of them use approaches where current T&E and V&V methods are not applicable by default, and adjustments or even new approaches are needed. Our research shows that there is great potential in using symbolic AI to test, evaluate, verify, or validate the predictions of a sub-symbolic model, making neurosymbolic AI an interesting research direction for safe, secure, and trustworthy AI.

The global shortage of air traffic controllers (ATCOs) has led to significant challenges. One of them is the high workload of ATCOs, often resulting in flight delays. This makes it essential to develop solutions that reduce ATCOs’ workload in order to increase capacity. One promising approach is the integration of decision-support systems. A typical task for which these systems are used for is the resolution of aircraft conflicts in the upper airspaces. A key challenge in implementing these support systems is to ensure a high acceptance and adoption rate of the proposed advisories. One potential solution to this problem is to personalize the advisories, aligning them with individual ATCOs’ preferences and conflict resolution strategies. As this approach offers many promising research directions, this literature review aims to provide a comprehensive overview of existing research in this domain and highlight potential opportunities and open challenges. Overall, 16 papers are discussed in detail to examine the diversity of conflict resolution strategies among ATCOs, the impact of personalization on the acceptance rate of advisories, the technical feasibility of implementing personalization, and the balance between personalized advisories and operational efficiency. Additionally, this paper highlights the opportunities such personalization presents, along with the unresolved challenges that should be addressed in future research.

With increasing automation of Air Traffic Management, new digital systems need to adapt to uncertainties caused by human behavior. One such uncertainty is the pilot response time after an air traffic control clearance issued by the Air Traffic Controller (ATCO). This study investigates factors influencing pilot response times using the Swedish Civil Air Traffic Control (SCAT) dataset by analysing roughly 830,000 clearances collected over 13 weeks. We find that pilot reactions vary significantly depending on altitude, time of day, and the maneuver delta requested by the ATCO. Lower altitudes (0–100 FL) exhibit shorter maneuver initiation delays compared to higher altitudes (300+ FL). The maneuver completion time for heading and level changes strongly correlates with the magnitude of change, whereas this correlation is weaker for speed adjustments. Although slight variations exist between airlines and aircraft types, these factors have less influence on the pilot response time. Based on these findings, we investigate the predictability of both initiation delay and maneuver completion delay. This is done by training and evaluating the performance of a machine learning model. Our model shows that maneuver completion times are highly predictable ( $$\mathbb {R}^ 2> 0.73$$ R 2 > 0.73 ) across all clearance types, while initiation delays prove more challenging to forecast accurately.

The current air traffic system is challenged by the growing volume of air traffic and a shortage of air traffic controllers (ATCOs). A potential solution is to increase the level of automation in air traffic control (ATC) by introducing a digital ATCO that is capable of working as a team partner with human ATCOs e.g., helping to resolve conflicts or optimize trajectories. Given the success of artificial intelligence (AI) in solving such complex, dynamic decision-making processes, it is a promising technology for developing these digital ATCOs. To enable the best possible cooperation between human and digital ATCOs, the latter could be fine-tuned to the decision-making patterns of human ATCOs, as human-conformal decisions tend to be more accepted by human ATCOs. One way to draw conclusions about the behavior of human ATCOs is to analyze Automatic Dependent Surveillance-Broadcast (ADS-B) data and the corresponding ATC-issued clearances and instructions. This data can be used as examples of the behavior of human ATCOs to train or adapt a digital ATCO. However, there are few publicly available datasets that provide both ADS-B and the associated clearances and instructions. Additionally, those that do exist are limited to specific regions and, therefore, may not be sufficient, as human ATCOs may have regional decision patterns or strategies that are not transferable to other areas. Therefore, to address this issue, this paper proposes an approach that uses XGBoost models trained on the Swedish Civil Air Traffic Control (SCAT) dataset, which includes both ATC-issued clearances and ADS-B data, to detect maneuvers that are likely to result from ATC instructions. Accordingly, this research is a step toward labeling any ADS-B data from publicly available sources with synthetic instructions, providing a method for generating training data to fine-tune digital ATCOs to human behavior. Using the proposed setup it was possible to achieve 95% accuracy on the detection of instructions. Also it was possible to achieve accuracies of over 90% percent when using normalized coordinates independent from the Swedish airspace, indicating that the models could be used with data from other regions. Additionally, it is shown that the flight path extracted from the ADS-B data can be used to reconstruct the type and value of the instruction that was likely issued.