Applications based on artificial intelligence (AI) promise benefits, ranging from improved performance to increased capabilities in many industries. In the aviation domain, one example is the new Airborne Collision Avoidance System (ACAS X). The current investigation aims at combining ACAS X and AI to maintain its performance while decreasing the memory footprint. However, the anticipation of AI being increasingly used confronts regulators with challenges in terms of safety assurance and certification. Consequently, the European Union Aviation Safety Agency (EASA) published a concept paper for machine learning applications in aviation. Both, the Concept of Operation (ConOps) in combination with an Operational Design Domain (ODD), are listed as objectives to be met for the safety analysis. From a developer’s perspective, this raises questions on how to effectively derive the ODD from ConOps and test the given system based on the ODD description. Based on an exemplary use case of a Near Mid-Air Collision avoidance between two aircraft through the advisories of ACAS X, a highly automated framework for generating and testing synthetic data is proposed. Using this framework, 1800 Near Mid-Air Collision scenario files are created and automatically executed in the simulation environment FlightGear. Scenario-based testing is used for the logging of ACAS X advisory data and evaluating it against predefined requirements. By this approach, an efficient way of verifying system requirements and conducting automated testing based on the ODD definition is demonstrated. Throughout this process, Model-Based Systems Engineering (MBSE) is used to reduce and manage complexity. The framework in this paper enables a systematic and highly automated approach for scenario generation based on the ODD.

A continuous increase in artificial intelligence (AI)-based functions can be expected for future aviation systems, posing significant challenges to traditional development processes. Established systems engineering frameworks, such as the V-model, are not adequately addressing the novel challenges associated with AI-based systems. Consequently, the European Union Aviation Safety Agency (EASA) introduced the W-shaped process, an advancement of the V-model, to set a regulatory framework for the novel challenges of AI Engineering. In contrast, the agile Development Operations (DevOps) approach, widely adopted in software development, promotes a never-ending iterative development process. This article proposes a novel concept that integrates aspects of DevOps into the W-shaped process to create an AI Engineering framework suitable for aviation-specific applications. Furthermore, it builds upon proven ideas and methods using AI Engineering efforts from other domains. The proposed extension of the W-shaped process, compatible with ongoing standardizations from the G34/WG-114 Standardization Working Group, a joint effort between EUROCAE and SAE, addresses the need for a rigorous development process for AI-based systems while acknowledging its limitations and potential for future advancements. The proposed framework allows for a re-evaluation of the AI/ML constituent based on operational information, enabling improvements of the system’s capabilities with each iteration.

A continuous increase of Artificial Intelligence (AI) based functions can be expected for future aviation systems, posing significant challenges to traditional development processes. Established systems engineering frameworks, such as the V-model, are not adequately addressing the novel challenges associated with AI-based systems. Consequently, the European Union Aviation Safety Agency (EASA) introduced the W-shaped process as an advancement of the V-model to set a regulatory framework for the novel challenges of AI Engineering. In contrast, the agile Development Operations (DevOps) approach, widely adopted in software development, promotes a never-ending iterative development process. This article proposes a novel concept that integrates aspects of DevOps into the W-shaped process to create an AI Engineering framework suitable for aviation-specific applications. Furthermore, it builds upon proven ideas and methods using AI Engineering efforts from other domains. The proposed extension of the W-shaped process, compatible with ongoing standardizations from the G34/WG-114 Standardization Working Group, a joint effort between EUROCAE and SAE, addresses the need for a rigorous development process for AI-based systems while acknowledging its limitations and potential for future advancements. The proposed framework allows for a re-evaluation of the AI/ML constituent based on information from operations, enabling improvement of the system’s capabilities in each iteration.