With the increasing use of unmanned aerial vehicles (UAVs) in military operations, autonomous cooperative decision-making for multiple UAVs in aerial confrontations has become a critical research challenge. This paper presents an attention-based multi-agent actor-critic (AMAAC) algorithm for UAV aerial confrontation decision-making. The algorithm combines multi-head attention and self-play within the centralized training-distributed execution (CTDE) framework, extending the actor-critic approach based on the missile hit probability prediction model (MHPAC) to multi-UAV scenarios. A fighter observation encoder (OFE) and a centralized critic network based on the attention mechanism are introduced to adapt to varying number of UAVs (different scales) and enhance training performance. Additionally, self-play-based extended training is used to generalize offensive and defensive strategies from small-scale aerial confrontations to larger scenarios. Experimental results demonstrate that the AMAAC algorithm achieves superior training effectiveness, and the strategies it produces perform well across various confrontation scales, even beyond the training scenario's scale. Compared to other decision-making algorithms, such as Multi-agent Proximal Policy Optimization (MAPPO), Multi-agent Hierarchical Policy Gradient (MAHPG), and the State-Event-Condition-Action (SECA) algorithm, the AMAAC-trained strategies yield higher win ratios and kill-death ratios in different scenarios, validating the algorithm's effectiveness and scalability.
10 September 2024 in IEEE Transactions on Vehicular Technology
The complexities inherent in beyond-visual-range (BVR) air combat research stem from various intertwined factors, including dynamic nonlinear interactions, evolving dynamics during confrontations, and significant delays in countermeasure outcomes. These complexities create challenges in managing system state variables and navigating a vast strategic solution space. Applying algorithms directly to address these challenges often worsens the complexities, leading to considerable deviations and convergence issues. This paper tackles these challenges by initially dissecting the core elements intrinsic to the confrontation process. It establishes a detailed BVR confrontation environment, scrutinizes the stage characteristics of typical confrontations, introduces a framework for BVR confrontation decision-making, and devises a rule-based state-switch decision method. Moreover, it extracts and analyzes key decision parameters, deliberating on the tactical significance of the chosen parameters. The proposed method offers robust interpretability and demonstrates strong decision-making capabilities through simulation results, showcasing its advantages in autonomous decision-making during confrontations.
1 October 2025 in IEEE Transactions on Vehicular Technology
In response to the intricate and dynamic nature of air combat, characterized by a vast state space dimension and nonlinear dynamics with strong coupling, this paper introduces a novel hierarchical hybrid architecture rooted in the principles of air combat mechanisms. The proposed approach systematically addresses close air combat challenges, aligning with the OODA concept in air combat. Each layer of the architecture is dedicated to a specific facet of air combat, enhancing clarity and inter-pretability. By incorporating an adversarial situation memory library, typical air combat scenarios are quantitatively analyzed, resulting in the development of a tactical scenario library that surpasses conventional expert knowledge bases in air combat. At the foundational layer, an aircraft's parameterized flight maneuver action library is established based on its characteristic flight curve, ensuring stable maneuvering strategies during confrontations. The Six Degrees of Freedom (6-DOF) model, embodying typical supersonic fighter dynamics characteristics, is tested in a close-range confrontation scenario of one-on-one air combat. The results show that the architecture outperforms state-of-the-art algorithms, achieving a relative winning rate of at least 61% under similar initial energy conditions.