Autonomous maneuver decision by an unmanned combat air vehicle (UCAV) is a critical part of air combat that requires both flight safety and tactical maneuvering. In this paper, an unmanned combat air vehicle air combat maneuver decision method based on a proximal policy optimization algorithm (PPO) is proposed. Firstly, a motion model of the unmanned combat air vehicle and a situation assessment model of air combat was established to describe the motion situation of the unmanned combat air vehicle. An enemy maneuver policy based on a situation assessment with a greedy algorithm was also proposed for air combat confrontation, which aimed to verify the performance of the proximal policy optimization algorithm. Then, an action space based on a basic maneuver library and a state observation space of the proximal policy optimization algorithm were constructed, and a reward function with situation reward shaping was designed for accelerating the convergence rate. Finally, a simulation of air combat confrontation was carried out, which showed that the agent using the proximal policy optimization algorithm learned to combine a series of basic maneuvers, such as diving, climb and circling, into tactical maneuvers and eventually defeated the enemy. The winning rate of the proximal policy optimization algorithm reached 62%, and the corresponding losing rate was only 11%.

To address the challenges of intelligent decision-making in complex and high-dimensional state–action spaces during joint operations simulations of sea–air unmanned systems, an end-to-end intelligent decision-making scheme is proposed. Initially, a highly versatile hierarchical intelligent decision-making method is designed for sea–air joint operations simulation scenarios. Subsequently, an approach combining intrinsic and extrinsic rewards is adopted to structurally mitigate the adverse effects of sparse rewards. Following this, a prominence detection method and a repetition penalty filtering method are devised, leading to the development of a hierarchical reinforcement learning algorithm based on a two-tier screening approach for potential subgoals. Finally, the feasibility of the proposed method is validated through ablation experiments and visualized simulation studies. Simulation results demonstrate that the presented method offers some reference value for research in intelligent decision-making for unmanned operations and can be applied to innovative studies in related response strategies.