The current research into decision-making strategies for air combat focuses on the performance of algorithms, while the selection of actions is often ignored, and the actions are often fixed in amplitude and limited in number in order to improve the convergence efficiency, making the strategy unable to give full play to the maneuverability of the aircraft. In this paper, a decision-making strategy for close-range air combat based on reinforcement learning with variable-scale actions is proposed; the actions are the variable-scale virtual pursuit angles and speeds. Firstly, a trajectory prediction method consisting of a real-time prediction, correction, and judgment of errors is proposed. The back propagation (BP) neural network and the long and short term memory (LSTM) neural network are used as base prediction network and correction prediction network, respectively. Secondly, the past, current, and future positions of the target aircraft are used as virtual pursuit points, and they are converted into virtual pursuit angles as the track angle commands using angle guidance law. Then, the proximity policy optimization (PPO) algorithm is applied to train the agent. The simulation results show that the attacking aircraft that uses the strategy proposed in this paper has a higher win rate during air combat and the attacking aircraft’s maneuverability is fully utilized.

The upset state is an unexpected flight state, which is characterized by an unintentional deviation from normal operating parameters. It is difficult for the pilot to recover the aircraft from the upset state accurately and quickly. In this paper, an upset recovery strategy and pilot assistance system (PAS) based on reinforcement learning is proposed. The man–machine closed-loop system was established and the upset state, such as a high angle of attack and large attitude angle, was induced. The upset recovery problem was transformed into a sequential decision problem, and the Markov decision model of upset recovery was established by taking the deflection change of the control surface as the action. The proximal policy optimization (PPO) algorithm was selected for the strategy training. The adaptive pilot model and the reinforcement learning method proposed in this paper were used to make the aircraft recover from the upset state. Based on the correspondence between the flight state, the recovery method, and the recovery result, the aircraft upset recovery safety envelopes were formed, and the four-level upset recovery PAS with alarm warning, coordinated control, and autonomous recovery modes was constructed. The results of the digital virtual flight simulation and ground flight test show that compared with a traditional single pilot, the aircraft upset recovery strategy, the upset recovery safety envelopes, and the PAS established in this study could reduce the handling burden of the pilot and improve the success rate and effect of upset recovery. This research has certain theoretical reference values for flight safety and pilot training.