The game of pursuit–evasion has always been a popular research subject in the field of Unmanned Aerial Vehicles (UAVs). Current evasion decision making based on reinforcement learning is generally trained only for specific pursuers, and it has limited performance for evading unknown pursuers and exhibits poor generalizability. To enhance the ability of an evasion policy learned by reinforcement learning (RL) to evade unknown pursuers, this paper proposes a pursuit UAV attitude estimation and pursuit strategy identification method and a Model Reference Policy Adaptation (MRPA) algorithm. Firstly, this paper constructs a Markov decision model for the pursuit–evasion game of UAVs that includes the pursuer’s attitude and trains an evasion policy for a specific pursuit strategy using the Soft Actor–Critic (SAC) algorithm. Secondly, this paper establishes a novel relative motion model of UAVs in pursuit–evasion games under the assumption that proportional guidance is used as the pursuit strategy, based on which the pursuit UAV attitude estimation and pursuit strategy identification algorithm is proposed to provide adequate information for decision making and policy adaptation. Furthermore, a Model Reference Policy Adaptation (MRPA) algorithm is presented to improve the generalizability of the evasion policy trained by RL in certain environments. Finally, various numerical simulations imply the precision of pursuit UAV attitude estimation and the accuracy of pursuit strategy identification. Also, the ablation experiment verifies that the MRPA algorithm can effectively enhance the performance of the evasion policy to deal with unknown pursuers.

In recent years, UAVs have faced increasingly severe and diversified missile threats. To address the challenge that reinforcement learning-based missile evasion algorithms struggle to adapt to various unknown missile types, we introduce a risk-sensitive PPO algorithm and propose a training framework incorporating multi-head attention mechanisms and dual-population adversarial training. The multi-head attention mechanism enables the policy network to extract latent features such as missile guidance laws from state sequences, while the dual-population adversarial approach ensures policy diversity and robustness. Compared to conventional self-play methods and GRU-based evasion strategies, our method demonstrates superior training efficiency and generates evasion policies with better adaptability to different missile types.