With the development of artificial intelligence and integrated sensor technologies, unmanned aerial vehicles (UAVs) are more and more applied in the air combats. A bottleneck that constrains the capability of UAVs against manned vehicles is the autonomous maneuver decision, which is a very challenging problem in the short-range air combat undergoing highly dynamic and uncertain maneuvers of enemies. In this paper, an autonomous maneuver decision model is proposed for the UAV short-range air combat based on reinforcement learning, which mainly includes the aircraft motion model, one-to-one short-range air combat evaluation model and the maneuver decision model based on deep Q network (DQN). However, such model includes a high dimensional state and action space which requires huge computation load for DQN training using traditional methods. Then, a phased training method, called "basic-confrontation", which is based on the idea that human beings gradually learn from simple to complex is proposed to help reduce the training time while getting suboptimal but efficient results. Finally, one-to-one short-range air combats are simulated under different target maneuver policies. Simulation results show that the proposed maneuver decision model and training method can help the UAV achieve autonomous decision in the air combats and obtain an effective decision policy to defeat the opponent.

UAV trajectory prediction is the core technology for autonomous UAV flight and is a prerequisite for control and navigation. In this paper, the UAV flight path prediction model is established by collecting the flight data of the actual UAV. Firstly, the UAV flight information collection and data preprocessing are carried out; Secondly, the UAV flight state recognition model is established based on the PCA-SVM model to identify five UAV flight states; Finally, the flight path prediction model of UAV based on flight state recognition is established, and the neural network model is established based on the flight path of five flight state recognition. The experimental results show that: 1) The accuracy of UAV flight state recognition based on PCA-SVM is more than 90%. 2) The average prediction error of the traditional neural network UAV trajectory is 0.422m, and the maximum error of the circling state is 0.84m. 3) The average prediction error of the UAV flight path based on flight state recognition is 0.214m, and the maximum error of the circling state is 0.41m. The model error is less than 0.5m. The results show that the prediction model with flight state recognition has significantly less error than the direct UAV trajectory prediction, and the prediction model with flight state recognition predicts better than the traditional Unscented Kalman Filter method.

With the rapid changes in the battlefield situation, the requirement of time for UAV groups to deal with complex tasks is getting higher, which puts forward higher requirements for the dynamic allocation of the UAV group. However, most of the existing methods focus on task pre-allocation, and the research on dynamic task allocation technology during task execution is not sufficient. Aiming at the high real-time requirement of the multi-UAV collaborative dynamic task allocation problem, this paper introduces the market auction mechanism to design a discrete particle swarm algorithm based on particle quality clustering by a hybrid architecture. The particle subpopulations are dynamically divided based on particle quality, which changes the topology of the algorithm. The market auction mechanism is introduced during particle initialization and task coordination to build high-quality particles. The algorithm is verified by constructing two emergencies of UAV sudden failure and a new emergency task.

In response to the challenges of estimating missile launch timing during close-range unmanned autonomous air combat in the future, this paper proposes an autonomous attack decision-making method based on hierarchical virtual Bayesian reinforcement learning (HVBRL). First, a six-degree-of-freedom (6-DOF) high-fidelity aircraft dynamics model along with missile dynamics and guidance rate models are constructed. Second, the HVBRL algorithm is introduced, where the low-level algorithm outputs control parameters and the high-level algorithm generates control commands. Given that the number of missile hits on a target under specific conditions follows a binomial distribution, a simple prior knowledge can be introduced through its conjugate prior, the Beta distribution, to avoid prolonged exploration of ineffective areas. Moreover, carrying only a limited number of missiles and predicting the number of hits by multiple virtual missiles in specific states through a neural network circumvents the computational complexity issue associated with carrying an excessive number of missiles. Finally, this paper presents the low-level training algorithm, the high-level training algorithm, and the high-level self-play training algorithm. Experimental results show that our method significantly reduces the simulation computational complexity. Compared with the Monte Carlo method carrying 1000 missiles, the simulation speed of the high-level training algorithm is increased by 32.75 times, and that of the high-level self-play algorithm is increased by 23 times. Moreover, the estimated missile hit probability with bias can effectively guide the timing of missile launches in close-range air combat, which has significant implications for intelligent autonomous air combat decision-making and operational analysis.

Intercepting a saturation attack from a loitering munition (LM) swarm with a swarm of unmanned aerial vehicles (UAVs) is a complex challenge. In this study, we developed a specific intercept mission simulation environment and proposed an approach based on the Gate Recurrent Unit Mean-Field Deep Deterministic Policy Gradient (GRU-MFDDPG) algorithm to tackle this issue. Based on the battle situation of both sides, we constructed a target assignment model for the UAV swarm based on the extensible Hungarian algorithm. To address the operational characteristics of large-scale UAV swarm interception missions, a partially observable mean-field game theory was integrated to modify the DDPG algorithm, with a Gate Recurrent Unit (GRU) incorporated to predict the movements of incoming loitering munitions accurately. The algorithm was trained under a “centralized training, distributed execution” framework. Simulation results demonstrate that this approach significantly enhances the interception success rate of the UAV swarm against large-scale loitering munitions.

Recently, developing multi-UAVs to cooperatively pursue a fast-moving target has become a research hotspot in the current world. Although deep reinforcement learning (DRL) has made a lot of achievements in the UAV pursuit game, there are still some problems such as high-dimensional parameter space, the ease of falling into local optimization, the long training time, and the low task success rate. To solve the above-mentioned issues, we propose an improved twin delayed deep deterministic policy gradient algorithm combining the genetic algorithm and maximum mean discrepancy method (GM-TD3) for multi-UAV cooperative pursuit of high-speed targets. Firstly, this paper combines GA-based evolutionary strategies with TD3 to generate action networks. Then, in order to avoid local optimization in the algorithm training process, the maximum mean difference (MMD) method is used to increase the diversity of the policy population in the updating process of the population parameters. Finally, by setting the sensitivity weights of the genetic memory buffer of UAV individuals, the mutation operator is improved to enhance the stability of the algorithm. In addition, this paper designs a hybrid reward function to accelerate the convergence speed of training. Through simulation experiments, we have verified that the training efficiency of the improved algorithm has been greatly improved, which can achieve faster convergence; the successful rate of the task has reached 95%, and further validated UAVs can better cooperate to complete the pursuit game task.