To solve the real-time complex mission-planning problem for Multiple heterogeneous Unmanned Aerial Vehicles (UAVs) in the dynamic environments, this paper addresses a new approach by effectively adapting the Consensus-Based Bundle Algorithms (CBBA) under the constraints of task timing, limited UAV resources, diverse types of tasks, dynamic addition of tasks, and real-time requirements. We introduce the dynamic task generation mechanism, which satisfied the task timing constraints. The tasks that require the cooperation of multiple UAVs are simplified into multiple sub-tasks to perform by a single UAV independently. We also introduce the asynchronous task allocation mechanism. This mechanism reduces the computational complexity of the algorithm and the communication time between UAVs. The partial task redistribution mechanism has been adopted for achieving the dynamic task allocation. The real-time performance of the algorithm is assured on the premise of optimal results. The feasibility and real-time performance of the algorithm are validated by conducting dynamic simulation experiments.

In response to the challenges of estimating missile launch timing during close-range unmanned autonomous air combat in the future, this paper proposes an autonomous attack decision-making method based on hierarchical virtual Bayesian reinforcement learning (HVBRL). First, a six-degree-of-freedom (6-DOF) high-fidelity aircraft dynamics model along with missile dynamics and guidance rate models are constructed. Second, the HVBRL algorithm is introduced, where the low-level algorithm outputs control parameters and the high-level algorithm generates control commands. Given that the number of missile hits on a target under specific conditions follows a binomial distribution, a simple prior knowledge can be introduced through its conjugate prior, the Beta distribution, to avoid prolonged exploration of ineffective areas. Moreover, carrying only a limited number of missiles and predicting the number of hits by multiple virtual missiles in specific states through a neural network circumvents the computational complexity issue associated with carrying an excessive number of missiles. Finally, this paper presents the low-level training algorithm, the high-level training algorithm, and the high-level self-play training algorithm. Experimental results show that our method significantly reduces the simulation computational complexity. Compared with the Monte Carlo method carrying 1000 missiles, the simulation speed of the high-level training algorithm is increased by 32.75 times, and that of the high-level self-play algorithm is increased by 23 times. Moreover, the estimated missile hit probability with bias can effectively guide the timing of missile launches in close-range air combat, which has significant implications for intelligent autonomous air combat decision-making and operational analysis.

The unmanned aerial vehicle (UAV) swarm technology is one of the research hotspots in recent years. With the continuous improvement of autonomous intelligence of UAV, the swarm technology of UAV will become one of the main trends of UAV development in the future. This paper studies the behavior decision-making process of UAV swarm rendezvous task based on the double deep Q network (DDQN) algorithm. We design a guided reward function to effectively solve the problem of algorithm convergence caused by the sparse return problem in deep reinforcement learning (DRL) for the long period task. We also propose the concept of temporary storage area, optimizing the memory playback unit of the traditional DDQN algorithm, improving the convergence speed of the algorithm, and speeding up the training process of the algorithm. Different from traditional task environment, this paper establishes a continuous state-space task environment model to improve the authentication process of UAV task environment. Based on the DDQN algorithm, the collaborative tasks of UAV swarm in different task scenarios are trained. The experimental results validate that the DDQN algorithm is efficient in terms of training UAV swarm to complete the given collaborative tasks while meeting the requirements of UAV swarm for centralization and autonomy, and improving the intelligence of UAV swarm collaborative task execution. The simulation results show that after training, the proposed UAV swarm can carry out the ren-dezvous task well, and the success rate of the mission reaches 90%.

The deep deterministic policy gradient (DDPG) algorithm is an off-policy method that combines two mainstream reinforcement learning methods based on value iteration and policy iteration. Using the DDPG algorithm, agents can explore and summarize the environment to achieve autonomous decisions in the continuous state space and action space. In this paper, a cooperative defense with DDPG via swarms of unmanned aerial vehicle (UAV) is developed and validated, which has shown promising practical value in the effect of defending. We solve the sparse rewards problem of reinforcement learning pair in a long-term task by building the reward function of UAV swarms and optimizing the learning process of artificial neural network based on the DDPG algorithm to reduce the vibration in the learning process. The experimental results show that the DDPG algorithm can guide the UAVs swarm to perform the defense task efficiently, meeting the requirements of a UAV swarm for non-centralization, autonomy, and promoting the intelligent development of UAVs swarm as well as the decision-making process.

Intercepting a saturation attack from a loitering munition (LM) swarm with a swarm of unmanned aerial vehicles (UAVs) is a complex challenge. In this study, we developed a specific intercept mission simulation environment and proposed an approach based on the Gate Recurrent Unit Mean-Field Deep Deterministic Policy Gradient (GRU-MFDDPG) algorithm to tackle this issue. Based on the battle situation of both sides, we constructed a target assignment model for the UAV swarm based on the extensible Hungarian algorithm. To address the operational characteristics of large-scale UAV swarm interception missions, a partially observable mean-field game theory was integrated to modify the DDPG algorithm, with a Gate Recurrent Unit (GRU) incorporated to predict the movements of incoming loitering munitions accurately. The algorithm was trained under a “centralized training, distributed execution” framework. Simulation results demonstrate that this approach significantly enhances the interception success rate of the UAV swarm against large-scale loitering munitions.

Recently, developing multi-UAVs to cooperatively pursue a fast-moving target has become a research hotspot in the current world. Although deep reinforcement learning (DRL) has made a lot of achievements in the UAV pursuit game, there are still some problems such as high-dimensional parameter space, the ease of falling into local optimization, the long training time, and the low task success rate. To solve the above-mentioned issues, we propose an improved twin delayed deep deterministic policy gradient algorithm combining the genetic algorithm and maximum mean discrepancy method (GM-TD3) for multi-UAV cooperative pursuit of high-speed targets. Firstly, this paper combines GA-based evolutionary strategies with TD3 to generate action networks. Then, in order to avoid local optimization in the algorithm training process, the maximum mean difference (MMD) method is used to increase the diversity of the policy population in the updating process of the population parameters. Finally, by setting the sensitivity weights of the genetic memory buffer of UAV individuals, the mutation operator is improved to enhance the stability of the algorithm. In addition, this paper designs a hybrid reward function to accelerate the convergence speed of training. Through simulation experiments, we have verified that the training efficiency of the improved algorithm has been greatly improved, which can achieve faster convergence; the successful rate of the task has reached 95%, and further validated UAVs can better cooperate to complete the pursuit game task.