How to operate an unmanned aerial vehicle (UAV) safely and efficiently in an interactive environment is challenging. A large amount of research has been devoted to improve the intelligence of a UAV while performing a mission, where finding an optimal maneuver decision-making policy of the UAV has become one of the key issues when we attempt to enable the UAV autonomy. In this paper, we propose a maneuver decision-making algorithm based on deep reinforcement learning, which generates efficient maneuvers for a UAV agent to execute the airdrop mission autonomously in an interactive environment. Particularly, the training set of the learning algorithm by the Prioritized Experience Replay is constructed, that can accelerate the convergence speed of decision network training in the algorithm. It is shown that a desirable and effective maneuver decision-making policy can be found by extensive experimental results.

Autonomous umanned aerial vehicle (UAV) manipulation is necessary for the defense department to execute tactical missions given by commanders in the future unmanned battlefield. A large amount of research has been devoted to improving the autonomous decision-making ability of UAV in an interactive environment, where finding the optimal maneuvering decision-making policy became one of the key issues for enabling the intelligence of UAV. In this paper, we propose a maneuvering decision-making algorithm for autonomous air-delivery based on deep reinforcement learning under the guidance of expert experience. Specifically, we refine the guidance towards area and guidance towards specific point tasks for the air-delivery process based on the traditional air-to-surface fire control methods. Moreover, we construct the UAV maneuvering decision-making model based on Markov decision processes (MDPs). Specifically, we present a reward shaping method for the guidance towards area and guidance towards specific point tasks using potential-based function and expert-guided advice. The proposed algorithm could accelerate the convergence of the maneuvering decision-making policy and increase the stability of the policy in terms of the output during the later stage of training process. The effectiveness of the proposed maneuvering decision-making policy is illustrated by the curves of training parameters and extensive experimental results for testing the trained policy.

Autonomous flight is a critical capability for unmanned aerial vehicles (UAVs), enabling applications in wildlife and plant protection, infrastructure inspection, search and rescue, and other complex missions. Although some learning-based methods have achieved considerable progress, traditional algorithms still struggle with real-world challenges, due to the partially observable nature of environments and limited experience regarding the properties of dynamic unknown environments where threats and targets are movable and unpredictable. To address these difficulties, it is necessary to achieve autonomous guidance for UAVs performing long-range missions in dynamic environments (LRGDEs), and to develop a novel end-to-end algorithm that can overcome partial observability under limited state transitions. In this paper, we propose an RNN-enhanced Diverse Curriculum-driven Learning Algorithm (REDCRL) based on deep reinforcement learning. We modify the structure of traditional actor–critic networks and introduce Bi-LSTM into policy networks (referred to as Bi-LSTM-modified Policy Networks (BLPNs)) to alleviate observation incompleteness. Furthermore, to fully exploit the potential value of data and mitigate the problem of insufficient samples, we develop an Adaptive Multi-Feature Evaluation Experience Replay (AMFER) method to reshape the process of experience replay buffer construction and sampling. In addition, the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm is adopted to optimize UAV-maneuver decision policies. Compared with traditional algorithms, the proposed algorithm can accelerate policy convergence and improve the performance of the trained policy.

It’s difficult for traditional solutions consisting of path planning and trajectory tracking algorithms to solve UAV maneuvering decision-making problem while executing long-distance UAV airdrop mission, because it’s necessary for obtaining an available path to prepare detailed information about mission area, such as digital elevation model, mountain, and other all possible threats. Therefore, it’s difficult for traditional methods to achieve UAV guidance in an interactive environment with an uncertain number of threats. An end-to-end algorithm is needed to perform long-distance UAV airdrop mission, where finding an optimal maneuvering decision-making policy becomes one of the key issues for improving the autonomy of UAV. In this paper, an experience knowledge driven learning algorithm based on deep reinforcement learning for long-distance UAV airdrop is proposed, which introduces expert experience to modify reward function for fully utilizing the potential value of experience transitions. Moreover, the training scheme of UAV maneuvering policy is designed based on Twin Delayed Deep Deterministic Policy Gradient with Prioritized Experience Replay (PER-TD3). Particularly, the training scheme incorporates Curriculum Learning (CL) to accelerate the convergence of policy. Compared with the traditional reward model without the inspiration of experience knowledge, our proposed algorithm could accelerate the convergence of policy and improve the performance of trained policy.

Multi-UAV multi-sensor cooperative detection is critical for situational awareness in complex environments with stationary known targets. To overcome the shortcomings of existing task allocation models in fine-grained cooperation, stealth constraints, and large-scale optimization, this paper proposes a new offline mission pre-planning method for static target scenarios. A five-layer hierarchical resource scheduling model is established, covering task, UAV platform, sensor type, cooperative mode, and execution role. Stealth requirements are incorporated as hard constraints, and a bi-objective optimization model is formulated to maximize weighted detection efficacy. A statistical adaptive genetic algorithm (SAGA) is developed as an offline pre-planning algorithm, featuring a coupled chromosome encoding strategy that integrates the five decision layers and an adaptive crossover-mutation mechanism guided by the Wilcoxon signed-rank test with effect size monitoring. Simulation results across small, medium, and large scenarios show that the SAGA achieves the highest mean best fitness in all scenarios. Compared with the GA, the improvement is statistically significant in the large-scale scenario, while in small- and medium-scale scenarios the differences from the GA and AGA are not significant. Compared with an improved whale optimization algorithm, the SAGA achieves statistically significantly higher best fitness and shorter computational time in all scenarios and converges significantly faster in small- and medium-scale scenarios. However, the SAGA requires longer computation time than the GA and AGA due to the per-generation statistical test. All allocations strictly satisfy the prescribed stealth constraints in the simulation model. The proposed method provides a novel, quantitatively validated solution for fine-grained, constraint-aware, and computationally efficient multi-UAV multi-sensor cooperative detection mission pre-planning.