This paper investigates the path planning problem of an unmanned aerial vehicle (UAV) for completing a raid mission through ultra-low altitude flight in complex environments. The UAV needs to avoid radar detection areas, low-altitude static obstacles, and low-altitude dynamic obstacles during the flight process. Due to the uncertainty of low-altitude dynamic obstacle movement, this can slow down the convergence of existing algorithm models and also reduce the mission success rate of UAVs. In order to solve this problem, this paper designs a state detection method to encode the environmental state of the UAV’s direction of travel and compress the environmental state space. In considering the continuity of the state space and action space, the SD-TD3 algorithm is proposed in combination with the double-delayed deep deterministic policy gradient algorithm (TD3), which can accelerate the training convergence speed and improve the obstacle avoidance capability of the algorithm model. Further, to address the sparse reward problem of traditional reinforcement learning, a heuristic dynamic reward function is designed to give real-time rewards and guide the UAV to complete the task. The simulation results show that the training results of the SD-TD3 algorithm converge faster than the TD3 algorithm, and the actual results of the converged model are better.

Unmanned helicopter (UH) is often utilized for raid missions because it can evade radar detection by flying at ultra-low altitudes. Path planning is the key technology to realizing the autonomous action of UH. On the one hand, the dynamically changing radar coverage area and the mountains in the low airspace environment will seriously affect the flight safety of UH. On the other hand, the huge state space of the three-dimensional (3D) environment will also make traditional algorithms difficult to converge. To address the above problems, a memory-enhanced dueling deep Q-network (ME-dueling DQN) algorithm was proposed. First, a comprehensive reward function was designed, which can guide the algorithm to converge quickly and effectively improve the sparse reward problem. Then, we introduced a dual memory pool structure and proposed a memory-enhanced mechanism, which can reduce invalid exploration, further improve the learning efficiency of the algorithm, and make the algorithm more stable. Finally, the path planning ability of the proposed algorithm in multiple experimental environments was verified. Experiments showed that the proposed algorithm has good environmental adaptability and can help UH to accurately identify dangerous areas and plan a safe and reliable flight path.