Numerous algorithms have been proposed for quadrotor flight control. Conventional methods require massive labor of parameter adjustment. Deep reinforcement learning (DRL) methods also need enormous computation and complicated hyperparameter tuning, since most of them regard the quadrotor dynamics as a black box. To overcome the drawbacks of both conventional and learning-based methods, this letter proposes a DRL-based cascade quadrotor flight controller. Under the small-angle restriction, the quadrotor dynamics are decomposed into six subsystems, each containing only one degree of freedom (DOF) for the agent to control. Six agents are sequentially trained to fully control the corresponding DOF without any prior knowledge of quadrotor dynamic parameters. Experiments show that, with a total training time of 17 min, the proposed controller could accomplish the fixed point tracking task with an error lower than 5 mm, rise time lower than 1.5 s, and peak value lower than 1%. The controller could also track time-variant trajectories.

Obstacle avoidance plays a crucial role in ensuring the safe path planning of quadrotor unmanned aerial vehicles (QUAVs). In this study, we propose a hierarchical framework for obstacle avoidance, which combines the use of artificial potential field (APF) and deep reinforcement learning (DRL) for training low-level motion controllers. Unlike traditional potential field methods, our approach modifies the state information received by the motion controllers using the outputs of the APF path planner. Specifically, the assumed target position is pushed away from obstacles, resulting in adjustments to the perceived position errors. Additionally, we address path oscillations by incorporating the target’s velocity information, which is calculated based on the time-derivative of the repulsive force. Experimental results have validated the effectiveness of our proposed framework in avoiding collisions with obstacles and reducing oscillations.

Much research has been conducted on unmanned aerial vehicles (UAVs) across various fields. Due to the obstacle-ridden environments, there is a strong need for a high-performance autonomous navigation system. Deep reinforcement learning (DRL) has gained significant attention from researchers as a navigation algorithm, but existing works have several limitations. Most assume that the UAV moves with discrete actions or simplified dynamics. Numerous efforts in grid mapping primarily focus on expanding obstacle coverage without addressing the dimensionality curse arising from the constrained processing capacity of the network in handling redundant information. Furthermore, the transferability of the trained agent has rarely been tested. To address these limitations, this paper proposes a multiscale surrounding state, where the proximal and distal surroundings are separately represented by exact and statistical information to balance state coverage and dimensionality. The truncation operation is also introduced to improve the transferability. Lastly, a modular UAV navigation system with a DRL path planner is constructed. The experimental results demonstrate that the proposed autonomous navigation system can function in various unknown environments.

This paper introduces the prioritized population play with diversified partners (P3DP) to augment the robustness of tactical maneuver decision-making in unmanned aerial combat vehicles (UACVs) during dogfights when facing a spectrum of opponent strategies. Population play where multiple agents exist during the training phase is utilized to facilitate the initialization. Prioritized opponent sampling, favoring agents with higher historical win rates, ensures that the training process focuses on more competitive and challenging scenarios, therefore augmenting the quality of data. Furthermore, a diversity-guided approach is implemented, where each agent maintains individual policy entropy while collectively optimizing the population entropy (PE). This method augments the diversity within the population, preventing policies from becoming identical. Additionally, a partner mechanism is introduced, where a primary agent disregards the PE to protect the optimality of its policy from diversity consideration. Meanwhile, other agents act as partners, supplying various opponent policies to increase data diversity. Experiments validate that P3DP surpasses other training methodologies, enormously augmenting policy robustness.