Numerous algorithms have been proposed for quadrotor flight control. Conventional methods require massive labor of parameter adjustment. Deep reinforcement learning (DRL) methods also need enormous computation and complicated hyperparameter tuning, since most of them regard the quadrotor dynamics as a black box. To overcome the drawbacks of both conventional and learning-based methods, this letter proposes a DRL-based cascade quadrotor flight controller. Under the small-angle restriction, the quadrotor dynamics are decomposed into six subsystems, each containing only one degree of freedom (DOF) for the agent to control. Six agents are sequentially trained to fully control the corresponding DOF without any prior knowledge of quadrotor dynamic parameters. Experiments show that, with a total training time of 17 min, the proposed controller could accomplish the fixed point tracking task with an error lower than 5 mm, rise time lower than 1.5 s, and peak value lower than 1%. The controller could also track time-variant trajectories.

Obstacle avoidance plays a crucial role in ensuring the safe path planning of quadrotor unmanned aerial vehicles (QUAVs). In this study, we propose a hierarchical framework for obstacle avoidance, which combines the use of artificial potential field (APF) and deep reinforcement learning (DRL) for training low-level motion controllers. Unlike traditional potential field methods, our approach modifies the state information received by the motion controllers using the outputs of the APF path planner. Specifically, the assumed target position is pushed away from obstacles, resulting in adjustments to the perceived position errors. Additionally, we address path oscillations by incorporating the target’s velocity information, which is calculated based on the time-derivative of the repulsive force. Experimental results have validated the effectiveness of our proposed framework in avoiding collisions with obstacles and reducing oscillations.

Much research has been conducted on unmanned aerial vehicles (UAVs) across various fields. Due to the obstacle-ridden environments, there is a strong need for a high-performance autonomous navigation system. Deep reinforcement learning (DRL) has gained significant attention from researchers as a navigation algorithm, but existing works have several limitations. Most assume that the UAV moves with discrete actions or simplified dynamics. Numerous efforts in grid mapping primarily focus on expanding obstacle coverage without addressing the dimensionality curse arising from the constrained processing capacity of the network in handling redundant information. Furthermore, the transferability of the trained agent has rarely been tested. To address these limitations, this paper proposes a multiscale surrounding state, where the proximal and distal surroundings are separately represented by exact and statistical information to balance state coverage and dimensionality. The truncation operation is also introduced to improve the transferability. Lastly, a modular UAV navigation system with a DRL path planner is constructed. The experimental results demonstrate that the proposed autonomous navigation system can function in various unknown environments.

Online action detection plays a vital role in video action understanding and can be widely used in various video analysis applications. This task aims to detect actions at the current moment within long untrimmed video streams. However, accurately identifying action-background transitions that are ambiguous in terms of time during detection can be challenging due to the similarity between the action and background clips, adding to the difficulty in finding a suitable division between them. To address this issue, we propose a hard video clip mining method based on deep metric learning for online action detection named HCM. The HCM method first selects video clips that are hard to distinguish to determine the optimization objects. Then, a hard clip mining loss is adopted to push the features toward the centers of the categories to which they belong and away from others. Furthermore, we introduce an intra-class feature compaction loss to constrain the divergence of action features, ensuring the stability of their distribution. We evaluated the proposed method on two challenging online action detection datasets, THUMOS14 and TVSeries. The results show that HCM is effective and efficient in online action detection and action anticipation tasks.

Obstacle avoidance is an essential part of mobile robot path planning, since it ensures the safety of automatic control. This paper proposes an obstacle avoidance algorithm that combines artificial potential field with deep reinforcement learning (DRL). State regulation is presented so that the pre-defined velocity constraint could be satisfied. To guarantee the isotropy of the robot controller as well as reduce training complexity, coordinate transformation into normal direction and tangent direction is introduced, making it possible to use one-dimension controllers to work in a two-dimension task. Artificial potential field (APF) is modified such that the obstacle directly affects the intermediate target positions instead of the control commands, which can well be used to guide the previously trained one-dimension DRL controller. Experiment results show that the proposed algorithm successfully achieved obstacle avoidance tasks in single-agent and multi-agent scenarios.