In this paper, a novel deep reinforcement learning (DRL) method, and robust deep deterministic policy gradient (Robust-DDPG), is proposed for developing a controller that allows robust flying of an unmanned aerial vehicle (UAV) in dynamic uncertain environments. This technique is applicable in many fields, such as penetration and remote surveillance. The learning-based controller is constructed with an actor-critic framework, and can perform a dual-channel continuous control (roll and speed) of the UAV. To overcome the fragility and volatility of original DDPG, three critical learning tricks are introduced in Robust-DDPG: (1) Delayed-learning trick, providing stable learnings, while facing dynamic environments; (2) adversarial attack trick, improving policy’s adaptability to uncertain environments; (3) mixed exploration trick, enabling faster convergence of the model. The training experiments show great improvement in its convergence speed, convergence effect, and stability. The exploiting experiments demonstrate high efficiency in providing the UAV a shorter and smoother path. While, the generalization experiments verify its better adaptability to complicated, dynamic and uncertain environments, comparing to Deep Q Network (DQN) and DDPG algorithms.

In recent years, unmanned aerial vehicles (UAVs) have been considered an ideal relay platform for enhancing the communication between ground agents, because they fly at high altitudes and are easy to deploy with strong adaptabilities. Their maneuvering allows them to adjust their location to optimize the performance of links, which brings out the relay UAV autonomous mobility control problem. This work addressed the problem in a novel scene with mobile agents and completely unknown wireless channel properties, using only online measured information of received signal strength (RSS) and agent positions. The problem is challenging because of the unknown and dynamic radio frequency (RF) environment cause by agents and UAV maneuvering. We present a framework for both end-to-end communication and multi-agent-inter communication applications, and focus on proposing: (1) least square estimation-based channel approximation with consideration of environment effects and, (2) gradient-based optimal relay position seeking. Simulation results show that considering the environmental effects on channel parameters is meaningful and beneficial in using UAV as relays for the communication of multiple ground agents, and validate that the proposed algorithms optimizes the network performance by controlling the heading of the UAV.