Aiming at the non-cognitive environment such as dynamic and static obstacles and external interference, a real-time collision avoidance strategy based on flexible steering method under the framework of deep reinforcement learning (RL) is proposed. The strategy extracts effective state features required for obstacle avoidance by developing an efficient multibeam sonar detection information coding module, and integrates the safety and energy cost as the induced factors for online control decisions. Meanwhile, a novel sampling priority scheme is introduced to assist UUV to obtain a better obstacle avoidance capability in response to the problems of low input state coupling and sample efficiency in the training environment. In order to achieve the application of the control strategy obtained from simulation training to real UUV, a disturbance estimation and cancellation method based on dynamic compensation is designed. Finally, the effectiveness of the proposed strategy is confirmed by experiments in different complex obstacle environments.

The control strategy of an underdriven unmanned underwater vehicle (UUV) equipped with front sonar and actuator faults in a continuous task environment is investigated. Considering trajectory tracking and local path planning in complex-obstacle environments, we propose a task transition strategy under the event-triggered mechanism and design the corresponding state space and action space for the trajectory tracking task under the deep reinforcement learning framework. Meanwhile, a feed-forward compensation mechanism is designed to counteract the effects of external disturbances and actuator faults in combination with a reduced-order extended state observer. For the path planning task under the rapidly exploring random tree (RRT) framework, a reward component and angular factors are introduced to optimize the growth and exploration points of the extended tree under the consideration of the shortest distance, optimal energy consumption, and steering angle constraints. The effectiveness of the proposed method was verified through continuous task simulations of trajectory tracking and local path planning.

Autonomous collision avoidance for unmanned underwater vehicles (UUVs) in uncertain environments requires real-time decision-making under vehicle dynamic constraints; when multibeam sonar is used, additional sensing constraints arise from raw sonar feedback and its encoding. Deep reinforcement learning (DRL) holds promise for addressing this challenge, as it can effectively characterize complex sensor detection feedback and rapidly generate mappings from high-dimensional state spaces to collision avoidance control commands. A further challenge arises from the control signal oscillations associated with the DRL policies, which makes the direct deployment of obstacle avoidance policies on UUVs difficult and leads to excessive wear on the actuator system. To address these challenges, an improved controller based on twin-delayed deep deterministic policy gradient algorithm (TD3) is proposed. This controller embeds a feedback signal encoding module adapted to the detection characteristics of 3-D point-cloud multibeam sonar and incorporates an event-triggered regularization method to combat control signal oscillations. In addition, an adaptive obstacle avoidance training and deployment framework is devised, relying on a discrete sonar-encoding scheme rather than explicit obstacle distance/angle features. Lake experiments validate the sonar perception-based encoding module, while collision-avoidance performance is validated in high-fidelity simulations configured from the detection scenarios.