This study proposed the 3D path planning of an autonomous underwater vehicle (AUV) by using the hierarchical deep Q network (HDQN) combined with the prioritized experience replay. The path planning task was divided into three layers, which realized the dimensionality reduction of state space and solved the problem of dimension disaster. An artificial potential field was used to design the positive rewards of the algorithm to shorten the training time. According to the different requirements of the task, this study modified the rewards in the training process to obtain different paths. The path planning simulation and field tests were carried out. The results of the tests corroborated that the training time of the proposed method was shorter than that of the traditional method. The path obtained by simulation training was proved to be safe and effective.

In recent years, the research of multiple unmanned surface vessels collaboration has received great attention. More and more researchers have proposed different methods of multiple unmanned surface vessels collaboration, such as cooperative collision avoidance, formation, and rendezvous. Based on the significant advantages of biological swarm intelligence applications in these collaborative methods, this article summarizes the research methods of multiple unmanned surface vessels collaborative search and hunting from the perspective of swarm intelligence. First of all, this article summarizes the key technologies of multiple unmanned surface vessels collaborative search and hunting from the aspects of the multi-robot system, group communication, environment modeling, collaboration mechanism, and path planning. Then, it reviews some classic swarm intelligence algorithms, analyzes the advantages and disadvantages of these algorithms, and proposes optimization directions for existing disadvantages based on relevant literature. Finally, the article points out some existing problems in every stage and suggestions for future research.

This research aims to solve the safe navigation problem of autonomous underwater vehicles (AUVs) in deep ocean, which is a complex and changeable environment with various mountains. When an AUV reaches the deep sea navigation, it encounters many underwater canyons, and the hard valley walls threaten its safety seriously. To solve the problem on the safe driving of AUV in underwater canyons and address the potential of AUV autonomous obstacle avoidance in uncertain environments, an improved AUV path planning algorithm based on the deep deterministic policy gradient (DDPG) algorithm is proposed in this work. This method refers to an end-to-end path planning algorithm that optimizes the strategy directly. It takes sensor information as input and driving speed and yaw angle as outputs. The path planning algorithm can reach the predetermined target point while avoiding large-scale static obstacles, such as valley walls in the simulated underwater canyon environment, as well as sudden small-scale dynamic obstacles, such as marine life and other vehicles. In addition, this research aims at the multi-objective structure of the obstacle avoidance of path planning, modularized reward function design, and combined artificial potential field method to set continuous rewards. This research also proposes a new algorithm called deep SumTree-deterministic policy gradient algorithm (SumTree-DDPG), which improves the random storage and extraction strategy of DDPG algorithm experience samples. According to the importance of the experience samples, the samples are classified and stored in combination with the SumTree structure, high-quality samples are extracted continuously, and SumTree-DDPG algorithm finally improves the speed of the convergence model. Finally, this research uses Python language to write an underwater canyon simulation environment and builds a deep reinforcement learning simulation platform on a high-performance computer to conduct simulation learning training for AUV. Data simulation verified that the proposed path planning method can guide the under-actuated underwater robot to navigate to the target without colliding with any obstacles. In comparison with the DDPG algorithm, the stability, training’s total reward, and robustness of the improved Sumtree-DDPG algorithm planner in this study are better.

Due to the complex underwater environment, autonomous underwater vehicles (AUVs) may fail due to unknown obstacles in the process of performing tasks. Therefore, studying the real-time obstacle detection and avoidance methods for AUVs based on forward-looking sonar is very important. This article proposes a set of obstacle avoidance algorithms based on the images collected by the multibeam forward-looking sonar carried by AUVs. First, a new obstacle detection network based on a dual-path feature network based on a vision transformer (ViT-DPFN), which combines a convolutional neural network (CNN) and transformer, is proposed to detect obstacles in sonar images quickly and accurately. Then, combined with the current attitude data of AUVs, the obstacle position collected by the sonar image is corrected. Finally, using the fixed obstacle size and position data, the cost function is established based on the model predictive control (MPC) method, the constraint conditions are designed, and the external collision avoidance constraint is established so that AUVs can avoid obstacles in the trajectory tracking process. The simulation and field experiment results show that the proposed method improves the obstacle detection accuracy and processing speed and ensures the navigation safety of AUVs in complex obstacle environments, proving the proposed method’s advancement and effectiveness. The underwater acoustic target detection (UATD) dataset is available at https://github.com/zhenghaotianHEU/UATD-sonar-dataset .

In this paper, the three-dimensional (3D) path tracking control of an autonomous underwater vehicle (AUV) under the action of sea currents was researched. A novel reward function was proposed to improve learning ability and a disturbance observer was developed to observe the disturbance caused by currents. Based on existing models, the dynamic and kinematic models of the AUV were established. Deep Deterministic Policy Gradient, a deep reinforcement learning, was employed for designing the path tracking controller. Compared with the backstepping sliding mode controller, the controller proposed in this article showed excellent performance, at least in the particular study developed in this article. The improved reward function and the disturbance observer were also found to work well with improving path tracking performance.