Many achievements toward unmanned surface vehicles have been made using artificial intelligence theory to assist the decisions of the navigator. In particular, there has been rapid development in autonomous collision avoidance techniques that employ the intelligent algorithm of deep reinforcement learning. A novel USV collision avoidance algorithm based on deep reinforcement learning theory for real-time maneuvering is proposed. Many improvements toward the autonomous learning framework are carried out to improve the performance of USV collision avoidance, including prioritized experience replay, noisy network, double learning, and dueling architecture, which can significantly enhance the training effect. Additionally, considering the characteristics of the USV collision avoidance problem, two effective methods to enhance training efficiency are proposed. For better training, considering the international regulations for preventing collisions at sea and USV maneuverability, a complete and reliable USV collision avoidance training system is established, demonstrating an efficient learning process in complex encounter situations. A reward signal system in line with the USV characteristics is designed. Based on the Unity maritime virtual simulation platform, an abundant simulation environment for training and testing is designed. Through detailed analysis, verification, and comparison, the improved algorithm outperforms the pre-improved algorithm in terms of stability, average reward, rules learning, and collision avoidance effect, reducing 26.60% more accumulated course deviation and saving 1.13% more time.

The detection of unknown waters has been studied and applied in various fields, such as national defense, military operations, engineering surveying and mapping, and scene reconstruction. To improve exploration efficiency in unknown waters, this paper proposes a framework for autonomous exploration using unmanned surface vehicles (USVs). This framework, comprising a multi-stage exploration strategy and a hierarchical navigation strategy, is designed to mitigate the inherent restrictions between the exploration target point and exploration direction in USV operations. These two strategies are optimized for the exploration target point and feasible navigation route to address the problem of the USV’s limited mobility during exploration. Rapidly exploring random tree (RRT) and boundary detection methods are used in the local layer to find the boundary in front of and behind the USV, and the gain of the target point is optimized. The hierarchical navigation method is implemented in the global layer to plan appropriate navigation paths. The proposed method is tested in simulations in several virtual environments and contrasted with the conventional methods currently in use. The findings indicate that our strategy covers more ground more effectively than other methods (our method achieved an exploration efficiency ranging from 4.9 to 5.3 $m^2$/s, whereas traditional methods ranged from 2.3 to 3.9 $m^2$/s, which demonstrates that our approach can improve exploration efficiency by up to 200% compared to traditional methods), spending less time exploring while significantly reducing collision probability.

This paper addresses the collision avoidance of the underactuated unmanned surface vehicles (USVs) in the environment of random static obstacles based on an improved deep deterministic policy gradient (DDPG) algorithm. An improved algorithm based priority experience replay (PER) is developed to control the USV. Furthermore, the design of a marine environment includes obstacles with random initial positions. Moreover, the USV is modeled with the Norrbin model and the reward function set complies with the model and requirements of collision avoidance. The advantages of the developed collision avoidance scheme are that first, it could effectively prevent the emergence of paths that do not satisfy the actual navigation requirements of the USV and the PER mechanism can improve the USV collision avoidance accuracy rate; second, the designed environment is highly random, which is crucial for solving the problem that the method lacks practical usefulness. In addition, the proposed reward function optimizes the USV's trajectory so that it satisfies the requirements for both collision avoidance and appropriate USV motion. These features make the algorithm more useful. Finally, simulation and comparative experiments verify the effectiveness of the algorithm.