In this article, a deep reinforcement learning-based path-following control scheme is established for an under-actuated autonomous marine vehicle (AMV) in the presence of model uncertainties and unknown marine environment disturbances is presented. By virtue of light-of-sight guidance, a surge-heading joint guidance method is developed within the kinematic level, thereby enabling the AMV to follow the desired path accurately. Within the dynamic level, model uncertainties and time-varying environment disturbances are taken into account, and the reinforcement learning control method using the twin-delay deep deterministic policy gradient (TD3) is developed for the under-actuated vehicle, where path-following actions are generated via the state space and hybrid rewards. Additionally, actor-critic networks are developed using the long-short time memory (LSTM) network, and the vehicle can successfully make a decision by the aid of historical states, thus enhancing the convergence rate of dynamic controllers. Simulation results and comprehensive comparisons on a prototype AMV demonstrate the remarkable effectiveness and superiority of the proposed LSTM-TD3-based path-following control scheme.

With the rapid development of perception, decision-making, and control technologies, pursuit–evasion (PE) games with unmanned surface vehicles (USVs) have become an interesting research topic in military implementations and civilian areas. In this paper, we provide an overview of recent advances in the PE games with USVs. First, the motion model of USVs and successful criteria for PE games are presented. Next, some challenging issues in PE games with USVs are briefly discussed. Then, recent results on one-pursuer one-evader, multiple-pursuer one-evader, and multiple-pursuer multiple-evader with USVs are reviewed in detail. Finally, several theoretical and technical issues are suggested to direct future research, including target prediction, dynamic task allocation, brain-inspired decision-making, safe control, and PE experiments.

Autonomous surface vehicles (ASVs) have been widely applied in ocean engineering due to their small size, low cost, and high mobility. However, more relevant encirclement control methods with many-to-one are simple and do not consider the system dynamics. This article proposes a cooperative encirclement control method for ASVs against multiple targets based on multi-agent reinforcement learning. Firstly, a dynamic target allocation algorithm is designed based on location information of both vehicles and targets, enabling vehicles to select encirclement targets in real-time according to relative distances. Subsequently, the whole encirclement process is divided into multiple stages, and a multi-stage reward function is developed based on curriculum learning to guide ASVs in completing encirclement tasks progressively, from simpler to more complex scenarios. Then, the actor and critic networks incorporating long short-term memory are constructed, respectively, and a multi-agent soft actor-critic reinforcement learning algorithm is employed to train ASVs, enhancing cooperative target encirclement maneuvers. Finally, the effectiveness and superiority of the proposed method is validated through a six-on-two encirclement simulation.

In recent years, the field of autonomous marine vehicles has undergone remarkable advancements, with unmanned surface vehicles (USVs) and unmanned underwater vehicles (UUVs) demonstrating transformative potential for oceanographic exploration and marine applications

The world has witnessed the rapid development of autonomous marine vehicles, such as surface vehicles and underwater vehicles, which have created fruitful innovative approaches to previously unsolvable problems in marine and ocean engineering