The high performance and efficiency of multiple unmanned surface vehicles (multi-USV) promote the further civilian and military applications of coordinated USV. As the basis of multiple USVs’ cooperative work, considerable attention has been spent on developing the decentralized formation control of the USV swarm. Formation control of multiple USV belongs to the geometric problems of a multi-robot system. The main challenge is the way to generate and maintain the formation of a multi-robot system. The rapid development of reinforcement learning provides us with a new solution to deal with these problems. In this paper, we introduce a decentralized structure of the multi-USV system and employ reinforcement learning to deal with the formation control of a multi-USV system in a leader–follower topology. Therefore, we propose an asynchronous decentralized formation control scheme based on reinforcement learning for multiple USVs. First, a simplified USV model is established. Simultaneously, the formation shape model is built to provide formation parameters and to describe the physical relationship between USVs. Second, the advantage deep deterministic policy gradient algorithm (ADDPG) is proposed. Third, formation generation policies and formation maintenance policies based on the ADDPG are proposed to form and maintain the given geometry structure of the team of USVs during movement. Moreover, three new reward functions are designed and utilized to promote policy learning. Finally, various experiments are conducted to validate the performance of the proposed formation control scheme. Simulation results and contrast experiments demonstrate the efficiency and stability of the formation control scheme.
Multi-robot cooperative patrolling systems have been extensively employed in the civilian and military fields, including monitoring forest fires, marine search-and-rescue, and area patrol. Multi-robot area patrol problems refer to the activity that a team of robots works cooperatively and regularly to visit the key targets in the given area for security. Following consideration of the low cost and high safety of unmanned surface vehicles (USV), a team of USVs is organized to perform area patrol in a sophisticated maritime environment. In this paper, we establish a mathematical model considering the characteristics of the cooperative patrol task and the limited conditions of USVs. A hybrid partition-based patrolling scheme is proposed for a multi-USV system to visit targets with different importance levels in a maritime area. Firstly, a centralized area partition algorithm is utilized to partition the patrolling area according to the number of USVs. Secondly, a distributed path planning algorithm is applied to planning the patrolling path for each USV to visit the targets in a maritime environment to minimize the length of the patrolling path for the USV team. Finally, comparative experiments between the proposed scheme and other methods are carried out to validate the performance of the hybrid partition-based patrolling scheme. Simulation results and experimental analysis show the efficiency of the proposed hybrid partition-based patrolling scheme compared to several previous patrolling algorithms.
25 September 2023 in IEEE Journal of Oceanic Engineering
The remarkable advantages of multiunmanned surface vehicle (USV) systems in operation efficiency, accuracy, and robustness give them great application prospects in future maritime search missions. However, at this stage, reasonable mission models, efficient control methods, and effective verification have become the key problems that must be solved in the practical application of multi-USV systems in maritime search missions. Aiming at these problems, we establish a dynamic searching and updating mission model in this article. On the basis of the model, a cooperative search control method and an antidisturbance navigation control method based on the reinforcement learning method are proposed. By using the two methods, the cooperative search control and antidisturbance navigation control of a multi-USV system are combined for the first time. We design corresponding simulations and field tests to verify the proposed methods. The test results show that compared with other methods, the proposed methods give the multi-USV system certain advantages in search efficiency and accuracy.
As a new type of marine unmanned intelligent equipment, autonomous underwater vehicle (AUV) has been widely used in the field of ocean observation, maritime rescue, mine countermeasures, intelligence reconnaissance, etc. Especially in the underwater search mission, the technical advantages of AUV are particularly obvious. However, limited operational capability and sophisticated mission environments are also difficulties faced by AUV. To make better use of AUV in the search mission, we establish the DMACSS (distributed multi-AUVs collaborative search system) and propose the ACSLA (autonomous collaborative search learning algorithm) integrated into the DMACSS. Compared with the previous system, DMACSS adopts a distributed control structure to improve the system robustness and combines an information fusion mechanism and a time stamp mechanism, making each AUV in the system able to exchange and fuse information during the mission. ACSLA is an adaptive learning algorithm trained by the RL (Reinforcement learning) method with a tailored design of state information, reward function, and training framework, which can give the system optimal search path in real-time according to the environment. We test DMACSS and ACSLA in the simulation test. The test results demonstrate that the DMACSS runs stably, the search accuracy and efficiency of ACSLA outperform other search methods, thus better realizing the cooperation between AUVs, making the DMACSS find the target more accurately and faster.
Unmanned surface vehicles (USVs) as unmanned intelligent devices can replace humans to perform missions more efficiently and safely in dangerous areas. However, due to the complex navigation environment and special mission requirements, USVs face many challenges in emergency response missions for marine oil spill accidents. To solve these challenges in the emergency response mission of the ‘Sanchi’ oil tanker collision and explosion accident, we designed and deployed an USV to perform the missions of real-time scanning and water sampling in the shipwreck waters. Compared with the previous USVs, our USV owned the following characteristics: Firstly, the improved navigation control algorithms (path following and collision avoidance) can provide high navigation accuracy while ensuring navigation safety; Secondly, an improved launch and recovery system (LARS) enabled the USV to be quickly deployed and recovered in the mission area; Thirdly, a new sampling system was specially designed for the USV. Our USV completed the missions successfully, not only providing a lot of information for rescuers but also offering a scientific basis for follow-up work.
The cooperative patrol control of multiple unmanned surface vehicles (Multi-USVs) in dynamic aquatic environments presents significant challenges in global coverage efficiency and system robustness. The study proposes a cooperative patrol control algorithm for multiple unmanned surface vehicles (Multi-USVs) based on a hybrid embedded task state information model and reward reshaping techniques, addressing global coverage challenges in dynamic aquatic environments. By integrating patrol, collaboration, and obstacle information graphs, the algorithm generates kinematically feasible control actions in real time and optimizes the exploration-cooperation trade-off through a dense reward structure. Simulation results demonstrate that the algorithm achieves 99.75% coverage in a 1 km × 1 km task area, reducing completion time by 23% and 74% compared to anti-flocking and partition scanning algorithms, respectively, while maintaining collision rates between agents (CRBAA) and obstacles (CRBAO) below 0.15% and 0.5%. Compared to DDPG, SAC, and PPO frameworks, the proposed training framework (TFMUSV) achieves 28% higher rewards with 40% smaller fluctuations in later training stages. This study provides an efficient and reliable solution for autonomous monitoring and search-rescue missions in complex aquatic environments.