Maritime Autonomous Surface Ships (MASS) are becoming of interest to the maritime sector and are also on the agenda of the International Maritime Organization (IMO). With the boom in global maritime traffic, the number of ships is increasing rapidly. The use of intelligent technology to achieve autonomous collision avoidance is a hot issue widely discussed in the industry. In the endeavor to solve this problem, multi-ship coordinated collision avoidance has become a crucial challenge. This paper proposes a multi-ship autonomous collision avoidance decision-making algorithm by a data-driven method and adopts the Multi-agent Deep Reinforcement Learning (MADRL) framework for its design. Firstly, the overall framework of this paper and its components follow the principle of “reality as primary and simulation as supplementary”, so a real data-driven AIS (Automatic Identification System) dominates the model construction. Secondly, the agent’s observation state is determined by quantifying the hazardous area. Then, based on a full understanding of the International Regulations for Preventing Collisions at Sea (COLREGs) and the preliminary data collection, this paper combines the statistical results of the real water traffic data to guide and design the algorithm framework and selects the representative influencing factors to be designed in the collision avoidance decision-making algorithm’s reward function. Next, we train the algorithmic model using both real data and simulation data. Meanwhile, Prioritized Experience Replay (PER) is adopted to accelerate the model’s learning efficiency. Finally, 40 encounter scenarios are designed and extended to verify the algorithm performance based on the idea of the Imazu problem. The experimental results show that this algorithm can efficiently make a ship collision avoidance decision in compliance with COLREGs. Multi-agent learning through shared network policies can ensure that the agents pass beyond the safe distance in unknown environments. We can apply the trained model to the system with different numbers of agents to provide a reference for the research of autonomous collision avoidance in ships.

In order to reduce the influence of human factors on ship collision avoidance, this paper proposes an autonomous collision avoidance decision-making algorithm based on Double Deep Q Network (DDQN) with prioritized experience replay (PER). Based on a full understanding of the International Regulations for Preventing Collision at Sea (COLREGs), and the reward function of the autonomous collision avoidance decision-making algorithm is guided and designed in terms of the number of rudder operations, the amount of the cumulative steering angle during the avoidance process, the DCPA when clear of the other ship, the deviation distance at the end of the avoidance, the total magnitude of ship heading changes during the avoidance process and rule compliance by combining with the statistical results of real water traffic data. Ship encounter scenarios are constructed based on COLREGs, which are arranged and combined the sample training and testing sets for the agent; We compare the algorithm proposed in this paper with DQN, DQN with prioritized experience replay and DDQN algorithms by simulating the same encounter scenario, using this as the training set for the model; Finally, multi-ship encounter scenario simulations are set up as the testing set for the model to verify the performance of the algorithm. The results show that compared with three other reinforcement learning algorithms, DDQN with prioritized experience replay algorithm can make ship collision avoidance decision-making more efficient and improve the safety of ship autonomous navigation under COLREGs; and the trained model can be replicated and applied to a system with any number of agents to provide a reference for ship autonomous collision avoidance research.