We present a survey on marine object detection based on deep neural network approaches, which are state-of-the-art approaches for the development of autonomous ship navigation, maritime surveillance, shipping management, and other intelligent transportation system applications in the future. The fundamental task of maritime transportation surveillance and autonomous ship navigation is to construct a reachable visual perception system that requires high efficiency and high accuracy of marine object detection. Therefore, high-performance deep learning-based algorithms and high-quality marine-related datasets need to be summarized. This survey focuses on summarizing the methods and application scenarios of maritime object detection, analyzes the characteristics of different marine-related datasets, highlights the marine detection application of the YOLO series model, and also discusses the current limitations of object detection based on deep learning and possible breakthrough directions. The large-scale, multiscenario industrialized neural network training is an indispensable link to solve the practical application of marine object detection. A widely accepted and standardized large-scale marine object verification dataset should be proposed.

The autonomous decision-making model for ship navigation requires extensive interaction and trial-and-error in real, complex environments to ensure optimal decision-making performance and efficiency across various scenarios. However, existing approaches still encounter significant challenges in addressing the temporal features of state space and tackling complex dynamic collision avoidance tasks, primarily due to factors such as environmental uncertainty, the high dimensionality of the state space, and limited decision robustness. This paper proposes an adaptive temporal decision-making model based on reinforcement learning, which utilizes Long Short-Term Memory (LSTM) networks to capture temporal features of the state space. The model integrates an enhanced Proximal Policy Optimization (PPO) algorithm for efficient policy iteration optimization. Additionally, a simulation training environment is constructed, incorporating multi-factor coupled physical properties and ship dynamics equations. The environment maps variables such as wind speed, current velocity, and wave height, along with dynamic ship parameters, while considering the International Regulations for Preventing Collisions at Sea (COLREGs) in training the autonomous navigation decision-making model. Experimental results demonstrate that, compared to other neural network-based reinforcement learning methods, the proposed model excels in environmental adaptability, collision avoidance success rate, navigation stability, and trajectory optimization. The model’s decision resilience and state-space mapping align with real-world navigation scenarios, significantly improving the autonomous decision-making capability of ships in dynamic sea conditions and providing critical support for the advancement of intelligent shipping.

Increasing vessel traffic in narrow, winding inland waterways has heightened the risk of accidents, driving the need for improved surveillance and management. This study addresses the challenge of real-time processing and synchronization of voluminous video and AIS data for effective waterway management. We developed a surveillance method utilizing smart buoys equipped with sensors and edge computing devices, enabling dynamic spatiotemporal data fusion. The integration of AIS data with advanced computer vision techniques for target detection allows for real-time traffic analysis and provides detailed navigational dynamics of vessels. The method employs an enhanced Long Short-Term Memory network for precise trajectory prediction of AIS data and a single-stage target detection model for video data analysis. Experimental results demonstrate significant improvements in ship detection accuracy and tracking precision, with an average position prediction error of approximately 1.5 m, which outperforms existing methods. Additionally, a novel regional division and a Kalman filter-based method for AIS and video data fusion were proposed, effectively resolving the issues of data sparsity and coordinate transformation robustness under complex waterway conditions. This approach substantially advances the precision and efficiency of waterway monitoring systems, providing a robust theoretical and practical framework for the intelligent supervision of inland waterways.

Under the framework of Maritime Autonomous Surface Ships (MASS), Level 2 and Level 3 autonomous vessels are expected to play a significant role in future maritime traffic. As maritime traffic density continues to increase due to economic growth, especially in critical waterways, real-time monitoring and effective traffic management are essential for ensuring navigational safety. The Remote Operations Center (ROC) plays a central role in this process, ensuring comprehensive awareness of vessels within the waterway and enabling dynamic responses. Existing ROCs primarily rely on waterway traffic service systems (such as VTS) and departments like the Maritime Safety Administration, combining various tools such as marine radar, AIS data, and shore-based surveillance cameras to provide services. However, existing monitoring technologies face several limitations, such as the latency of AIS data, which makes synchronization with real-time video data challenging, and the limitations of marine radar in obstructed or complex waterways, which affect monitoring effectiveness. Therefore, this paper proposes a real-time vessel monitoring framework based on multi-source data fusion, focusing on using navigation buoys as front-end sensing nodes equipped with cameras, AIS receivers, VHF transceivers, and edge computing devices to achieve local data fusion and analysis. By monitoring the traffic situation in real time at the buoy edge and transmitting relevant information to the ROC, this study establishes a collaborative monitoring and vessel traffic service framework between buoys and the ROC, aiming to enhance monitoring capabilities and service quality in critical waterways.