Infrared (IR) cameras have been important surveillance sensors for autonomous surface vessels; however, their detection ranges are limited by low resolution. In this study, we collect maritime IR images, analyze the characteristics of those images, and develop datasets for training and testing. Then, a new maritime IR image super-resolution network, maritime infrared super-resolution using cascaded residual network, is developed to reconstruct IR images using a scale of 4. Moreover, different loss functions have different effects on output images; a loss function is set to be a combination of three loss functions, including mean absolute error, mean squared error, and perceptual loss. Peak signal-to-noise ratio and structural similarity index measure cannot effectively describe super-resolution performance. As the novel evaluation metric, Canny edge detection method is used because edges are important for human and target detection algorithms. Finally, experiments are conducted and the results demonstrate that the developed residual network can achieve high-quality reconstructed maritime IR images.

Merchant ships sometimes fail to detect small ships at night and in poor visibility, leading to urgent situations and even collisions. Infrared (IR) cameras have inherent advantages in small target detection and become essential environmental awareness equipment on unmanned ships. The existing target detection models are complex and difficult to deploy on small devices. Lightweight detection algorithms are needed with the increase in the number of shipborne cameras. Therefore, herein, a lightweight model for small IR ship detection was selected as the research object. IR videos were collected in the Bohai Strait, the image sampling interval was calculated, and an IR dataset of small ships was constructed. Based on the analysis of the characteristics of the IR ship images, gamma transform was used to preprocess the images, which increased the gray difference between the target and background. The backbone of YOLOv5 was replaced with that of Mobilev3 to improve the computing efficiency. Finally, the results showed that the parameters of the proposed model were reduced by 83% compared with those of the YOLOv5m model, while the detection performance was almost the same.

In unmanned surface vehicle (USV) platform based marine radar target detection, as a result of low radar installation height, the presence of target occlusion by islands, nearby vessels as well as near-shore infrastructure is very common. This makes it difficult for a ship target to be consistently detected and tracked, leading to potential track loss and collision risks. However, efficient solutions for identifying occlusion and maintaining sound tracking performance in a detection occluded maritime environment are not well explored in the literature. In this paper, an event-driven occupancy grid assisted anti-occlusion approach is proposed. When the event of target missed detection occurs, the proposed algorithm utilizes available information from electronic navigation chart (ENC), automatic identification system (AIS) as well as marine radar to build an occupancy grid map for reasoning whether the radar signal is blocked by any existing objects lying in between of the target and the USV. An occlusion adaptive target tracking framework is thus constructed to improve the stability of the target tracking. Both the simulation and real-data validation results show that the proposed algorithm is able to improve the target tracking performance in a detection occlusion environment.

In this paper, we release a new large-scale dataset containing multiple categories of ships and floating objects at sea, which we call MASS-LSVD. It is used to train and validate target detection algorithms and future large models for ship autopiloting. The dataset was captured by a visible light camera installed aboard the world’s first intelligent research, teaching, and training ship, “Xinhongzhuan”. This MASS (maritime autonomous surface ship) was operated by Dalian Maritime University, China. We have collected more than 4000 h of video of the “Xinhongzhuan” vessel’s voyage in the Bohai Sea and other areas, which are carefully classified and filtered to cover as much as possible the various types of sample data in the marine environment, such as light intensity, weather, hull shading, data from ocean-going voyages, entering and exiting ports, etc. The dataset contains 64,263 1K-resolution images captured from video footage, covering four main ship types: Fishing Boat, Bulk Carrier, Cruise Ship, Container ship, and an ‘Other Ships’ class, for vessels that cannot be specifically classified. The dataset currently contains 64,263 pairs of 1K-resolution images covering four common ship types (fishing boat, bulk carrier, cruise ship, container, and other ship, where no specific ship type can be determined). All the images have been labeled with high-precision manual bounding boxes. In this paper, the MASS-LSVD dataset is used as the basis for training various target detection algorithms and comparing them with other datasets, which compensates for the lack of first-view images in the vessel target detection dataset, and MASS-LSVD is expected to be used to facilitate the research and application of autonomous ship navigation models in the framework of computer vision.

Infrared (IR) and visible image fusion has become an important technology for maritime environment awareness. In this study, we employed refined features fusion and Sobel loss for maritime IR and visible image fusion. First, we used the encoder in Densefuse net to transform the image to be fused from the pixel domain to the feature domain. Then, the features’ information was accurately measured, and the refined features were fused and sent to the decoder to reconstruct the fused images. Mixed loss, commonly employed in image fusion, eliminates the effect of different losses but also introduces hyperparameters. We innovatively used Sobel loss to measure the input and fused images to reduce the hyperparameters in the loss function. Finally, experiments were performed, and our algorithm achieved good fusion results in the qualitative and quantitative evaluations.