Forward-looking sonar (FLS) image segmentation plays a significant role in ocean engineering. However, the existing image segmentation algorithms present difficulties in extracting features from FLS images with weak semantic information, complex backgrounds and strong environmental noises. Convolutional neural networks (CNNs) have demonstrated remarkable capabilities in semantic segmentation tasks, but the locality of convolution limits the ability to extract global context and long-range semantic information. The effective extraction of global contextual information is indispensable for achieving accurate segmentation results in sonar image processing. In this paper, we propose a novel semantic segmentation architecture for forward-looking sonar images called SonarNet. SonarNet is based on a hybrid CNN-Transformer-HOG framework and comprises four modules. 1) The Global-Local Encoder can extract both global and detailed feature information of the underwater target; 2) the Network Decoder converts the high-semantic feature map into a pixel-level classification; 3) as a bridge between dual encoders, the Global-Local Fusion Module ensures semantic consistency between different encoders; 4) the HOG Feature Encoder and Fusion can extract traditional manual features and perform feature alignment. We conducted comprehensive ablation experiments to validate the efficacy of the designed modules. Finally, experimentation revealed that SonarNet significantly outperforms other CNN-based and CNN-Transformer FLS image segmentation methods.

Target detection in side-scan sonar images plays a significant role in ocean engineering. However, the target images are usually severely interfered by the complex background and strong environmental noise, which makes it difficult to extract robust features from small targets and makes the target detection task quite challenging. In this paper, a novel small target detection method in sonar images is proposed based on the low-rank sparse matrix factorization. Initially, the side-scan sonar images are preprocessed so as to highlight the individual differences of the target. Then, the problems of target feature extraction and noise removal are characterized as the problem of matrix decomposition. An improved Robust Principal Component Analysis algorithm is used to extract target information, and the fast proximal gradient method is used to optimize the solution. The original sonar image is reconstructed into the low-rank background matrix, the sparse target matrix, and the noise matrix. Eventually, a morphological operation is used to filter out the noise and refine the target edges in the target matrix for improving the accuracy of target detection. Experimental results show that the proposed method not only achieves better detection performance in comparison to the conventional baseline algorithms but also performs robustly in various signal-to-clutter ratio conditions.

In recent years, unmanned surface vehicles (USVs) have played an increasingly important role in various applications. Due to the expansion of USV application scenes from common marine areas to inland waters with complex environments, environmental perception has become an essential requirement for autonomous navigation systems of USVs. Traditional perception methods utilize either LiDAR or radar to construct volumetric maps for environmental perception. To improve the accuracy of perception systems and reduce deployment costs, this paper proposes a novel radar and camera fusion volumetric map network named FVMNet for real-time volumetric perception. FVMNet is based on a novel radar and image fusion architecture and comprises four modules. 1) The radar and image encoders can extract different features; 2) Only using in training stage without extra valid time costs, auxiliary segmentation head advances the image encoder; 3) To eliminate the representation difference between image features and radar features, the BEV spatial transformer module transfers image feature representations from the perspective view to BEV space; 4) The fusion segmentation head predicts the volumetric perception results. Compared to other baseline methods that use a single modality, FVMNet achieves state-of-the-art accuracy in public USVInland dataset and our collected wharf dataset. We conducted comprehensive ablation experiments to validate the efficacy of the designed modules. Moreover, the proposed method demonstrates generalization in zero-shot real-world scenarios and robustness under extreme weather conditions.

While the intricate underwater environment leads to blurry and faint features of sonar targets, SonarNet has depicted great success due to its high model capabilities and multi-feature fusion mechanism. However, their remarkable performance is accompanied by heavier backbones and larger model sizes to achieve benefits at the cost of increased complexity. The research on fast segmenters deployed on edge devices is urgently inquired. In this paper, we analyze the current best-performing sonar image segmentation network named SonarNet. Based on the analysis, we propose a lightweight local feature grafted vision transformer embedding network for forward-looking sonar images called EsonarNet, which promotes a priority balance between efficiency and accuracy. EsonarNet is based on a hybrid local-global feature grafting architecture and comprises four modules. (1) Expand from the traditional convolutional neural network (CNN) and histogram of oriented gradients (HOG), a lightweight sonar semantic segmentation model based on hybrid CNN-Transformer-HOG fusion encoding and decoding, while preserving high efficiency applied to hardware resources. (2) Lightweight encoder units are employed in our EsonarNet, including a designed spatial mobile convolution (SMBConv) and efficient vision transformer module. (3) Serving as a transitional liaison between the CNN encoder and transformer encoder, the LGFI module focuses on dispersing local semantic information to facilitate long-distance computations, and the GLAU module computes correlations through dot products to restore inductive bias. (4) The HOG features are introduced into EsonarNet through the LHDGM module to ensure the coherence and compatibility of the acquired traditional and abstract information with different semantics. Ultimately, experimental results demonstrate that EsonarNet outperforms other methods for FLS image segmentation in efficiency.

In recent years, the significance of unmanned surface vehicles (USVs) has grown substantially across a wide range of applications. Monocular cameras, as the most common perception sensors deployed in USVs, offer the inherent advantages of rich semantic information and low-cost deployment. However, visual perception methods for USVs face challenges from water surface application and lose efficiency in harsh weather cases. To achieve robust perception capabilities through monocular cameras for USVs, we propose a panoptic water surface visual perception framework that can accomplish various perception tasks, including drivable area segmentation, object detection and raindrop segmentation in water surface scenes. In addition, our framework provides a SAM-driven training method to improve the robustness of the proposed model through low-cost model iteration. Moreover, the proposed perception method demonstrates excellent accuracy, robustness and high inference speed compared to other light-weight baseline methods and can be deployed to low-power embedded platforms in USVs.