Traditional visual simultaneous localization and mapping (SLAM) methods primarily rely on passive vision, such as monocular cameras, which often exhibit reduced accuracy in localization and struggle to create dense maps in low-light underwater conditions. In this article, a tightly coupled hybrid visual inertial navigation system (VINS), named Hybrid-VINS, is proposed by fuzing active and passive vision, which enables robust localization and dense mapping underwater. Specifically, a self-designed active vision device called underwater binocular structured light (UBSL) is integrated to provide more accurate depth estimation of the scene, enhancing the initialization and visual feature tracking process of monocular VINS. As well as it addresses the deficiency of VINS in dense mapping. In addition, a more robust hybrid vision-aided loop closure detection algorithm is proposed to mitigate issues stemming from pure passive vision misjudgments and mismatches. Furthermore, an underwater autonomous hybrid vision system is developed in both simulated and real-world underwater environments to collect various datasets for verifying the performance of Hybrid-VINS. Experimental results demonstrate that, compared with passive VINS, Hybrid-VINS holds greater promise for underwater high-precision localization and dense mapping.
7 November 2023 in IEEE/ASME Transactions on Mechatronics
Due to the advantages of high measurement accuracy and good robustness, structured light vision (SLV) methods have been extensively applied for 3-D reconstruction in air and water scene. However, most current SLV systems cannot adapt to different media without recalibration, and have the disadvantages of complicated calibration process including SLV model parameter calibration in air and water. In order to overcome these problems, a SLV 3-D reconstruction system for different media considering refraction is proposed. First, an accurate SLV measurement model is established for 3-D reconstruction in different media, which considers structured light plane refraction model and camera refraction model simultaneously. Specifically, the relationship of the SLV model parameters in air and water are proposed. Second, a simple and efficient calibration method of SLV model parameters is presented. After SLV model calibration in air and refraction interface parameters calibration, the SLV model parameters in different media can be obtained accurately without recalibration. Finally, a compact SLV system is developed, and calibration experiments and 3-D reconstruction experiments in air and different turbid water are conducted. Experiment results show that the proposed system could obtain satisfactory 3-D reconstruction performance in air and different turbidity water.
29 October 2024 in IEEE Transactions on Industrial Electronics
At present, underwater robot self-localization methods mainly rely on inertial measurement unit (IMU) and Doppler velocity log (DVL) based dead reckoning (DR) and standard camera-based vision localization methods. However, the DR method has the problem of error accumulation with the increase of distance and the standard camera-based vision method is difficult to achieve accurate localization due to little information and sparse features of standard images in low-light underwater environment. To address these problems, a tightly coupled event, standard image, inertia, and acoustic fusion method is proposed to realize robust and accurate self-localization of underwater robots. First, the high-frequency IMU data is fused to generate motion compensation event frames, and the feature points of event frames and standard frames are extracted and tracked respectively. Then, a joint optimization problem of cost function is established by using the reprojection error term of the event frame and standard frame, the IMU inertia error term and the DVL velocity error term, avoiding the explicit switching between event frame and standard frame. Finally, a multisensor-based self-localization system for underwater robots is developed, and a series of event motion compensation and self-localization experiments in real underwater environment are carried out. Experimental results show that the proposed method has better localization accuracy than standard camera, IMU, and DVL fusion methods.
16 April 2024 in IEEE Robotics and Automation Letters
Planar structures, prevalent in man-made environments, can be observed by a camera for significant periods of time due to their large spatial presence. These structures provide strong planar regularities for Simultaneous Localization and Mapping (SLAM) systems, facilitating long-term navigation. Therefore, we propose a novel point-plane-based stereo SLAM system, fully regularized by plane features within a unified non-linear optimization framework. The core of our method is an accurate and efficient stereo plane extraction algorithm with strict 2D and 3D outlier rejection mechanisms, effectively extracting main planes from robust stereo correspondences and enabling real-time point-plane association. Furthermore, we introduce a novel optimization formulation, incorporating geometric feature (point and plane) and across-feature (point-on-plane) constraints that promote each other through the mutual constraints between associated point and plane features, which fully exploits plane constraints to improve the performance of SLAM system. The proposed plane extraction algorithm is evaluated on the EuRoC MAV dataset, achieving significant improvements in number, accuracy, reliability, and efficiency over the state-of-the-art (SOTA) stereo point-plane-based system (Zhang et al., 2021). The results of an ablation study on two public datasets show that the proposed SLAM system outperforms (Zhang et al., 2021) in both accuracy and robustness, and further demonstrate the mutual enhancement between the two types of constraints.
15 November 2023 in IEEE Transactions on Multimedia
Local feature extraction consists of keypoint detection and local descriptor extraction. Firstly, in keypoint detector learning, existing covariance constraint loss functions cannot constrain the probability distribution shapes in local probability maps that surround keypoints. And existing auxiliary peak loss functions, which are used to alleviate the problem, impair the performance of local feature methods. To solve this problem, we propose a novel Covariant Peak constraint Loss (CP Loss) which is defined as the expectations of local probability maps' position errors. Minimizing our CP Loss can make local probability maps accurately peak at reliable keypoints. Secondly, in descriptor learning, the Neural Reprojection Error (NRE) aims at constraining dense descriptor maps of images. But we argue that only those descriptors of keypoints need to be constrained. Thus, we propose a novel Conditional Neural Reprojection Error (CNRE) that is only conditioned on keypoints. Compared with the NRE, our CNRE can achieve much higher efficiency and produce more keypoint-specific descriptors with better matching performance. We use our CP Loss and CNRE to train a local feature network named as CPCN-Feat. Experimental results show that our CPCN-Feat achieves state-of-the-art performance on four challenging benchmarks.
16 March 2026 in IEEE Transactions on Instrumentation and Measurement
Underwater pipelines play an important role in offshore oil and gas transmission, and underwater cables are vital for global interconnection communication. However, due to the harsh underwater environment, underwater pipelines and cables are easily damaged. In order to complete the inspection of underwater pipelines and cables, researchers have proposed many inspection platforms, detection and tracking methods to improve inspection efficiency and automation level. In this paper, the state-of-the-art techniques for underwater pipeline and cable external inspection are comprehensively reviewed. Firstly, underwater pipeline and cable inspection platforms are classified and introduced, and their advantages and disadvantages are discussed. Then, underwater pipeline and cable detection methods including acoustic method, passive optical method, active optical method and electromagnetic method are introduced, and their performance comparison are provided. Besides, underwater pipeline and cable tracking methods are analyzed. Finally, we discuss the key issues and future directions of underwater pipeline and cable inspection. This review will help researchers better understand the current research progress in underwater pipeline and cable inspection platforms, detection and tracking methods, which provides guidance for further research and deployment.