With the development of learning-based autonomous underwater exploration method of robotic fish, how to improve data quality and sampling efficiency, so as to achieve better control performance, becomes a challenging research subject. To address this issue, in this article, a feasible real-world deep reinforcement learning framework for autonomous underwater exploration of robotic fish is proposed, which ably avoids model discrepancy generated in virtual training. The designed framework consists of three phases: 1) teaching initialization; 2) regular update of reinforcement learning; and 3) phased consolidation training. Especially, reasonable teaching initialization improves the data sampling efficiency and stabilizes the real-world early training. The consolidation training ensures the reproducibility of good controllers by interim imitation learning in the middle training phase. Extensive underwater experiments on a novel self-developed biomimetic robotic shark show that the proposed real-world learning method significantly improves the safety and efficiency of autonomous exploration based on local sensor information, providing a promising solution for exploring in uncharted wild waters.

In this article, we propose a tightly coupled visual-inertial-acoustic sensor fusion method to improve the autonomous localization accuracy of underwater vehicles. To address the performance degradation encountered by existing visual or visual-inertial simultaneous localization and mapping systems when applied in underwater environments, we integrate the Doppler velocity log (DVL), an acoustic velocity sensor, to provide additional motion information. To fully leverage the complementary characteristics among visual, inertial, and acoustic sensors, we perform multimodal information fusion in both frontend tracking and backend mapping processes. Specifically, in the frontend tracking process, we first predict the vehicle’s pose using the angular velocity measurements from the gyroscope and linear velocity measurements from the DVL. Thereafter, measurements performed by the three sensors between adjacent camera frames are utilized to construct visual reprojection error, inertial error, and DVL displacement error, which are jointly minimized to obtain a more accurate pose estimation at the current frame. In the backend mapping process, we utilize gyroscope and DVL measurements to construct relative pose change residuals between keyframes, which are minimized together with visual and inertial residuals to further refine the poses of the keyframes within the local map. Experimental results on both simulated and real-world underwater datasets demonstrate that the proposed fusion method improves the localization accuracy by more than 30% compared to the current state-of-the-art ORB-SLAM3 stereo-inertial method, validating the potential of the proposed method in practical underwater applications.

This article proposes a tightly coupled visual-acoustic sensor fusion method for self-localization of a biomimetic robotic shark. To address the decreased localization accuracy of visual-based simultaneous localization and mapping systems employed on a robotic fish in underwater environments, we integrate velocity measurements from the acoustic sensor Doppler velocity log (DVL) into a visual odometry. To fully exploit the local position change information contained in velocity measurements, DVL measurements are fused in two stages of visual tracking. Specifically, we first employ the velocity measurements to improve the initial camera pose estimation during visual tracking, aiming to provide a better initial value for subsequent pose optimization. Thereafter, these velocity measurements are directly employed to constrain the camera position change between two adjacent frames by constructing a DVL residual term, which is optimized jointly with the visual residual to obtain a more accurate camera pose. Extensive experiments are conducted on both self-collected simulated datasets and real-world underwater datasets. Experimental results demonstrate that the proposed visual-acoustic fusion method can effectively improve the localization accuracy for the robotic shark by more than 50% compared to a pure visual system, providing valuable guidance for improving the autonomous localization capability of underwater biomimetic robots.

This paper proposes a novel tightly coupled monocular-inertial-pressure sensor fusion method for underwater localization of a biomimetic robotic manta. Based on the ORB-SLAM3 monocular visual-inertial odometry model, the depth measurement from a pressure sensor is incorporated and a novel approach is provided to associate low-frequency pressure measurements with high-frequency frames by utilizing only the relative depth between two adjacent keyframes as measurements. To address the challenge of scale estimation uncertainty in monocular odometry systems, a two-step monocular initialization strategy is proposed, involving an initial estimate based on visual-pressure measurements and a subsequent tightly coupled-inertial pressure depth residual construction, which results in a significantly improved scale estimate compared to conventional monocular inertial odometry systems. After successful initialization of the monocular system, a novel visual-inertial-pressure joint optimization method is proposed to enhance the localization and attitude estimation accuracy. Extensive experiments are carried out on both open-source datasets and real-world underwater datasets collected by a biomimetic robotic manta. The experimental results demonstrate the effectiveness of the proposed method in significantly improving both the position and attitude estimation of the biomimetic robotic manta. This research provides valuable insights for enhancing the underwater localization capability of biomimetic robotic systems.

Due to the sparse underwater textures, pure visual simultaneous localization and mapping (SLAM) is limited, restricting the localization ability of underwater robots. To improve the robustness and accuracy of visual SLAM underwater, a multifeature and multimodal information fusion framework for underwater robot self-localization, named PL-VAP, is proposed. First, by integrating line features with point features, a point-line fusion residual evaluation algorithm is introduced to address the issue of limited underwater features. In addition, PL-VAP integrates the Doppler velocity log (DVL) and pressure sensors to address the challenges of visual information in underwater environments. Specifically, in the frontend, the robot’s pose is predicted using DVL velocity data, followed by point-line-acoustic-pressure joint factor graph optimization, where multimodal data—including point and line features, acoustic velocity, and pressure sensor depth—are jointly optimized to improve pose estimation. In the backend, a visual-acoustic-pressure bundle adjustment refines relative poses between keyframes by fusing depth observations from the pressure sensor, low-frequency DVL pose estimates, and point-line features for global optimization. To validate PL-VAP, an underwater multisensor robotic system was developed. Experimental results from pool and 16-m-deep sinkhole scenarios demonstrate that PL-VAP significantly enhances the positioning accuracy and robustness of underwater robots through multifeature and multimodal information fusion.

This paper aims to present a novel method for underwater three-dimensional (3D) point cloud reconstruction by estimating the pseudo front view from a single sonar image. Since the pseudo front view contains elevation, azimuth, and depth information, it can be easily transformed into a point cloud using formulas that relate polar and Cartesian coordinate systems. To achieve this, we utilize the U-Net++ network as the baseline to generate the pseudo front view. To address the challenge of scale compression caused by using inverse depth as the ground truth during training process, we employ the scale-invariant logarithmic loss (SILog) as one of the loss functions. This approach helps generate better training outcomes by balancing the extreme values in the depth ground truth. Additionally, we introduce a novel sonar-specific loss function, the sonar weight loss (SWL), designed to capture the unique characteristics of sonar image generation and to differentiate the training of sonar pseudo front view depth images from general depth image training. Finally, we conduct extensive experiments using both open-source datasets (i.e., water tank and floating object) and real-world scenarios to comprehensively evaluate the effectiveness of the proposed method. The experimental results demonstrate its robustness and accuracy in 3D reconstruction tasks. On the Water Tank dataset, the proposed method achieves an 87.24% reduction in 3D reconstruction error compared to A2FNET. On the Floating Object dataset, it further outperforms ElevateNet (a more advanced baseline than A2FNET), reducing the 3D reconstruction error by 74.13%. These results highlight the effectiveness of the proposed method and its practical potential for underwater 3D reconstruction using imaging sonars.