Bionic robots possess inherent advantages for underwater operations, and research on motion control and intelligent decision making has expanded their application scope. In recent years, the application of reinforcement learning algorithms in the field of bionic underwater robots has gained considerable attention, and continues to grow. In this paper, we present a comprehensive survey of the accomplishments of reinforcement learning algorithms in the field of bionic underwater robots. Firstly, we classify existing reinforcement learning methods and introduce control tasks and decision making tasks based on the composition of bionic underwater robots. We further discuss the advantages and challenges of reinforcement learning for bionic robots in underwater environments. Secondly, we review the establishment of existing reinforcement learning algorithms for bionic underwater robots from different task perspectives. Thirdly, we explore the existing training and deployment solutions of reinforcement learning algorithms for bionic underwater robots, focusing on the challenges posed by complex underwater environments and underactuated bionic robots. Finally, the limitations and future development directions of reinforcement learning in the field of bionic underwater robots are discussed. This survey provides a foundation for exploring reinforcement learning control and decision making methods for bionic underwater robots, and provides insights for future research.

Adaptive and efficient cooperative control is a crucial capability for multirobotic fish systems, as it can substantially enhance their performance in complex underwater tasks. The pursuit and evasion dynamics in such topics have gained significant attention from the scientific community. In this article, we present a novel adaptive algorithm tailored specifically for cooperative pursuit among multirobotic fish systems. Benefiting from the integration of attraction mechanisms and reinforcement learning techniques, the proposed method empowers the robotic fish to make adaptive decisions based on local observations and environmental cues. Meanwhile, a state transition environment has been customized to the unique dynamics of robotic fish, equipping the cooperative pursuit strategy to fulfill practical application requirements and facilitate adaptation across diverse platforms. Besides, based on the curriculum learning approach, a decentralized pursuit policy is also formulated and implemented within the developed robotic fish system. Simulations and real-world experiments have validated the efficiency and adaptability of this cooperative pursuit strategy. This research offers valuable insights and contributions to the exploration of cooperative control in multirobotic fish systems, addressing the critical challenge of achieving adaptive and efficient coordination in complex underwater environments.

The control of multi-robot systems, particularly in the pursuit-evasion (PE) with multiple robots, has gained significant attention in both academic and non-academic settings. However, the collaborative operation of multi-robotic fish systems encounters substantial challenges due to the complex underwater environment and unique movement mode. In this paper, we propose a multi-agent reinforcement learning (MARL) approach to develop a viable strategy for underwater cooperative pursuit. Initially, considering the hydrodynamic model and motion characteristics of robotic fish, we construct a specific simulation environment with multiple fish-like agents, which provides a highly realistic state transition model. Next, we develop a MARL-based strategy learning framework that incorporates appropriate reward functions and agent actions for policy learning. Finally, a series of comprehensive simulations and practical experiments are conducted to validate the effectiveness of the proposed method and confirm its successful application in underwater pursuit scenarios. These findings offer valuable insights for further research in underwater multiple robot systems.

Directly learning the swimming behaviors of real fish can significantly enhance the swimming performance of bionic robotic fish. This paper presents a novel transfer learning method based on a dynamic trajectory control approach for the robotic fish to learn swimming skills from real fish. First, we develop a fish motion capture system and a crucial motion extraction approach to realize precise decomposition of fish motions and collect abundant meaningful features from a snakehead fish as pre-training data. Next, we construct a two-stage transfer learning method based on Deep Deterministic Policy Gradient (DDPG), including an offline and an online stage. Specifically, in the offline stage, the obtained pre-training data is processed for experience learning within a DDPG-based network, whereas in the online stage, a dynamic trajectory tracking method is utilized to refine the robotic fish’s motions in real time based on the learned strategies. Experimental results on a self-developed four-joint robotic fish show that the proposed method effectively extracts and transfers biological motion features into the motion control of the robotic fish. Compared to the conventional CPG method, the proposed approach exhibits stronger acceleration capabilities and more efficient swimming, resulting in enhanced maneuverability of the robotic fish. Overall, this approach provides a technical foundation for bionic robotics to learn from nature.

In ocean engineering, path following serves as a fundamental capability for autonomous underwater vehicles (AUVs), enabling essential operations such as environmental exploration and inspection. However, for robotic dolphins employing dorsoventral undulatory propulsion, the periodic pitching induces strong coupling between propulsion and attitude, posing significant challenges for precise path following in disturbed environments. In this paper, a real-time robust path-following control framework is proposed for robotic dolphins to address these challenges. First, a novel robotic dolphin platform is presented by integrating a dorsoventral propulsion mechanism with a passive peduncle joint, followed by the systematic formulation of a full-state dynamic model. Then, a minimum-snap-based path optimizer is constructed to generate smooth and dynamically feasible trajectories, improving path quality and motion safety. Subsequently, a robust model predictive controller is developed, which incorporates control surface dynamics, a nonlinear disturbance observer, and a Sigmoid-based disturbance-grading mechanism to ensure fast attitude response and precise tracking performance. Finally, extensive simulations under various environmental disturbances validate the effectiveness of the proposed approach in both trajectory optimization and robust path following. The proposed framework not only demonstrates strong robustness in path following and disturbance rejection, but also provides practical guidance for future underwater missions such as long-term environmental monitoring, inspection, and rescue.

Accurate extrinsic calibration is essential for integrating optical and acoustic imaging sensors in underwater robotic systems, enabling precise multimodal perception in challenging aquatic environments. However, existing calibration methods typically depend on artificial targets or manual cross-modal feature matching, which are often impractical or unreliable in real-world underwater scenarios. To address these limitations, this article proposes a targetless extrinsic calibration method for optical-acoustic stereo imaging systems, which automatically estimates and refines extrinsic parameters without the need for predefined targets. The proposed method comprises a three-stage pipeline. First, the camera motion is estimated via visual structure-from-motion (SfM), while the sonar motion is recovered using the proposed acoustic bundle adjustment algorithm and the symmetric dual-pose ambiguity elimination strategy, both specifically designed to address the unique nonlinear projection model of imaging sonar. Finally, to further improve accuracy, 3D acoustic landmarks are utilized to formulate 3D-2D reprojection constraints, and a nonlinear optimization minimizes the geometric error to refine the extrinsic parameters. The proposed method is extensively validated in both simulated and real-world underwater environments. Experimental results demonstrate accurate and robust performance under varying noise levels and system configurations, with the final calibration results accurately satisfying the epipolar constraints of optical-acoustic stereo imaging systems.