In this paper, aiming at a fleet of manned-unmanned surface vehicles (MUSVs), a novel visual servo swarming (VSS) mechanism is deliberately established by embodying visibility maintenance, swarm aggregation, collision avoidance and velocity matching. By making full use of line-of-sight ranges and angles between neighbors, a swarm of unmanned surface vehicles (USVs) with unknown inertia masses, internal dynamics and external disturbances can cooperate with a manned surface vehicle (MSV), thereby emerging flexibly collective behaviors in GPS-denied environments. To endow MUSVs with individually flexible behaviors, the VSS mechanism renders velocity matching of USVs with the MSV executing human-intelligence intention. Meanwhile, barrier Lyapunov functions are employed to reliably maintain visibility and avoid collisions among individuals, simultaneously. Distributed neural approximators using reduced-dimension inputs are devised to estimate unknown dynamics of USVs, while residual uncertainties are thoroughly suppressed by robust adaptations, thereby contributing to a model-free VSS (MVSS) scheme. Eventually, together with projection-based adaptive laws, the MVSS-based controller ensures uniform boundedness of estimation parameters and asymptotic convergence of regulation errors. Simulation results demonstrate remarkable efficacy in terms of collective performance.

In this paper, in the presence of completely unknown dynamics and unmeasurable velocities, a dynamics-level finite-time fuzzy monocular visual servo (DFFMVS) scheme is created for regulating an unmanned surface vehicle (USV) to the desired pose. Main contributions are as follows: 1) With the aid of homography decomposition, a novel homography-based visual servo (HBVS) structure for a USV with both kinematics and dynamics is firstly established such that complex unknowns including unmeasurable poses and velocities, image depth, system dynamics, and time-varying inertia are sufficiently encapsulated; 2) Using finite-time observer technique, finite-time velocity observer (FVO) based visual-servo error dynamics are elaboratively formulated, and thereby facilitating backstepping synthesis; 3) By virtue of the FVO, the adaptive fuzzy dynamics approximator (AFDA) together with adaptive residual feedback is deployed to compensate complex unknowns, and thereby contributing to accurate regulation of pose errors; and 4) A completely model-free monocular visual servo approach only using a camera is eventually invented. Simulation studies on a benchmark prototype USV demonstrate that the proposed DFFMVS scheme has remarkable performance with significant superiority in both visual servo and unknowns observation.

In this paper, a two-level visual servo strategy is elaborately devised for an unmanned surface vehicle (USV) equipped with a pan-tilt camera, so as to exactly track the desired trajectory around a visual target without direct position measurements. In the lower level, a barrier function-based adaptive pseudo-inverse (BFAP) controller is specially designed for the camera to keep the target in sight. Together with a finite-time position observer (FPO) and a finite-time extended state observer (FESO), a model-free finite-time trajectory-tracking control (MFTC) scheme is naturally synthesized for the USV on the higher level. Prominent advantages are presented as follows: 1) The BFAP controller can not only circumvent the singularity issue in a simpler manner, but also solve the field-of-view problem thoroughly in spite of unknown image depth; 2) The FPO provides a new vision-based method to locate the USV by rapidly calibrating a constant extrinsic parameter of the camera online, achieving higher positioning accuracy; and 3) The MFTC scheme allows all model information of the USV to be unknown, which is more favorable to practical implementations. Stability analyses are strictly made by the Lyapunov theory, and simulation studies conducted on the prototype CyberShip II comprehensively demonstrate remarkable performance of the proposed BFAP controller and MFTC scheme.

In this paper, suffering from unmatched visual-servo uncertainties and unknown dynamics/disturbances, an extreme learning-based monocular visual servo (ELMVS) scheme is developed for maneuvering an unmanned surface vessel (USV) to reach the desired pose. By virtue of the backstepping philosophy, complex visual-servo unknowns are elaborately encapsulated into lumped nonlinearities which are further accurately accommodated by devising a single-hidden layer feedforward network (SLFN) based adaptive compensating identifier (SACI). Within the SACI architecture, hidden nodes are completely model-free and are randomly generated without tedious learning, and thereby dramatically expediting fast-dynamics identification. Moreover, by exploiting approximation residuals, direct hyperbolic-tangent links between input and output layers are deployed to enhance identification accuracy. Eventually, the Lyapunov synthesis guarantees that the proposed ELMVS scheme can asymptotically render visual-servo errors arbitrarily small while target features can be kept within the field of view. Remarkable performance and superiority is finally demonstrated on a prototype USV.

The steering dynamics of marine robots play a pivotal role in achieving precise maneuvering. However, complex and unpredictable ocean disturbances pose challenges for rapid online learning of accurate dynamics. This paper presents the Online Memory Koopman Learning (OM-Koop) framework, a hybrid model that combines physical priors, online data and stability preserving mechanisms to solve the nonlinear dynamic capture challenge and dynamically adapt to the marine environment. Firstly, we construct the Koopman operator-based uncertainty model online using the state error of the steering model and sliding window methods. The model can effectively capture the nonlinear features that are not represented in the predefined steering model. To ensure stability, the eigenvalues of the Koopman operator are constrained during online learning, guaranteeing Lyapunov stability. Secondly, in order to improve the efficiency of online learning, the Long Short-Term Memory (LSTM) neural network is involved in the construction process of the Koopman operator, which enhances the model’s memory capability. Finally, through field experiments using Autonomous Surface Vehicles (ASVs) and Autonomous Underwater Vehicles (AUVs) in field environments, comparative analyses with other learning strategies show that OM-Koop has excellent adaptability and robustness while guaranteeing Lyapunov stability. Note to Practitioners—The motivation of this article is that the steering dynamic behavior of marine robots is highly affected by unpredictable ocean environments, which poses great challenges in achieving precise manipulation in practical applications. In this paper, we propose the OM-Koop framework to address these challenges by integrating physical priors, online data learning, and stability preserving mechanisms. Theoretical analyses show that the proposed framework ensures Lyapunov stability while dynamically adapting to nonlinear disturbing forces imposed by the environment. Field experiments on AUV and ASV validate the robustness and adaptability of the framework, demonstrating its potential for practical deployment in dynamic marine environments. And the proposed framework has prospects for practical applications in other robotic systems.