In this article, we present an adaptive reinforcement learning optimal tracking control (RLOTC) algorithm for an underactuated surface vessel subject to modeling uncertainties and time-varying external disturbances. By integrating backstepping technique with the optimized control design, we show that the desired optimal tracking performance of vessel control is guaranteed due to the fact that the virtual and actual control inputs are designed as optimized solutions of every subsystem. To enhance the robustness of vessel control systems, we employ neural network (NN) approximators to approximate uncertain vessel dynamics and present adaptive control technique to estimate the upper boundedness of external disturbances. Under the reinforcement learning framework, we construct actor–critic networks to solve the Hamilton–Jacobi–Bellman equations corresponding to subsystems of surface vessel to achieve the optimized control. The optimized control algorithm can synchronously train the adaptive parameters not only for actor–critic networks but also for NN approximators and adaptive control. By Lyapunov stability theorem, we show that the RLOTC algorithm can ensure the semiglobal uniform ultimate boundedness of the closed-loop systems. Compared with the existing reinforcement learning control results, the presented RLOTC algorithm can compensate for uncertain vessel dynamics and unknown disturbances, and obtain the optimized control performance by considering optimization in every backstepping design. Simulation studies on an underactuated surface vessel are given to illustrate the effectiveness of the RLOTC algorithm.

Learning capability is of great significance for improving the autonomy of the marine surface vehicle (MSV). In this paper, we aim to address the problem of exerting the cooperative learning capability of the formation control system despite the presence of input saturation and switching topology. A distributed adaptive state estimator is developed for the MSVs to achieve precise estimation of the virtual leader’s states. To eliminate the input saturation’s adverse effects on control and learning, an auxiliary system and a dynamic estimator are constructed, respectively. Based on the dynamic estimator, a novel cooperative neural network (NN) weight update law is developed. Subsequently, a neural formation control strategy is proposed for the multi-MSV system. With the help of deterministic learning theory, this collaborative NN-based formation control is then shown to be capable of achieving locally-accurately learning of the uncertain system dynamics along the union orbit of each MSV, where the learned knowledge is shown to be stored in constant NN model afterwards. Furthermore, this obtained knowledge is re-utilized to develop an experience-based formation control algorithm to reduce computational burden and save energy. Finally, simulations are performed to demonstrate the efficacy of the proposed formation control strategies.

An adaptive optimal consensus control design technique is presented for uncertain multiagent systems with prescribed performance guarantees using reinforcement learning (RL) algorithm. Firstly, an adaptive neural network identifier is employed to learn the knowledge of uncertain system dynamics and a disturbance observer is developed to compensate for time-varying disturbances. Secondly, a critic-network learning structure is established to obtain the approximate solution of Hamilton-Jacobi-Bellman (HJB) equations of multiagent systems. Then, an experience replay method is applied to update the critic network weights without requiring persistence of excitation condition. Thirdly, RL-based optimized consensus controllers are developed such that (i) the cost function is minimized, (ii) transient and steady-state performances of consensus error systems are guaranteed, and (iii) uniform ultimate boundedness of the closed-loop systems is achieved. Finally, application to consensus control of unmanned surface vehicles with uncertain hydrodynamic dampings is given to demonstrate the effectiveness of the optimal control design technique.

This article presents an online reinforcement learning (RL) algorithm to learn the distributed optimal containment control solution for underactuated surface vehicles subject to modeling uncertainties and unknown ocean disturbances. First, the request of exact knowledge of vehicle system dynamics is avoided by constructing an adaptive neural network identifier. The unknown lumped disturbances are estimated by disturbance observers. Second, an actor-critic-based RL algorithm is developed to release the persistence of excitation condition. Third, based on RL algorithm and backstepping procedure, we design adaptive finite-time optimal containment controllers such that the containment errors converge into a small neighborhood around zero in finite time and satisfy the predefined performance specifications. Finally, simulation studies on underactuated surface vehicles are provided to verify the effectiveness of the presented optimal control algorithm.

Vision-based Marine ship detection has been explored for many years, effectively improving maritime transportation management. However, many ship detection methods get troubles on the movable platform, which are summarized as follows: 1) high time efficiency is needed to handle the rapid changing of maritime scenes so that ships can be detected successfully on movable platform; 2) different appearances caused by sizes and viewpoints enlarge the intra-class distance of ships, which exceed the representation capability of some features like histogram of oriented gradients (HOG) with fixed size. For addressing these issues, we propose a rapid ship detection method based on multi-size gradient features and multi-branch support vector machine (SVM) in a ``coarse-to-fine'' manner that can be applied on a movable platform. The proposed multi-size gradient features are used to represent the ships with different sizes, including the coarse and fine gradient features. To speed up the detection process in a sliding way, the coarse ship locations are firstly generated only based on the coarse gradient features, which highly reduces the computational cost. Then, the multi-size gradient features extracted from these locations are determined by a multi-branch SVM model which is designed to deal with features of different dimensions and improve the precision of ship position with fine gradient features. The proposed method is tested on the changing background where ships have different sizes. Experimental results show that the proposed method obtains average precision (AP) of 0.795 and its detection speed achieves 60.6 frame per second, which achieve real-time performance and satisfactory detection precision.