This paper aims to solve the path following problem for an underactuated unmanned-surface-vessel (USV) based on deep reinforcement learning (DRL). A smoothly-convergent DRL (SCDRL) method is proposed based on the deep Q network (DQN) and reinforcement learning. In this new method, an improved DQN structure was developed as a decision-making network to reduce the complexity of the control law for the path following of a three-degree of freedom USV model. An exploring function was proposed based on the adaptive gradient descent to extract the training knowledge for the DQN from the empirical data. In addition, a new reward function was designed to evaluate the output decisions of the DQN, and hence, to reinforce the decision-making network in controlling the USV path following. Numerical simulations were conducted to evaluate the performance of the proposed method. The analysis results demonstrate that the proposed SCDRL converges more smoothly than the traditional deep Q learning while the path following error of the SCDRL is comparable to existing methods. Thanks to good usability and generality of the proposed method for USV path following, it can be applied to practical applications.

This article addresses the problem of path following for underactuated unmanned surface vessels (USVs) formation via a modified deep reinforcement learning with random braking (DRLRB). A formation control model based on deep reinforcement learning (DRL) is constructed to urge USVs to form a preset formation. Specifically, an efficient reward function is designed from the perspective of velocity and error distance of each USV related to the given formation, and then a novel random braking mechanism is formulated to prevent the training of the decision-making network from falling into the local optimum and failing to achieve the training objectives. Following that, a virtual leader-based path-following guidance system is developed for the USV formation problem. Wherein, with the aid of DRLRB, our proposed system can adjust formation automatically and flexibly even when some USVs deviate from the formation. Simulation verifies the effectiveness and superiority of our formation and path-following control strategy.

This paper investigates the tracking control problem of marine surface vessels (MSVs) in the presence of uncertain dynamics and external disturbances. The facts that actuators are subject to undesirable faults and input saturation are taken into account. Benefiting from the smoothness of the Gaussian error function, a novel saturation function is introduced to replace each nonsmooth actuator saturation nonlinearity. Applying the hand position approach, the original motion dynamics of underactuated MSVs are transformed into a standard integral cascade form so that the vector design method can be used to solve the control problem for underactuated MSVs. By combining the neural network technique and virtual parameter learning algorithm with the vector design method, and introducing an event triggering mechanism, a novel event-triggered indirect neuroadaptive fault-tolerant control scheme is proposed, which has several notable characteristics compared with most existing strategies: 1) it is not only robust and adaptive to uncertain dynamics and external disturbances but is also tolerant to undesirable actuator faults and saturation; 2) it reduces the acting frequency of actuators, thereby decreasing the mechanical wear of the MSV actuators, via the event-triggered control (ETC) technique; 3) it guarantees stable tracking without the a priori knowledge of the dynamics of the MSVs, external disturbances or actuator faults; and 4) it only involves two parameter adaptations—a virtual parameter and a lower bound on the uncertain gains of the actuators—and is thus more affordable to implement. On the basis of the Lyapunov theorem, it is verified that all signals in the tracking control system of the underactuated MSVs are bounded. Finally, the effectiveness of the proposed control scheme is demonstrated by simulations and comparative results.

This paper presents a novel cooperative unmanned surface vehicle-unmanned aerial vehicle (USV-UAV) platform to form a powerful combination, which offers foundations for collaborative task executed by the coupled USV-UAV systems. Adjustable buoys and unique carrier deck for the USV are designed to guarantee landing safety and transportation of UAV. The deck of USV is equipped with a series of sensors, and a multiultrasonic joint dynamic positioning algorithm is introduced for resolving the positioning problem of the coupled USV-UAV systems. To fulfill effective guidance for the landing operation of UAV, we design a hierarchical landing guide point generation algorithm to obtain a sequence of guide points. By employing the above sequential guide points, high-quality paths are planned for the UAV. Cooperative dynamic positioning process of the USV-UAV systems is elucidated, and then UAV can achieve landing on the deck of USV steadily. Our cooperative USV-UAV platform is validated by simulation and water experiments.