Deep reinforcement learning is an artificial intelligence technology that combines deep learning and reinforcement learning and has been widely applied in multiple fields. As a type of deep reinforcement learning algorithm, the A3C (Asynchronous Advantage Actor-Critic) algorithm can effectively utilize computer resources and improve training efficiency by synchronously training Actor-Critic in multiple threads. Inspired by the excellent performance of the A3C algorithm, this paper uses the A3C algorithm to solve the UUV (Unmanned Underwater Vehicle) collision avoidance planning problem in unknown environments. This collision avoidance planning algorithm can have the ability to plan in real-time while ensuring a shorter path length, and the output action space can meet the kinematic constraints of UUVs. In response to the problem of UUV collision avoidance planning, this paper designs the state space, action space, and reward function. The simulation results show that the A3C collision avoidance planning algorithm can guide a UUV to avoid obstacles and reach the preset target point. The path planned by this algorithm meets the heading constraints of the UUV, and the planning time is short, which can meet the requirements of real-time planning.

Underwater target tracking with signal propagation delay is a major challenge. The widely-used unscented Gauss-Helmert filter (UGHF) algorithm shows good tracking performance in bearing-only filters with signal propagation delay. However, UGHF algorithm is a nonlinear filtering method based on 2-order unscented transformation (UT), which can only match the accuracy of the 3-order Taylor expansion term of Gauss-Hermite model (GHM). Moreover, UGHF is a Gaussian filter, and when the measurement noise presents non-Gaussian characteristics, the tracking accuracy will be reduced. In order to improve the tracking accuracy of underwater bearing-only target, this paper proposes a high-order UGHF (HOUGHF) algorithm using high-order UT instead of 2-order UT. In order to further improve the tracking accuracy of the tracking algorithm under non-Gaussian noise, in this paper, maximum correntropy criterion (MCC) is used to replace the original minimum mean square error (MMSE) to update HOGGHF’s posterior estimation. The maximum correntropy HOGGHF (MCHOGGHF) algorithm is proposed by using MCC based on Gaussian kernel function. At the same time, in order to deal with the change of measurement noise, a method of adaptive Gaussian kernel width is presented. Simulation and sea trial results show that the two algorithm are superior to UGHF algorithm and MCHOGGH algorithm is superior to HOGGHF algorithm in estimation accuracy under non-Gaussian noise.

The complex underwater environment poses significant challenges for unmanned underwater vehicles (UUVs), particularly in terms of communication constraints and the need for precise cooperative obstacle avoidance and trajectory tracking. Addressing these challenges solely through position information is crucial in this field. This study explores the intricate task of managing a group of UUVs as they navigate obstacles and follow a given trajectory, all based on position information. A new dynamic interactive topology framework utilizing sonar technology has been developed for the UUVs. This framework not only provides position information for the UUV swarm but also for the surrounding obstacles, enhancing situational awareness. Additionally, a bio-inspired cooperative control strategy designed for UUV swarms utilizing sonar interaction topology is introduced. This innovative method eliminates the need for velocity data from neighboring UUVs, instead relying solely on position information to achieve swarm cooperative control, obstacle avoidance, and trajectory adherence. The effectiveness of this method is validated through extensive simulations. The results show that the proposed method demonstrates improved sensitivity in obstacle detection, enabling faster trajectory tracking while maintaining a safer distance compared to traditional methods. Ultimately, this innovative strategy not only enhances operational efficiency but also enhances safety measures in UUV swarm operations.

Multi-platform bearing-only target tracking poses a significant challenge in underwater target tracking. Presently, the field often relies on suboptimal nonlinear filters, which unfortunately exhibit numerical instability when confronted with highly nonlinear systems. Hence, this article suggests employing the Maximum Likelihood Kalman Filtering (MLKF) algorithm for multi-platform bearing-only target tracking. This approach first obtains the Maximum Likelihood Estimate (MLE) of the bearings from multiple nonlinear measurements, which are then converted into the target’s position in cartesian coordinates for Kalman Filter (KF). The KF algorithm is derived based on the assumption of Gaussian noise. However, in engineering practice, measurement noise is often non-Gaussian, which can lead to a decline in tracking accuracy of the KF algorithm. To address this, the Maximum Likelihood Maximum Correntropy KF (MLMCKF) algorithm is proposed, which utilizes the Maximum Correntropy Criterion (MCC) instead of the original Minimum Mean Square Error (MMSE) criterion. Simulation results show that the measurement values transformed by the ML method follow a Gaussian distribution. Moreover, under non-Gaussian measurement noise, the proposed MCMLKF algorithm achieves superior tracking accuracy compa to existing non-Gaussian algorithms.

The expansion of underwater scenarios and missions highlights the crucial need for autonomous underwater vehicles (AUVs) to make informed decisions. Therefore, developing an efficient decision-making framework is vital to enhance productivity in executing complex tasks within tight time constraints. This paper delves into task planning and reconstruction within the AUV control decision system to enable intelligent completion of intricate underwater tasks. Behavior trees (BTs) offer a structured approach to organizing the switching structure of a hybrid dynamical system (HDS), originally introduced in the computer game programming community. In this research, an intelligent search algorithm, MCTS-QPSO (Monte Carlo tree search and quantum particle swarm optimization), is proposed to bolster the AUV’s capacity in planning complex task decision control systems. This algorithm tackles the issue of the time-consuming manual design of control systems by effectively integrating BTs. By assessing a predefined set of subtasks and actions in tandem with the complex task scenario, a reward function is formulated for MCTS to pinpoint the optimal subtree set. The QPSO algorithm is then leveraged for subtree integration, treating it as an optimal path search problem from the root node to the leaf node. This process optimizes the search subtree, thereby enhancing the robustness and security of the control architecture. To expedite search speed and algorithm convergence, this paper recommends reducing the search space by pre-grouping conditions and states within the behavior tree. The efficacy and superiority of the proposed algorithm are validated through security and timeliness evaluations of the BT, along with comparisons with other algorithms for automatic AUV decision control behavior tree design. Ultimately, the effectiveness and superiority of the proposed algorithm are corroborated through simulations on a multi-AUV complex task platform, showcasing its practical applicability and efficiency in real-world underwater scenarios.