Ship path planning is a central challenge in autonomous navigation for unmanned surface vehicles and maritime autonomous surface ships. It is not simply a shortest-path problem, but a constrained sequential decision process that must reconcile collision risk, route efficiency, COLREGs compliance, vessel dynamics, and environmental uncertainty. Here we review the field through a unified framework based on planning scope, decision basis, and deployment requirements. We examine search- and sampling-based, geometric and rule-based, optimization-based, learning-driven, and hybrid methods, with particular emphasis on deep reinforcement learning for discrete decisions, continuous manoeuvring, multi-vessel interaction, and safety-oriented control. Representative studies are compared across objective and reward design, state representation, exploration and policy optimization, rule integration, disturbance modelling, simulation platforms, and operational validation. The synthesis identifies persistent barriers, including ambiguous rule formalization, partial observability, strategic coupling among vessels, inconsistent benchmarks, limited cross-scenario generalization, and insufficient full-scale validation. We further discuss priority directions in explicit safety constraints, digital twins, transfer and meta-learning, world models, scalable multi-agent coordination, and large-model-assisted mission reasoning. We argue that progress will depend less on further algorithmic proliferation than on integrated, verifiable architectures that combine data-driven adaptation with model-based structure, standardized evaluation, and staged real-world assurance.

Performance degradation caused by temporal-difference signal fluctuations and the inflexibility of fixed proximal constraints remains a key challenge in policy optimization algorithms. To address these issues, this article develops a Coherence-Adaptive Proximal Optimization (CAPO) method. We first derive a temporal-coherence-aware advantage estimation mechanism by measuring the directional consistency of temporal-difference residuals within a local time window. Based on this mechanism, short-horizon and long-horizon advantage estimates are adaptively integrated to provide a more stable and temporally reliable policy improvement signal. CAPO is then instantiated within the PPO framework by incorporating the temporal coherence mechanism into both advantage construction and proximal policy updates. The proposed method preserves the stable optimization structure of PPO while adaptively adjusting the clipping range according to sample-wise temporal coherence. In this way, CAPO allows more sufficient policy improvement when the learning signal is temporally consistent and imposes more conservative updates when the signal is unstable. Experiments on six representative OpenAI Gym control tasks show that CAPO achieves better or comparable performance than standard PPO in most benchmark environments, with statistically reliable improvements in three tasks and comparable final performance in two tasks. Meanwhile, the results also show that CAPO does not uniformly outperform PPO in all environments, indicating that its advantage is task-dependent. Overall, CAPO improves training stability and convergence behavior in several representative tasks, especially when local temporal feedback provides reliable policy-improvement information.

Performance degradation caused by temporal-difference signal fluctuations and the inflexibility of fixed proximal constraints remains a key challenge in policy optimization algorithms. To address these issues, this article develops a Coherence-Adaptive Proximal Optimization (CAPO) method. We first derive a temporal-coherence-aware advantage estimation mechanism by measuring the directional consistency of temporal-difference residuals within a local time window. Based on this mechanism, short-horizon and long-horizon advantage estimates are adaptively integrated to provide a more stable and temporally reliable policy improvement signal. CAPO is then instantiated within the PPO framework by incorporating the temporal coherence mechanism into both advantage construction and proximal policy updates. The proposed method preserves the stable optimization structure of PPO while adaptively adjusting the clipping range according to sample-wise temporal coherence. In this way, CAPO allows more sufficient policy improvement when the learning signal is temporally consistent and imposes more conservative updates when the signal is unstable. Experiments on several representative OpenAI Gym control tasks show that CAPOachieves better or comparable performance than standard PPO in most benchmark environments, with improved training stability and convergence behavior, especially in tasks where local temporal feedback provides reliable policy improvement information.

Ship path planning is a central challenge in autonomous navigation for unmanned surface vehicles and maritime autonomous surface ships. It is not simply a shortest-path problem, but a constrained sequential decision process that must reconcile collision risk, route efficiency, COLREGs compliance, vessel dynamics, and environmental uncertainty. Here we review the field through a unified framework based on planning scope, decision basis, and deployment requirements. We examine search- and sampling-based, geometric and rule-based, optimization-based, learning-driven, and hybrid methods, with particular emphasis on deep reinforcement learning for discrete decisions, continuous maneuvering, multi-vessel interaction, and safety-oriented control. Representative studies are compared across objective and reward design, state representation, exploration and policy optimization, rule integration, disturbance modeling, simulation platforms, and operational validation. The synthesis identifies persistent barriers, including ambiguous rule formalization, partial observability, strategic coupling among vessels, inconsistent benchmarks, limited cross-scenario generalization, and insufficient full-scale validation. We further discuss priority directions in explicit safety constraints, digital twins, transfer and meta-learning, world models, scalable multi-agent coordination, and large-model-assisted mission reasoning. We argue that progress will depend less on further algorithmic proliferation than on integrated, verifiable architectures that combine data-driven adaptation with model-based structure, standardized evaluation, and staged real-world assurance.

Deep reinforcement learning (DRL) offers a promising approach to autonomous navigation and collision avoidance for unmanned surface vehicles (USVs) by learning decision-making policies through interaction with maritime environments. However, existing methods often couple goal-directed navigation and obstacle avoidance within a single policy network, without explicitly distinguishing their contributions or regulating the strength of avoidance intervention. This implicit coupling may induce premature deviations from the intended course or inadequate avoidance maneuvers, reducing navigation efficiency and increasing collision risk. To address this limitation, we propose proximal policy optimization with a policy-splitting network (PSN-PPO), which restructures the PPO actor into jointly trained navigation and avoidance branches. The navigation branch generates a nominal navigation output, while the avoidance branch provides a bounded correction. A learnable risk gate modulates this correction based on encounter risk assessed over a finite prediction horizon. In the absence of obstacles or when the predicted risk falls below the activation threshold, the policy mean is determined solely by the navigation branch. A closed-loop simulation framework incorporating a three-degree-of-freedom vessel motion model is used to compare PSN-PPO with standard PPO across five scenarios: obstacle-free navigation, static-obstacle avoidance, head-on encounters, overtaking, and crossing. Both methods are trained independently in each scenario. Simulation results show that PSN-PPO improves navigation efficiency and collision-avoidance performance relative to standard PPO in the evaluated scenarios.

This study examines whether an explicit separation of navigation and avoidance yields useful control structure and performance gains for small unmanned surface vehicles in simulation. The policy-splitting proximal policy optimization actor (PSN-PPO) combines a navigation mean, a bounded avoidance residual, and a geometric risk gate with a learnable exponent, while retaining the PPO update and a separate critic. Five independent models per method are trained in each of five scenarios and evaluated on common nominal and perturbed cases. Crossing arrival is 100% for PSN-PPO versus 80% for PPO, but arrival without soft-domain intrusion is $25.6\%$ versus $37.6\%$; static-obstacle arrival is $50.4\%$ versus $80\%$. A 48-run equal-interaction study in static and crossing tasks finds no consistent advantage of full gating over always-on correction. Navigation-only control also achieves full crossing arrival, and the learned gate exponent remains close to its initial value. Simulated braking, turning, and paired threshold tests characterize the numerical maneuvering and risk scales. The contribution is an explicitly constrained actor decomposition and an empirical assessment of its limits; the results do not establish an additional performance benefit from the complete adaptive gate in these benchmark families.