Autonomous pilot is crucial in integrally promoting the autonomy of an unmanned surface vehicle (USV). However, the integration mechanism of decision and control is still unclear within the entire autonomy. In this paper, by organically bridging path planning and tracking, an autonomous pilot framework with waypoints generation, path smoothing and policy guidance of a USV in congested waters is established, for the first time. Incorporating elite and diversity operations into the genetic algorithm (GA), an elite-duplication GA (EGA) strategy is devised to optimally generate sparse waypoints in a constrained space. The B-spline technique is further deployed to make flexibly smooth interpolation facilitating path smoothing supported by optimal sparse-waypoints. Seamlessly bridged by the parametric smooth path, deep reinforcement learning (DRL) technique is resorted to continuously extract in-depth pilotage policies, i.e., mappings from path tracking errors, collision risks and control constraints to continuous control forces/torques. Eventually, the entire spline-bridged EGA-DRL (SED) framework merits autonomous global-pilotage and local-reaction in an organically modular manner. Comprehensive validations and comparisons in various real-world geographies demonstrate the effectiveness and superiority of the proposed SED autonomous pilot framework.

Most of the existing typical ship domains have been comprehensively reviewed and classified. Most of these ship domains are described in a geometrical manner that is difficult to apply to practices and simulations in marine traffic engineering. According to different types of geometrical ship domains, we have proposed mathematical models, based on which a unified analytical framework has been established. It is feasible and practical for the analytical models to be applied to the assessment of navigational safety, collision avoidance and trajectory planning, etc. Finally, some computer simulations and comparative studies of the proposed domain model have been presented and the simulation results show that the uniform analytical framework for ship domains is effective and identical to the original geometrical ones. It should be noted that the analytical domain models could be directly applied in any collision risk, collision avoidance or VTS system while the geometrical ones would be more illustrative but less practical or analytical.

Collaborative control of a dual-arm robot refers to collision avoidance and working together to accomplish a task. To prevent the collision of two arms, the control strategy of a robot arm needs to avoid competition and to cooperate with the other one during motion planning. In this paper, a dual-arm deep deterministic policy gradient (DADDPG) algorithm is proposed based on deep reinforcement learning of multi-agent cooperation. Firstly, the construction method of a replay buffer in a hindsight experience replay algorithm is introduced. The modeling and training method of the multi-agent deep deterministic policy gradient algorithm is explained. Secondly, a control strategy is assigned to each robotic arm. The arms share their observations and actions. The dual-arm robot is trained based on a mechanism of “rewarding cooperation and punishing competition”. Finally, the effectiveness of the algorithm is verified in the Reach, Push, and Pick up simulation environment built in this study. The experiment results show that the robot trained by the DADDPG algorithm can achieve cooperative tasks. The algorithm can make the robots explore the action space autonomously and reduce the level of competition with each other. The collaborative robots have better adaptability to coordination tasks.

Grey wolf optimization (GWO) is a meta-heuristic algorithm inspired by the hierarchy and hunting behavior of grey wolves. GWO has the superiorities of simpler concept and fewer adjustment parameters, and has been widely used in different fields. However, there are some disadvantages in avoiding prematurity and falling into local optimum. This paper presents an improved grey wolf optimization (IGWO) to ameliorate these drawbacks. Firstly, a modified position update mechanism for pursuing high quality solutions is developed. By designing an ameliorative position update formula, a proper balance between the exploration and exploitation is achieved. Moreover, the leadership hierarchy is strengthened by proposing adaptive weights of α, β and δ. Then, a dynamic local optimum escape strategy is proposed to reinforce the ability of the algorithm to escape from the local stagnations. Finally, some individuals are repositioned with the aid of the positions of the leaders. These individuals are pulled to new positions near the leaders, helping to accelerate the convergence of the algorithm. To verify the effectiveness of IGWO, a series of contrast experiments are conducted. On the one hand, IGWO is compared with some state-of-the-art GWO variants and several promising meta-heuristic algorithms on 20 benchmark functions. Experimental results indicate that IGWO performs better than other competitors. On the other hand, the applicability of IGWO is verified by a robot global path planning problem, and simulation results demonstrate that IGWO can plan shorter and safer paths. Therefore, IGWO is successfully applied to the path planning as a new method.