Autonomous unmanned systems have become an attractive vehicle for a myriad of military and civilian applications. This can be partly attributed to their ability to bring payloads for utility, sensing, and other uses for various applications autonomously. However, a key challenge in realizing autonomous unmanned systems is the ability to perform complex group missions, which require coordination and collaboration among multiple platforms. This paper presents a cooperative navigating task approach that enables multiple unmanned surface vehicles (multi-USV) to autonomously capture a maneuvering target while avoiding both static and dynamic obstacles. The approach adopts a hybrid multi-agent deep reinforcement learning framework that leverages heuristic mechanisms to guide the group mission learning of the vehicles. Specifically, the proposed framework consists of two stages. In the first stage, navigation subgoal sets are generated based on expert knowledge, and a goal selection heuristic model based on the immune network model is used to select navigation targets during training. Next, the selected goals’ executions are learned using actor-critic proximal policy optimization. The simulation results with multi-USV target capture show that the proposed approach is capable of abstracting and guiding the unmanned vehicle group coordination learning and achieving a generally optimized mission execution.

With the development of intelligent technology, multi-agent systems have been widely applied in military and civilian fields. Compared to a single platform, multi-agent systems can complete more dangerous, difficult, and heavy tasks. However, due to the limited autonomy of unmanned platforms and the regulatory needs of personnel, multi-agent systems cooperating with manned platforms to perform tasks have been more widely promoted at this stage of development. This paper addresses a differential game method for cooperative decision-making of a multi-agent system cooperating with the manned platform for the target-pursuit problem. The manned platform pursues the target according to a certain trajectory, and its state can be obtained by the multi-agent system. Firstly, for the case that the target moves with a fixed trajectory, the target-pursuit problem in a manned–unmanned environment is viewed in the form of game based on a communication graph among agents. Secondly, strategies of all agents are proposed while maintaining their group cohesion. A set of coupled differential equations is solved to implement strategy calculation. Compared to purely unmanned systems, the strategies combine the advantages of the manned platform and add a reference item, which can achieve team cohesion relatively quickly. Furthermore, a brief analysis is made on the scenarios where the target is in another case or adopts other strategies. Finally, comparative simulations have verified the effectiveness and synergy of the strategy.

An unmanned swarm is usually composed of a group of homogeneous or heterogeneous hardware platforms, software control systems, and interfaces for human–computer interaction that operate collectively to achieve a specific goal by information interaction. They exhibit robustness and fault tolerance when facing complex missions, making it crucial in military, transportation, intelligent traffic, and other fields. However, the coupling between the hardware and software of a heterogeneous unmanned swarm can indeed have significant implications for system flexibility, software development and deployment, and hardware maintenance. Over the years, there has been a significant shift from traditional hardware-focused control systems to a greater emphasis on the core software layer. In this paper, a distributed network architecture is proposed to solve this problem, in which hardware resources are abstracted and represented to accomplish standardization and unification by defining a consistent and uniform set of data formats, and a resource pool of hardware data is constructed to realize the function that the number and scale of platforms is irrelevant, the task module can be plug-and-play at any time, and the software can be configured on demand. The resource scheduling of a single platform is achieved through process and thread communication using shared memory, while the resource scheduling of a cross platform is achieved through a network using request and response and subscription and notification. As a result, it can satisfy the development of functional modules in a software-defined mode and gradually improve the intelligence capability of an unmanned swarm. Based on the above architecture, the overall framework of the autonomous navigation system and the collaborative control system has been successfully established. Finally, a hardware-in-the-loop simulation environment is constructed, and the integration and verification of the proposed distributed architecture is carried out by the cooperative formation experiment, which proves the feasibility of this proposal.