Unmanned aerial vehicles (UAVs) are commonly employed in pursuit and rescue missions, where the target’s trajectory is unknown. Traditional methods, such as evolutionary algorithms and ant colony optimization, can generate a search route in a given scenario. However, when the scene changes, the solution needs to be recalculated. In contrast, more advanced deep reinforcement learning methods can train an agent that can be directly applied to a similar task without recalculation. Nevertheless, there are several challenges when the agent learns how to search for unknown dynamic targets. In this search task, the rewards are random and sparse, which makes learning difficult. In addition, because of the need for the agent to adapt to various scenario settings, interactions required between the agent and the environment are more comparable to typical reinforcement learning tasks. These challenges increase the difficulty of training agents. To address these issues, we propose the OC-MAPPO method, which combines optimal control (OC) and Multi-Agent Proximal Policy Optimization (MAPPO) with GPU parallelization. The optimal control model provides the agent with continuous and stable rewards. Through parallelized models, the agent can interact with the environment and collect data more rapidly. Experimental results demonstrate that the proposed method can help the agent learn faster, and the algorithm demonstrated a 26.97% increase in the success rate compared to genetic algorithms.

Unmanned aerial vehicles (UAVs) are becoming more and more widely used in battlefield reconnaissance and target strikes because of their high cost-effectiveness, but task planning for large-scale UAV swarms is a problem that needs to be solved. To solve the high-risk problem caused by incomplete information for the combat area and the potential coordination between targets when a heterogeneous UAV swarm performs reconnaissance and strike missions, this paper proposes a distributed task-allocation algorithm. The method prioritizes tasks by evaluating the swarm’s capability superiority to tasks to reduce the search space, uses the time coordination mechanism and deterrent maneuver strategy to reduce the risk of reconnaissance missions, and uses the distributed negotiation mechanism to allocate reconnaissance tasks and coordinated strike tasks. The simulation results under the distributed framework verify the effectiveness of the distributed negotiation mechanism, and the comparative experiments under different strategies show that the time coordination mechanism and the deterrent maneuver strategy can effectively reduce the mission risk when the target is unknown. The comparison with the centralized global optimization algorithm verifies the efficiency and effectiveness of the proposed method when applied to large-scale UAV swarms. Since the distributed negotiation task-allocation architecture avoids dependence on the highly reliable network and the central node, it can further improve the reliability and scalability of the swarm, and make it applicable to more complex combat environments.

As a promising approach to the rendezvous problem in cognitive radio networks (CRNs), a blind rendezvous paradigm has recently spawned various asynchronous channel-hopping (CH) rendezvous schemes that enable any two neighboring secondary users to meet among multiple channels within a finite time, even with no synchronization. However, the rendezvous delay that was derived in existing work is unable to reflect the actual channel access delay performance in multiuser CRNs, as it only relies on hopping patterns of the elaborately designed CH sequences (CHSs) and ignores the impact of network factors (e.g., channel availability and multiuser contention). In this paper, channel access delay is investigated by jointly considering asynchronous CH rendezvous schemes, channel availability, and a medium-access control (MAC) protocol in a single-hop multiuser CRN. To coordinate multiuser contention, a carrier sense multiple access with collision avoidance (CSMA/CA) MAC is adopted by tailoring the IEEE 802.11 distributed coordination function (DCF) properly to the operation features of existing asynchronous CH rendezvous schemes. The channel access delay is analyzed based on a modified Bianchi model and an absorbing Markov chain model, which captures the aggregate effect of hopping patterns of CHSs, the dynamic nature of channel availability, and the behavior of the MAC protocol. The analytical results are verified through extensive simulations. Both simulation and analytical results reveal that the rendezvous delay in existing asynchronous CH rendezvous schemes is insufficient to ensure satisfactory channel access delay in multiuser CRNs.

Due to dynamic channel availability in cognitive radio networks (CRNs), rendezvous problem is known as the most challenging issue in the design of media access control (MAC) protocol, which is a key step for secondary users (SUs) to start communication. With the concept of blind rendezvous, numerous channel-hopping sequence (CHS) based rendezvous schemes have been proposed to solve this problem in these years. Currently, little attention is paid to the design of a carrier sensing multiple access/collision avoidance (CSMA/CA) MAC based on these rendezvous schemes and also the rendezvous de-synchronization problem brought by multiple rendezvous networking paradigm into MAC design. To this end, we propose a cooperative channel-hopping based CSMA/CA MAC (named CoCH-CSMA/CA MAC) which works on the top of existing CHS based rendezvous schemes. Resulting from the rendezvous de-synchronization problem, a new type of collision, named false collision, is identified. As a SU cannot discern the false collision by itself, we design a cooperative control feedback scheme which employs correlation-based signal detection to reduce the coordination overhead of cooperation and helps SUs to avoid backoff misbehavior. Moreover, we analyze the behavior of our MAC protocol and its advantage. Extensive simulations prove that the cooperative control feedback scheme can effectively alleviate the impact of rendezvous de-synchronization problem on backoff misbehavior and improve network performance.

In this paper, we propose a three-layer distributed simulation network architecture, which consists of a distributed virtual simulation network, a perception and control subnetwork, and a cooperative communication service network. The simulation architecture runs on a distributed platform, which can provide unique virtual scenarios and multiple simulation services for the verification of basic perception, control, and planning algorithms of a single-robot system and can verify the distributed collaboration algorithms of heterogeneous multirobot systems. Further, we design simulation experimental scenarios for classic heterogeneous robotic systems such as unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs). Through the analysis of experimental measurement data, we draw several important conclusions: firstly, the replication time characteristics and update frequency characteristics of entity synchronization in our system indicate that the replication time of entity synchronization in our system is relatively short, and the update frequency can meet the needs of multirobot collaboration and ensure the real-time use and accuracy of the system; secondly, we analyze the bandwidth usage of data frames in the whole session and observe that the server side occupies almost half of the data throughput during the whole session, which indicates that the allocation and utilization of data transmission in our system is reasonable; and finally, we construct a bandwidth estimation surface model to estimate the bandwidth requirements of the current model when scaling the server-side scale and synchronization-state scale, which provides an important reference for better planning and optimizing of the resource allocation and performance of the system. Based on this distributed simulation framework, future research will improve the key technical details, including further refining the coupling object dynamic model update method to support the simulation theory of the coupling relationship between system objects, studying the impact of spatiotemporal consistency of distributed systems on multirobot control and decision making, and in-depth research on the impact of collaborative frameworks combined with multirobot systems for specific tasks.