Unmanned aerial vehicles (UAVs) serve as aerial base stations to provide controlled wireless connections for ground users. Due to their constraints on both mobility and energy consumption, a key problem is how to deploy UAVs adaptively in a geographic area with changing traffic demand of mobile users, while meeting the aforementioned constraints. In this article, we propose a Quality of Experience (QoE)-driven and energy-efficient adaptive deployment strategy for multi-UAV networks based on hybrid deep reinforcement learning (DRL) to solve the problem of incomplete information game, where the UAVs can adjust their moving directions and distance to serve users who move randomly in the target area. Through the hybrid DRL with centralized training and distributed testing, UAVs can be trained offline to obtain the global state information and learn a completely distributed control strategy, with which each UAV only needs to take actions based on its observed state in the real deployment to be fully adaptive. Moreover, in order to improve the speed and effect of learning, we improve hybrid reinforcement learning, by adding genetic algorithms and temporal difference error-based resampling optimization mechanism. The simulation results show that the hybrid DRL algorithm has better efficiency and robustness in multi-UAV control, and has better performance in terms of QoE, energy consumption, and average throughput, by which average throughput can be increased by 20%–60%.

Flocking aims to drive a group of agents connecting with each other to interact in a complex way, thereby emerging the behavior of group aggregation. Owing to the prevalence and complexity of cooperation and competition relationships among agents, it is practical and challenging to explore the realization of flocking behavior on multiagent coopetition networks. With that in mind, this article focuses on the study of flocking behavior for Cucker–Smale multiagent model over coopetition networks with communication delays. In this model, the cooperation/competition degree between agents is portrayed as a weight function with respect to communication distance, e.g., the closer (farther) the communication distance, the stronger (weaker) the cooperation/competition degree. With the help of analysis tools consisting of edge composites and substochastic matrices, the algebraic relationships between cooperation and competition degrees are established to ensure the emergence of flocking behavior. Moreover, mathematical expressions are given to show the effect of different communication delays on the agents’ final aggregation upper bound and convergence rate. In the end, the theoretical results are validated through computer simulations.

Cellular networks are difficult to meet emergency rescue due to the destruction of base stations and infrastructure caused by natural disasters. Unmanned Ground Vehicles (UGVs) and other mobile communication devices encounter significant challenges when operating in disaster areas due to limited coverage and resources. To tackle this problem, this paper integrates Unmanned Aerial Vehicles (UAVs) into the emergency communication network and constructs an air-ground integration network architecture with UAV-UGV collaboration. Specifically, multi-UGV collaborate to collect disaster information, and multi-aerial intelligent reflecting surfaces with high maneuverability can effectively assist UGVs in transmitting the collected data to the remote control center. However, there is also a serious challenge to optimize the collaboration strategy between UGVs and UAVs. To address the concern, the collaboration between UAVs and UGVs is modeled as bipartite graph, where UAVs and UGVs belong to different sets of nodes, respectively. The problem is transformed into a matching game based on the bipartite graph. A stable Bidirectional Matching Game (BMG) algorithm is proposed, where matching players maximize the utility by adjusting the selection strategy. Extensive experimental results show that the proposed BMG algorithm outperforms other benchmark algorithms in terms of utility for both UAVs and UGVs.

Unmanned Aerial Vehicles (UAVs) can be deployed as aerial wireless base stations which dynamically cover the wireless communication networks for Ground Users (GUs). The most challenging problem is how to control multi-UAVs to achieve on-demand coverage of wireless communication networks while maintaining connectivity among them. In this paper, the cooperative trajectory optimization of UAVs is studied to maximize the communication efficiency in the dynamic deployment of UAVs for emergency communication scenarios. We transform the problem into a Markov game problem and propose a distributed trajectory optimization algorithm, Double-Stream Attention multi-agent Actor-Critic (DSAAC), based on Multi-Agent Deep Reinforcement Learning (MADRL). The throughput, safety distance, and power consumption of UAVs are comprehensively taken into account for designing a practical reward function. For complex emergency communication scenarios, we design a double data stream network structure that provides a capacity for the Actor network to process state changes. Thus, UAVs can sense the movement trends of the GUs as well as other UAVs. To establish effective cooperation strategies for UAVs, we develop a hierarchical multi-head attention encoder in the Critic network. This encoder can reduce the redundant information through the attention mechanism, which resolves the problem of the curse of dimensionality as the number of both UAVs and GUs increases. We construct a simulation environment for emergency networks with multi-UAVs and compare the effects of the different numbers of GUs and UAVs on algorithms. The DSAAC algorithm improves communication efficiency by 56.7%, throughput by 71.2%, energy saving by 19.8%, and reduces the number of crashes by 57.7%.