The popularization of unmanned aerial vehicles (UAVs) has boosted various civil applications such as traffic monitoring, in which the effective coordination of the UAV swarm plays a significant role in expanding the monitoring range and enhancing the execution efficiency. However, due to the isolated local environments as well as the heterogeneous execution capabilities, it is challenging to achieve highly consistent actions. In this article, we incorporate the federated learning framework with the imitation learning technique to coordinate the UAVs' maneuvers by interactively imitating the leader UAV's operations. During the interagent global model download phase, we utilize the generative adversarial imitation learning (GAIL) model to accurately follow the leader UAV's operations by removing the biased estimates of imitation parameters. While in the intraagent local model training phase, we utilize the self-imitation learning (SIL) model to correct delicate imitation errors by virtue of the follower UAVs' own historical valuable experiences. In order to achieve more efficient distributed parameter interactions, we regularize the federated gradient updates and eventually yield coordinated swarm policies. We evaluate the proposed algorithm in the UAV-based traffic monitoring scenario. Evaluation results demonstrate the superiorities on training and execution efficiencies.

Cellular networks are difficult to meet emergency rescue due to the destruction of base stations and infrastructure caused by natural disasters. Unmanned Ground Vehicles (UGVs) and other mobile communication devices encounter significant challenges when operating in disaster areas due to limited coverage and resources. To tackle this problem, this paper integrates Unmanned Aerial Vehicles (UAVs) into the emergency communication network and constructs an air-ground integration network architecture with UAV-UGV collaboration. Specifically, multi-UGV collaborate to collect disaster information, and multi-aerial intelligent reflecting surfaces with high maneuverability can effectively assist UGVs in transmitting the collected data to the remote control center. However, there is also a serious challenge to optimize the collaboration strategy between UGVs and UAVs. To address the concern, the collaboration between UAVs and UGVs is modeled as bipartite graph, where UAVs and UGVs belong to different sets of nodes, respectively. The problem is transformed into a matching game based on the bipartite graph. A stable Bidirectional Matching Game (BMG) algorithm is proposed, where matching players maximize the utility by adjusting the selection strategy. Extensive experimental results show that the proposed BMG algorithm outperforms other benchmark algorithms in terms of utility for both UAVs and UGVs.

Unmanned Aerial Vehicles (UAVs) can be deployed as aerial wireless base stations which dynamically cover the wireless communication networks for Ground Users (GUs). The most challenging problem is how to control multi-UAVs to achieve on-demand coverage of wireless communication networks while maintaining connectivity among them. In this paper, the cooperative trajectory optimization of UAVs is studied to maximize the communication efficiency in the dynamic deployment of UAVs for emergency communication scenarios. We transform the problem into a Markov game problem and propose a distributed trajectory optimization algorithm, Double-Stream Attention multi-agent Actor-Critic (DSAAC), based on Multi-Agent Deep Reinforcement Learning (MADRL). The throughput, safety distance, and power consumption of UAVs are comprehensively taken into account for designing a practical reward function. For complex emergency communication scenarios, we design a double data stream network structure that provides a capacity for the Actor network to process state changes. Thus, UAVs can sense the movement trends of the GUs as well as other UAVs. To establish effective cooperation strategies for UAVs, we develop a hierarchical multi-head attention encoder in the Critic network. This encoder can reduce the redundant information through the attention mechanism, which resolves the problem of the curse of dimensionality as the number of both UAVs and GUs increases. We construct a simulation environment for emergency networks with multi-UAVs and compare the effects of the different numbers of GUs and UAVs on algorithms. The DSAAC algorithm improves communication efficiency by 56.7%, throughput by 71.2%, energy saving by 19.8%, and reduces the number of crashes by 57.7%.

6G Internet of Things (IoT) is envisioned to provide large-scale network connections and high data transmission rates to satisfy the diverse needs of IoT nodes. The wireless powered communication network (WPCN) is the essential part of the future 6G IoT, which can provide nodes with reliable and efficient data and energy transmission. In complex environments, wireless power transmissions are inefficient due to transmission distance and obstacles. To address these concerns, we propose a novel quality of experience (QoE)-driven framework for aerial intelligent reflective surface (IRS)-assisted WPCN, which exploits the maneuverability of unmanned aerial vehicle (UAV) to improve the network performance. In the framework, we construct a nonlinear satisfaction function to quantify the QoE and design an adaptive reflective units configuration scheme based on the QoE to reduce resource consumption (e.g., energy) while satisfying the QoE requirements. The optimization problem of maximizing average throughput is formulated by jointly optimizing the aerial IRS flight trajectory, node association variable, time slot allocation ratio, and IRS phase. The existence of coupling between optimization variables and the nonconvexity lead to the difficulty of solving the optimization problem directly. To effectively solve the above optimization problem, the block coordinate descent (BCD) algorithm is utilized to decompose the optimization problem into four subproblems to be solved separately. Simulation results demonstrate that the proposed scheme can significantly enhance the throughput compared with other schemes.

Multi-agent multi-target search strategies can be utilized in complex scenarios such as post-disaster search and rescue by unmanned aerial vehicles. To solve the problem of fixed target and trajectory, the current multi-agent multi-target search strategies are mainly based on deep reinforcement learning (DRL). However, the training of agents by the DRL tend to be brittle due to their sensitivity to the training environment, which makes the strategies learned by the agents fall into local optima frequently, resulting in poor system robustness. Additionally, sparse rewards in DRL will lead to the problems such as difficulty in system convergence and low utilization efficiency of the sampled data. To address the problem that the robustness of the agents is weakened and the sparse rewards exist in the multi-objective search environment, we propose a MiniMax Multi-agent Deep Deterministic Policy Gradient based on the Parallel Hindsight Experience Replay (PHER-M3DDPG) algorithm, which adopts the framework of centralized training and decentralized execution in continuous action space. To enhance the system robustness, the PHER-M3DDPG algorithm employs a minimax learning architecture, which adaptively adjusts the learning strategy of agents by involving adversarial disturbances. In addition, to solve the sparse rewards problem, the PHER-M3DDPG algorithm adopts a parallel hindsight experience replay mechanism to increase the efficiency of data utilization by involving virtual learning targets and batch processing of the sampled data. Simulation results show that the PHER-M3DDPG algorithm outperforms the existing algorithms in terms of convergence speed and the task completion time in a multi-target search environment.

Localization is a key technology to ensure the effective operation of wireless sensor networks in different environments. Due to the prevalence of cyber-attacks in real-world application scenarios, ensuring the accuracy of localization under denial-of-service (DoS) attacks is a growing concern. Existing research focuses on distributed localization of static sensor networks under DoS attacks. This article aims to extend the study of distributed localization in mobile sensor networks subject to DoS attacks. Under DoS attacks, communication between sensor nodes can become intermittent, resulting in the time-varying characteristic for communication networks among all sensor nodes, which poses a challenge for successful localization. To overcome this challenge, this article proposes a distributed iterative localization algorithm using relative barycentric coordinates and distance measurements. Based on a hybrid approach composed of graph composition and sub-stochastic matrix, a comprehensive analysis of the convergence, rate and complexity of the localization algorithm is presented. At last, the theoretical results are verified by experimental examples.