5 February 2024 in IEEE Transactions on Communications
The deployment of UAVs as aerial base stations (BSs) has been considered as a promising supplement to the ground networks, which can quickly build an emergency communication network in a disaster area or significantly relief the communication burden imposed by hot-spots. However, the application of UAVs as aerial BSs is constrained by the limited onboard energy and communication coverage of UAVs. In particular, for a large target area, multiple UAVs should be deployed to meet the communication requirements. Therefore, designing the optimal trajectories of multiple UAVs is crucial to boost the UAV network performance. Inspired by the promising future of UAV BSs, this paper aims at proposing a distributed 3-dimensional (3D) trajectory design algorithm for multiple UAVs to optimize the system performance. We formulate the trajectory design problem as a multi-objective optimization problem to improve the user equipment (UE) access rate, ensure fair access opportunities, increase transmitted data volume and reduce energy consumption. Further inspired by the decision-making ability of deep reinforcement learning (DRL) in complex environments, we propose a DRL based trajectory design algorithm for multiple UAVs, namely DMTD, in which UAVs can explore both the optimal flight altitude and the potential UE distribution area in the iterative interactions with the environment, and then select the optimal flight trajectories to boost the network performance from multiple aspects. Extensive experimental results under different UE distributions have demonstrated that the proposed DMTD algorithm can find the optimal altitude to provide maximum coverage. Moreover, DMTD beats existing algorithms by providing high UE access rate, ensuring fair network service and increasing total transmitted data volume at the cost of a relatively low energy consumption. Especially in the scenes with dense and randomly distributed UEs, DMTD provides a UE access rate close to 0.9 and transmits 6 times of data volume than existing algorithms.
7 October 2025 in IEEE Transactions on Communications
Deploying the Unmanned Aerial Vehicle (UAV) swarm to assist data collection in Internet of Things (IoT) has emerged as a highly promising solution. Motivated by intelligent decision-making abilities, deep reinforcement learning (DRL) has been widely applied to optimize data collection performance by trajectory planning or resource allocation. However, due to the differing timescales at which trajectory planning and resource allocation operate and the excessively large decision space when optimizing them simultaneously, most existing DRL-based algorithms focus on optimizing trajectory planning or resource allocation separately. Moreover, given the mutual influence of trajectory planning and resource allocation on data collection performance, achieving their joint optimization is of paramount importance. To address this issue, we divide the data collection task into three stages, including clustering, trajectory planning, and resource allocation. Then, we propose a hierarchical reinforcement learning-based algorithm, referred to as HRL-TPRA, for coordinated optimization among these stages. In the clustering stage, we develop a balanced-constraint clustering algorithm to partition sensing nodes (SNs) into clusters with similar data collection demands. In the trajectory planning stage, the high-level agent of HRL-TPRA assigns each UAV to an appropriate cluster, reducing the swarm’s energy consumption and optimizing the alignment between available resources and data collection demands. In the resource allocation stage, the low-level agent of HRL-TPRA controls the association between UAVs and SNs, further enhancing data collection efficiency. HRL-TPRA achieves collaborative optimization among all stages, effectively reducing the Age of Information (AoI) for all SNs, and significantly improving the data collection volume and energy efficiency of the UAV swarm. Extensive experimental results demonstrate that HRL-TPRA outperforms existing optimization algorithms in terms of multiple performance metrics.
21 November 2025 in IEEE Transactions on Communications
Motivated by the intelligent decision-making ability, Deep Reinforcement Learning (DRL) has been extensively applied in multi-Unmanned Aerial Vehicle (UAV) trajectory planning. This article investigates the application of DRL in Three-Dimensional (3D) trajectory planning in environments with obstacles, where multiple UAVs act as aerial Base Stations (BSs) to provide services to ground user hotspots. The existing DRL-based algorithms require a significant amount of trial and error iterations to obtain sufficiently high-performing agents. To cope with this drawback, we introduce Transfer Learning (TL) into DRL, enabling the UAV agent to possess prior knowledge upon initialization, thereby quickly adapting to unfamiliar environments and significantly improving performance. Considering the limited local observations of UAVs, a multi-modal fusion autoencoder is proposed for extracting and compressing cross-modal features from global observations to obtain the global fusion state, which enhances the perception capabilities of UAVs without incurring excessive communication overhead. Finally, we propose an exploration novelty-driven collaborative trajectory planning algorithm for multiple UAVs, which ensures obstacle avoidance and enhances the UAVs’ exploration capabilities to cover all hotspots. We adopt a probabilistic channel model and discretize both the time and DRL action space to achieve a balance between practicality and tractability. Extensive experiments demonstrate that our proposed transfer reinforcement learning method can improve the initial performance of the UAV agents by 76%. The perception and exploration-enhanced trajectory planning algorithm significantly increased the exploration efficiency and improved hotspot coverage by 50%.
26 November 2025 in IEEE Transactions on Cognitive Communications and Networking
Deploying the Unmanned Aerial Vehicle (UAV) formation as aerial base stations to construct aerial communication networks in hotspot areas holds considerable promise. However, planning the trajectories of UAV formation in complex and unknown environments, while ensuring obstacle avoidance and formation maintenance, presents unpreceding challenges. In this paper, we propose a hierarchical reinforcement learning-based trajectory planning algorithm for UAV formation. This algorithm implements two-timescale trajectory planning within a leader-follower control framework, where the leader UAV (LUAV) plans the shortest safe trajectory to the hotspot area on a large timescale and the follower UAVs (FUAVs) are responsible for obstacle avoidance and formation maintenance on a small timescale. The LUAV and FUAVs collaborate across different timescales to achieve joint trajectory optimization. To tackle the sparse reward problem in existing learning-based trajectory planning algorithms, we introduce an intrinsic curiosity-driven module that integrates historical information to enhance the exploration of the UAV formation in unknown environments. Our algorithm enhances the UAV formation’s capability to handle complex obstacles and maintain the formation, thereby improving overall performance. Simulation results demonstrate that our algorithm achieves complete obstacle avoidance with a 100% UAV survival rate. Compared to existing algorithms, our proposed algorithm reduces the trajectory length by 14% and improves the formation maintaining performance by over 90%.
10 September 2025 in TechRxiv
Deploying Unmanned Aerial Vehicles (UAVs) for data collection in post-disaster scenarios is a highly promising solution for rapid communication recovery. In this paper, we formulate the trajectory planning for multiple UAVs collecting data from Data Points (DPs) as a combinatorial optimization problem, aiming to minimize the Age of Information (AoI), maximize the timeliness metric and reduce the energy consumption of UAVs to achieve energy-efficient data collection. Existing Deep Reinforcement Learning (DRL) based methods rely on manually designed reward functions, which struggle to effectively guide the DRL update due to complex post-disaster environments and multiple optimization objectives, thereby limiting the optimization performance. Notably, Large Language Models (LLMs), with their vast prior knowledge and advanced reasoning advantages, have demonstrated performance surpassing human levels in multiple domains. Therefore, we propose an LLMempowered Multi-UAV Trajectory Planning algorithm based on a modified Pointer network (LLMPtr-MTP), which integrates the reasoning advantages of LLMs to enhance DRL decisionmaking capabilities, achieving energy-efficient data collection. In LLMPtr-MTP, we develop an LLM reward shaping algorithm to generate a high-quality reward function that guides the LLMPtr-MTP agent in handling complex environments and improving the optimization performance of multiple objectives. Moreover, we design a modified Pointer network of the LLMPtr-MTP agent to achieve coordinated decision-making for multiple UAVs, significantly reducing the action space for the agent to solve the combinatorial problem and improving coordination efficiency of UAVs. Experimental results demonstrate that LLMPtr-MTP outperforms comparison algorithms in both AoI and timeliness metrics, with energy efficiency improved by 107%.