16 July 2022 in Digital Communications and Networks
16 July 2022 in Computer Communications
30 November 2022 in Journal of Computer Science and Technology
14 November 2024 in IEEE Internet of Things Journal
Aerial-terrestrial network (ATN) framework is currently the dominant method of the internet of unmanned agents (IUA) for integrating both aerial vehicles and terrestrial sensors. In ATN scenarios, existing work shows that route planning utilizing deep reinforcement learning (DRL) is important as it can conserve energy for aerial vehicles or diminish network latency. However, utilizing DRL approaches in a multi-agent ATN environment may lead to inefficiencies when sharing original interaction data. In addition, direct data exchange between agents can raise significant privacy concerns. To address these challenges, this work develops a novel federated reinforcement learning (FRL) approach called FRIGE, which takes advantage of ensemble learning for DRL-based methods. Specifically, this paper develops a prioritized gradient ensemble technique for local DRL agents, which constructs twin networks to obtain prioritized gradients without incurring additional interactive costs. After aggregating local models on the server, the latest iteration of the global model is used to update the networks of both agents and their twin networks for subsequent rounds of training. Extensive experiments are conducted on two typical ATN route planning tasks to validate FRIGE’s advancement and generalization capabilities. The numerical results demonstrate that FRIGE not only enhances data sharing efficiency among all agents but also maintains privacy while delivering superior solutions for ATN route planning tasks.
2 February 2024 in IEEE Transactions on Network Science and Engineering
The deployment of Unmanned Aerial Vehicles (UAVs) alongside wireless communication networks as Aerial-Terrestrial Network (ATN) in vast and remote wilderness regions shows a growing trend in the current era. Simultaneously, the adoption of 6G technologies is imperative for the effective performance of ATN tasks, such as communication relaying, data transmission, and remote device management. However, the deployment of multiple UAVs in ATN faces challenges in energy management and coordination, particularly in rigorous environments where the power supply is restricted. To overcome these challenges, this work develops Umix, an innovative approach grounded in Deep Reinforcement Learning (DRL) method Qmix, engineered to refine the trajectories and operational dynamics of multiple UAVs. Umix is deployed to provide energy-efficient solutions for multiple UAVs, taking into account the limitations of battery capacity. It features an encoder-decoder architecture that adeptly captures the historical trajectory data of UAVs. Furthermore, it integrates a weight assignment network that is adept at evaluating and distributing tasks across the UAVs with high efficiency. Extensive experiments are conducted to demonstrate the advancement of the developed method Umix. The numerical results show that Umix can conserve more energy and time, which outperforms benchmarks under the settings of different amounts of UAVs and sensors.