Beijing Engineering and Technology Research Center for Convergence Networks and Ubiquitous Services, University of Science and Technology Beijing, Beijing, China
Auto-generated profile, based on publicly available information
Xulong Li
Beijing Engineering and Technology Research Center for Convergence Networks and Ubiquitous Services, University of Science and Technology Beijing, Beijing, China
In this paper, multi-UAVs serve as mobile aerial ISAC platforms to sense and communicate with on-ground target users. To optimize the communication and sensing performance, we formulate a joint user association, UAV trajectory planning and power allocation problem to maximize the minimum weighted spectral efficiency among UAVs. This paper exploits the centralized and the decentralized deep reinforcement learning (DRL) solutions to solve the sequential decision-making problem. On one hand, we first introduce the centralized soft actor-critic (SAC) algorithm. Then, we explore the equivalent transformation of the optimization objective based on symmetric group, propose the random and the adaptive data augmentation schemes to design the replay memory buffer of SAC, and accordingly propose SAC algorithms assisted by data augmentation to tackle the transformed problem. On the other hand, the multi-agent soft actor-critic (MASAC), a decentralized solution, is also introduced to solve this sequential decision-making problem. The experiment results reveal the effectiveness of the centralized and the decentralized solutions in considered scenarios. Specifically, the SAC assisted by the adaptive scheme significantly outperforms other centralized solutions in the training speed and the weighted spectral efficiency. Meanwhile, the decentralized MASAC algorithm behaves best in the early training speed.
4 December 2023 in IEEE Transactions on Vehicular Technology
Compared with centralized cloud computing, mobile edge computing (MEC) enables Internet of Things (IoT) devices to offload computation-intensive tasks to the edge of the network closer to them for processing, which can effectively save energy consumption of IoT devices and alleviate network congestion and high latency problems. However, the traditional terrestrial MEC system cannot adapt to scenarios such as rapid network recovery after disasters and emergency rescue due to its poor flexibility and high deployment cost, and assembling edge servers to unmanned aerial vehicles (UAVs) to assist in rapidly building mobile edge networks is a feasible solution. Therefore, this paper considers a multi-UAV-enabled MEC network with the optimization objectives of maximizing the processing success rate of computational tasks and fairness of system, while minimizing the processing delay of computational tasks. We investigate the computation offloading problem and the trajectory planning problem from the perspectives of IoT devices and UAVs, respectively. Then model them as Markov decision processes (MDPs) and propose a joint optimization scheme based on expert knowledge-assisted multi-agent reinforcement learning algorithms. Simulation results show that the proposed algorithm has significant advantages over baseline algorithms in terms of processing success rate and delay of computational tasks as well as fairness of the system.
6 August 2024 in IEEE Transactions on Mobile Computing
Mobile edge computing (MEC) has attracted widespread attention due to its ability to effectively alleviate the cloud computing load and significantly reduce latency. However, the potential eavesdroppers challenge the security of the MEC systems and the rapid development of artificial intelligence (AI) has made this security situation more severe. In most existing studies, the eavesdroppers are non-intelligent and it is assumed that they are fixed or move in a simple manner. Obviously, there is a gap from such an assumption to the real conditions that the eavesdropping unmanned aerial vehicles (UAVs) may adjust their flight paths intelligently. To better reflect real-world scenarios, we consider a multi-UAV-assisted MEC system in the presence of intelligent eavesdroppers and propose an adversarial multi-agent reinforcement learning (MARL)-based scheme for secure computational offloading and resource allocation. With this scheme, we aim to solve the zero-sum game between the legitimate UAVs and the eavesdropping UAVs, in which the two types of UAVs take turns acting as the agents of MARL to alternately optimize their respective opposing objectives. The simulation experimental results indicate that the proposed scheme significantly outperforms the existing baseline methods in dealing with the intelligent eavesdropping UAVs, and ensures high energy efficiency of Internet of Things (IoT) devices even in the worst-case scenario when dealing with potential eavesdropping threats.
Mobile edge computing (MEC) as a promising technology enables it to satisfy ever-increasing demands for low-latency and ultra-reliable services. However, due to the limitations of computing capability and the dynamic network environment, it is challenging to process massive data with low latency. In this paper, we consider a dynamic MEC network with a high-performance edge server, multiple time-varying channels, and multiple mobile devices. We aim to find a policy that can maximize the processing success rate of computational tasks and the fairness index of the system while minimizing the process delays. To this end, we propose a heuristic-assisted multiagent reinforcement learning (RL) based framework to realize the joint optimization of computation offloading and resource allocation. On one hand, heuristic search is introduced in this framework to find a better resource allocation policy in edge servers and further assist the multiagent RL algorithm to determine offloading policy in mobile devices. On the other hand, a novel parametrized multiagent RL algorithm based on soft actor-critic (SAC) is also proposed to broaden the effectiveness and availability of the proposed framework. Simulation results of the average cumulative reward, success rate, processing delay, and fairness index fully verify the superiority of the proposed framework and algorithm for supporting this problem.
Joint task scheduling and resource allocation for unmanned aerial vehicle (UAV)-assisted integrated sensing and communication (ISAC) in emergency rescue activities has become an essential and challenging problem. However, the existing works have only considered such a problem for standalone UAV networks without considering the cooperation between UAVs and ground base stations (BSs), nor have they considered the uncertainty in terms of the availability of BSs due to damage/reconstruction in disaster events. In this paper, we consider a novel post-disaster UAV-assisted ISAC system where the UAVs are used to supplement the networking capacity of out-of-service ground BSs while using their radio signals for sensing. We apply transfer learning with deep reinforcement learning (DRL) to learn task scheduling and resource allocation strategies that can rapidly adapt to uncertainty in the environment. Experimental results show that the proposed algorithm outperforms the state-of-the-art in both communication and sensing performance and convergence speed. Moreover, the transfer learning-based DRL shows faster convergence and better robustness when the availability of BSs suddenly changes.
1 November 2025 in IEEE Transactions on Intelligent Transportation Systems