7 April 2021 in IEEE Internet of Things Journal
Due to the high maneuverability and flexibility, unmanned aerial vehicles (UAVs) have been considered as a promising paradigm to assist mobile edge computing (MEC) in many scenarios including disaster rescue and field operation. Most existing research focuses on the study of trajectory and computation-offloading scheduling for UAV-assisted MEC in stationary environments, and could face challenges in dynamic environments where the locations of UAVs and mobile devices (MDs) vary significantly. Some latest research attempts to develop scheduling policies for dynamic environments by means of reinforcement learning (RL). However, as these need to explore in high-dimensional state and action space, they may fail to cover in large-scale networks where multiple UAVs serve numerous MDs. To address this challenge, we leverage the idea of “divide-and-conquer” and propose HT3O, a scalable scheduling approach for large-scale UAV-assisted MEC. First, HT3O is built with neural networks via deep RL to obtain real-time scheduling policies for MEC in dynamic environments. More importantly, to make HT3O more scalable, we decompose the scheduling problem into two-layered subproblems and optimize them alternately via hierarchical RL. This not only substantially reduces the complexity of each subproblem, but also improves the convergence efficiency. Experimental results show that HT3O can achieve promising performance improvements over state-of-the-art approaches.
19 November 2021 in IEEE Transactions on Vehicular Technology
Mobile edge computing (MEC) has been considered as a promising paradigm to support the growing popularity of mobile devices (MDs) with similar capabilities as cloud computing. Most existing research focuses on MEC enabled by terrestrial base stations (BSs), which is unable to work in certain scenarios, e.g., disaster rescue and field operation. Some researchers have been making efforts on studying MEC assisted by unmanned-aerial-vehicles (UAVs) and developed lots of efficient scheduling algorithms. However, MEC assisted only by UAVs has limited capability and is unsuitable for heavy-computation applications. To address the issue, this paper proposes a novel UAV-and-BS hybrid enabled MEC system, where multiple UAVs and one BS are deployed to facilitate the provisioning of MEC services either directly from UAVs or indirectly from the BS. Considering maximizing the lifetime of all MDs, the energy-efficient scheduling problem is formulated as minimizing the energy consumption of all MDs by jointly optimizing UAV trajectories, task associations, computing-and-transmitting resource allocations. The optimization problem is further decomposed into three sub-problems and solved by the proposed hybrid heuristic and learning based scheduling algorithm to reduce the complexity. Experimental results show that the proposed algorithm can achieve promising performance improvements over baseline algorithms.
29 September 2021 in IEEE Transactions on Services Computing
Mobile edge computing (MEC) has been envisioned as a promising paradigm that could effectively enhance the computational capacity of wireless user devices (WUDs) and quality of experience of mobile applications. One of the most crucial issues of MEC is computation offloading, which decides how to offload WUDs’ tasks to edge severs for further intensive computation. Conventional mathematical programming-based offloading approaches could face troubles in dynamic MEC environments due to the time-varying channel conditions (caused primarily by WUD mobility). To address the problem, reinforcement learning (RL) based offloading approaches have been proposed, which develop offloading policies by mapping MEC states to offloading actions. However, these approaches could fail to converge in large-scale MEC due to the exponentially-growing state and action spaces. In this article, we propose a novel online computation offloading approach that could effectively reduce task latency and energy consumption in dynamic MEC with large-scale WUDs. First, a RL-based computation offloading and energy transmission algorithm is proposed to accelerate the learning process. Then, a joint optimization method is adopted to develop the allocating algorithm, which obtains near-optimal solutions for energy and computation resources allocation. Simulation results show that the proposed approach can converge efficiently and achieve significant performance improvements over baseline approaches.
28 September 2023 in IEEE Transactions on Mobile Computing
One of the key issues in mobile edge computing (MEC) is computation offloading, most policies of which are developed based on mathematical programming (MP). Due to the high computational complexity of iterative programming in MP-based policies, recent years have seen a popular trend to develop offloading policies based on deep reinforcement learning (DRL). However, on account of the poor generalization ability of DRL models in MEC environments with different network sizes and settings, it is difficult to directly apply DRL-based offloading policies in unseen MEC environments. Motivated by this, we propose a DRL-based environment-adaptive offloading framework (DEAT), including a size-adaptive scheme (SIED) and setting-adaptive component (SEAL). SIED leverages the idea of ‘time division multiplexing’ to adapt to varying MEC network sizes and order-unaware feature extraction to mitigate impacts of different size-changing orders. SEAL adopts system dynamics embedding and offloading policy embedding, which guide the finding of the closest pre-training MEC environment and offloading policy, respectively, to achieve fast setting-adaptation with only few exploring interactions in unseen MEC environments. Extensive experiments are conducted via both simulation and testbed to demonstrate the adaptation performance advantages of DEAT in unseen MEC environments compared to the state-of-the-art offloading approaches.
28 October 2024 in IEEE Transactions on Mobile Computing
Mobile edge computing (MEC), along with device-to-device (D2D) assisted MEC (D-MEC), are promising technologies that could improve the quality-of-experience for mobile devices (MDs) by offloading their tasks to edge servers or nearby idle MDs. There is a popular trend to develop distributed task offloading algorithms using multi-agent reinforcement learning (MARL), whose adoption of central critics during training makes the offloading still size-sensitive. Therefore, this paper proposes a Size-Insensitive Task Offloading (SITOff) algorithm for D-MEC based on fully-distributed offloading without maintaining any central venue. Specifically, taking advantage of the inherent graph-like structure of D-MEC, SITOff adopts graphs to represent MDs’ states and relationships and form each MD's local knowledge about D-MEC through graph computation. Furthermore, considering the limitation of local knowledge in performing whole performance-oriented offloading, each MD utilizes D2D-transmitting to exchange knowledge with its neighbors and form a comprehensive knowledge about D-MEC to enhance the coordination of distributed offloading. Additionally, regarding the different impacts of neighbors’ knowledge, each MD leverages attention mechanisms to selectively learn its neighbors’ knowledge during knowledge-exchange. Extensive experimental results show the superiority of SITOff over state-of-the-art MARL-based offloading algorithms in D-MEC with various MDs, and the easy collaboration of SITOff with curriculum-learning for large-scale D-MEC offloading.