Zheyuan Hu

Zheyuan Hu

Auto-generated profile, based on publicly available information

Zheyuan Hu

School of Computer Science and Engineering, Beihang University, Beijing, China

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
7 April 2021 in IEEE Internet of Things Journal
Verified
Journal Article
Closed access
TRTao Ren
Tao Ren
Jianwei NiuJianwei Niu
Jianwei Niu
BDBin Dai
Bin Dai
XLXuefeng Liu
Xuefeng Liu
ZHZheyuan Hu
Zheyuan Hu
Mohsen GuizaniMohsen Guizani
Mohsen Guizani
Due to the high maneuverability and flexibility, unmanned aerial vehicles (UAVs) have been considered as a promising paradigm to assist mobile edge computing (MEC) in many scenarios including disaster rescue and field operation. Most existing research focuses on the study of trajectory and computation-offloading scheduling for UAV-assisted MEC in stationary environments, and could face challenges in dynamic environments where the locations of UAVs and mobile devices (MDs) vary significantly. Some latest research attempts to develop scheduling policies for dynamic environments by means of reinforcement learning (RL). However, as these need to explore in high-dimensional state and action space, they may fail to cover in large-scale networks where multiple UAVs serve numerous MDs. To address this challenge, we leverage the idea of “divide-and-conquer” and propose HT3O, a scalable scheduling approach for large-scale UAV-assisted MEC. First, HT3O is built with neural networks via deep RL to obtain real-time scheduling policies for MEC in dynamic environments. More importantly, to make HT3O more scalable, we decompose the scheduling problem into two-layered subproblems and optimize them alternately via hierarchical RL. This not only substantially reduces the complexity of each subproblem, but also improves the convergence efficiency. Experimental results show that HT3O can achieve promising performance improvements over state-of-the-art approaches.

19 November 2021 in IEEE Transactions on Vehicular Technology
Verified
Journal Article
Closed access
BDBin Dai
Bin Dai
Jianwei NiuJianwei Niu
Jianwei Niu
TRTao Ren
Tao Ren
ZHZheyuan Hu
Zheyuan Hu
MAMohammed Atiquzzaman
Mohammed Atiquzzaman
Mobile edge computing (MEC) has been considered as a promising paradigm to support the growing popularity of mobile devices (MDs) with similar capabilities as cloud computing. Most existing research focuses on MEC enabled by terrestrial base stations (BSs), which is unable to work in certain scenarios, e.g., disaster rescue and field operation. Some researchers have been making efforts on studying MEC assisted by unmanned-aerial-vehicles (UAVs) and developed lots of efficient scheduling algorithms. However, MEC assisted only by UAVs has limited capability and is unsuitable for heavy-computation applications. To address the issue, this paper proposes a novel UAV-and-BS hybrid enabled MEC system, where multiple UAVs and one BS are deployed to facilitate the provisioning of MEC services either directly from UAVs or indirectly from the BS. Considering maximizing the lifetime of all MDs, the energy-efficient scheduling problem is formulated as minimizing the energy consumption of all MDs by jointly optimizing UAV trajectories, task associations, computing-and-transmitting resource allocations. The optimization problem is further decomposed into three sub-problems and solved by the proposed hybrid heuristic and learning based scheduling algorithm to reduce the complexity. Experimental results show that the proposed algorithm can achieve promising performance improvements over baseline algorithms.

29 September 2021 in IEEE Transactions on Services Computing
Verified
Journal Article
Closed access
ZHZheyuan Hu
Zheyuan Hu
Jianwei NiuJianwei Niu
Jianwei Niu
TRTao Ren
Tao Ren
BDBin Dai
Bin Dai
QLQingfeng Li
Qingfeng Li
SSajal K. Das
Sajal K. Das
Mobile edge computing (MEC) has been envisioned as a promising paradigm that could effectively enhance the computational capacity of wireless user devices (WUDs) and quality of experience of mobile applications. One of the most crucial issues of MEC is computation offloading, which decides how to offload WUDs’ tasks to edge severs for further intensive computation. Conventional mathematical programming-based offloading approaches could face troubles in dynamic MEC environments due to the time-varying channel conditions (caused primarily by WUD mobility). To address the problem, reinforcement learning (RL) based offloading approaches have been proposed, which develop offloading policies by mapping MEC states to offloading actions. However, these approaches could fail to converge in large-scale MEC due to the exponentially-growing state and action spaces. In this article, we propose a novel online computation offloading approach that could effectively reduce task latency and energy consumption in dynamic MEC with large-scale WUDs. First, a RL-based computation offloading and energy transmission algorithm is proposed to accelerate the learning process. Then, a joint optimization method is adopted to develop the allocating algorithm, which obtains near-optimal solutions for energy and computation resources allocation. Simulation results show that the proposed approach can converge efficiently and achieve significant performance improvements over baseline approaches.

28 September 2023 in IEEE Transactions on Mobile Computing
Verified
Journal Article
Closed access
ZHZheyuan Hu
Zheyuan Hu
Jianwei NiuJianwei Niu
Jianwei Niu
TRTao Ren
Tao Ren
Mohsen GuizaniMohsen Guizani
Mohsen Guizani
One of the key issues in mobile edge computing (MEC) is computation offloading, most policies of which are developed based on mathematical programming (MP). Due to the high computational complexity of iterative programming in MP-based policies, recent years have seen a popular trend to develop offloading policies based on deep reinforcement learning (DRL). However, on account of the poor generalization ability of DRL models in MEC environments with different network sizes and settings, it is difficult to directly apply DRL-based offloading policies in unseen MEC environments. Motivated by this, we propose a DRL-based environment-adaptive offloading framework (DEAT), including a size-adaptive scheme (SIED) and setting-adaptive component (SEAL). SIED leverages the idea of ‘time division multiplexing’ to adapt to varying MEC network sizes and order-unaware feature extraction to mitigate impacts of different size-changing orders. SEAL adopts system dynamics embedding and offloading policy embedding, which guide the finding of the closest pre-training MEC environment and offloading policy, respectively, to achieve fast setting-adaptation with only few exploring interactions in unseen MEC environments. Extensive experiments are conducted via both simulation and testbed to demonstrate the adaptation performance advantages of DEAT in unseen MEC environments compared to the state-of-the-art offloading approaches.

29 May 2023
Verified
Conference Paper
Open access
KXKelvin Xu
Kelvin Xu
ZHZheyuan Hu
Zheyuan Hu
RDRia Doshi
Ria Doshi
ARAaron Rovinsky
Aaron Rovinsky
VKVikas Kumar
Vikas Kumar
SLSergey Levine
Sergey Levine
Complex and contact-rich robotic manipulation tasks, particularly those that involve multi-fingered hands and underactuated object manipulation, present a significant challenge to any control method. Methods based on reinforcement learning offer an appealing choice for such settings, as they can enable robots to learn to delicately balance contact forces and dexterously reposition objects without strong modeling assumptions. However, running reinforcement learning on real-world dexterous manipulation systems often requires significant manual engineering. This negates the benefits of autonomous data collection and ease of use that reinforcement learning should in principle provide. In this paper, we describe a system for vision-based dexterous manipulation that provides a “programming-free” approach for users to define new tasks and enable robots with complex multi-fingered hands to learn to perform them through interaction. The core principle under-lying our system is that, in a vision-based setting, users should be able to provide high-level intermediate supervision that circumvents challenges in teleoperation or kinesthetic teaching which allows a robot to not only learn a task efficiently but also to autonomously practice. Our system includes a framework for users to define a final task and intermediate sub-tasks with image examples, a reinforcement learning procedure that learns the task autonomously without interventions, and experimental results with a four-finger robotic hand learning multi-stage object manipulation tasks directly in the real world, without simulation, manual modeling, or reward engineering.

28 October 2024 in IEEE Transactions on Mobile Computing
Verified
Journal Article
Closed access
ZHZheyuan Hu
Zheyuan Hu
Jianwei NiuJianwei Niu
Jianwei Niu
TRTao Ren
Tao Ren
XLXuefeng Liu
Xuefeng Liu
Mohsen GuizaniMohsen Guizani
Mohsen Guizani
Mobile edge computing (MEC), along with device-to-device (D2D) assisted MEC (D-MEC), are promising technologies that could improve the quality-of-experience for mobile devices (MDs) by offloading their tasks to edge servers or nearby idle MDs. There is a popular trend to develop distributed task offloading algorithms using multi-agent reinforcement learning (MARL), whose adoption of central critics during training makes the offloading still size-sensitive. Therefore, this paper proposes a Size-Insensitive Task Offloading (SITOff) algorithm for D-MEC based on fully-distributed offloading without maintaining any central venue. Specifically, taking advantage of the inherent graph-like structure of D-MEC, SITOff adopts graphs to represent MDs’ states and relationships and form each MD's local knowledge about D-MEC through graph computation. Furthermore, considering the limitation of local knowledge in performing whole performance-oriented offloading, each MD utilizes D2D-transmitting to exchange knowledge with its neighbors and form a comprehensive knowledge about D-MEC to enhance the coordination of distributed offloading. Additionally, regarding the different impacts of neighbors’ knowledge, each MD leverages attention mechanisms to selectively learn its neighbors’ knowledge during knowledge-exchange. Extensive experimental results show the superiority of SITOff over state-of-the-art MARL-based offloading algorithms in D-MEC with various MDs, and the easy collaboration of SITOff with curriculum-learning for large-scale D-MEC offloading.