Wei Zhu

Wei Zhu

Auto-generated profile, based on publicly available information

Wei Zhu

Georgia Institute of Technology

Research Focus & Metrics

Academic Career Length

9Years (2018-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
20 July 2022 in IEEE Transactions on Industrial Electronics
Verified
Journal Article
Open access
WZWei Zhu
Wei Zhu
MHMitsuhiro Hayashibe
Mitsuhiro Hayashibe
We present a hierarchical deep reinforcement learning (DRL) framework with prominent sampling efficiency and sim-to-real transfer ability for fast and safe navigation: the low-level DRL policy enables the robot to move toward the target position and keep a safe distance to obstacles simultaneously; the high-level DRL policy is supplemented to further enhance the navigation safety. We select a waypoint located on the path from the robot to the ultimate goal as the subgoal to reduce the state space and avoid sparse reward. Moreover, the path is generated based on either a local or a global map, which can significantly improve the sampling efficiency, safety, and generalization ability of the proposed DRL framework. Additionally, a target-directed representation for the action space can be derived based on the subgoal to improve the motion efficiency and reduce the action space. In order to demonstrate the eminent sampling efficiency, motion performance, obstacle avoidance, and generalization ability of the proposed framework, we implement sufficient comparisons with the nonlearning navigation methods and DRL-based baselines, with videos, data, code, and other supplemental material shown on our website.

29 September 2021 in IEEE Transactions on Neural Networks and Learning Systems
Verified
Journal Article
Closed access
WZWei Zhu
Wei Zhu
XGXian Guo
Xian Guo
DODai Owaki
Dai Owaki
KKKyo Kutsuzawa
Kyo Kutsuzawa
MHMitsuhiro Hayashibe
Mitsuhiro Hayashibe
The state-of-the-art reinforcement learning (RL) techniques have made innumerable advancements in robot control, especially in combination with deep neural networks (DNNs), known as deep reinforcement learning (DRL). In this article, instead of reviewing the theoretical studies on RL, which were almost fully completed several decades ago, we summarize some state-of-the-art techniques added to commonly used RL frameworks for robot control. We mainly review bioinspired robots (BIRs) because they can learn to locomote or produce natural behaviors similar to animals and humans. With the ultimate goal of practical applications in real world, we further narrow our review scope to techniques that could aid in sim-to-real transfer. We categorized these techniques into four groups: 1) use of accurate simulators; 2) use of kinematic and dynamic models; 3) use of hierarchical and distributed controllers; and 4) use of demonstrations. The purposes of these four groups of techniques are to supply general and accurate environments for RL training, improve sampling efficiency, divide and conquer complex motion tasks and redundant robot structures, and acquire natural skills. We found that, by synthetically using these techniques, it is possible to deploy RL on physical BIRs in actuality.

24 December 2019 in IEEE Transactions on Neural Networks and Learning Systems
Verified
Journal Article
Closed access
WZWei Zhu
Wei Zhu
XGXian Guo
Xian Guo
YFYongchun Fang
Yongchun Fang
XZXueyou Zhang
Xueyou Zhang
Reinforcement learning (RL) combined with deep neural networks has led to a number of great achievements for robot control in virtual computer environments, where sufficient data can be obtained without any difficulty to train various models. However, thus far, only few and relatively simple tasks have been accomplished for practical robots, which is mainly caused by the following two reasons. First, training with real robots, especially with dynamic systems, is too complicated to be fully and accurately represented in simulations. Second, it is very costly to obtain training data from real systems. To address these two problems effectively, in this article, a path-integral-based RL algorithm is proposed for the task of path following of an autoassembly mobile robot, wherein three kernel techniques are introduced. First, a generalized path-integral-control approach is proposed to obtain the numerical solution of a stochastic dynamical system, wherein the calculation of the gradient and kinematics inverse is avoided to ensure fast and reliable training convergence. Second, a novel parameterization method using Lyapunov techniques is introduced into the RL algorithm to ensure good performance of the system when directly transferring simulation results into practical systems. Third, the optimal parameters for all discrete initial states are first learned offline and then tuned online to improve the generalization and real-time performance. In addition to the optimization control for the mobile robot, the proposed method also possesses general applicability for a class of nonlinear systems such as crane systems. Simulation and experimental results are included and analyzed to illustrate the superior performance of the proposed algorithm.

3 December 2018 in IEEE Transactions on Industrial Electronics
Verified
Journal Article
Closed access
XGXian Guo
Xian Guo
WZWei Zhu
Wei Zhu
YFYongchun Fang
Yongchun Fang
In this paper, a novel guided motion planning method is proposed for snake-like robots, which decomposes the configuration space into the fiber space and the gait space, with the motion planned in the low-dimension fiber space, so as to avoid the dimensionality curse problem. More specifically, firstly, based on geometry mechanics, the kinematic connection is derived, which successfully maps the motion in the fiber space intothatinthegaitspace.Accordingtothekinematicconnection, a novel bidirectional motion control method for snake-like robots is proposed to improve the fiexibility which can produce not only the forward locomotion but also the backward locomotion additionally. The convergence of the bidirectional motion control method is proved based on averaging theory. Then, the motion planning problem is solved in the low-dimensional fiber space,for which the HJB equation is used to generate the optimal control inthe fiber space to avoid the well known curse of dimensionality. Finally, combing the bidirectional motion control and the motion planned in the fiber space, a guided motion planning method in the configuration space is proposed. Numerical simulations and experiments based on the real snake-like robot platform are performed, whose results demonstrate that the proposed algorithm is valid and robust.

23 July 2025 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
FWFeiyang Wu
Feiyang Wu
XNXavier Nal
Xavier Nal
JJJaehwi Jang
Jaehwi Jang
WZWei Zhu
Wei Zhu
ZGZhaoyuan Gu
Zhaoyuan Gu
YZYe Zhao
Ye Zhao
Humanoid robots promise transformative capabilities for industrial and service applications. While recent advances in Reinforcement Learning (RL) yield impressive results in locomotion, manipulation, and navigation, the proposed methods typically require enormous simulation samples to account for real-world variability. This work proposes a novel one-stage training framework—Learn to Teach (L2T)—which unifies teacher and student policy learning. Our approach recycles simulator samples and synchronizes the learning trajectories through shared dynamics, significantly reducing sample complexities and training time while achieving state-of-the-art performance. Furthermore, we validate the RL variant (L2T-RL) through extensive simulations and hardware tests on the Digit robot, demonstrating zero-shot sim-to-real transfer and robust performance over 12+ diverse terrains without depth estimation modules.

8 May 2023 in IEEE Robotics and Automation Letters
Verified
Journal Article
Open access
WZWei Zhu
Wei Zhu
MHMitsuhiro Hayashibe
Mitsuhiro Hayashibe
Navigation among pedestrians is a crucial capability of service robots; however, it is a challenge to manage time-varying environments stably. Recent deep reinforcement learning (DRL)-based approaches to crowd navigation have yielded numerous promising applications. However, they rely heavily on initial imitation learning and colossal positive datasets. Moreover, the difficulties in accurately localizing robots, detecting and tracking humans, representing and generalizing reciprocal human relationships restrict their deployment in real-world problems. We propose a Dreamer-based motion planner for collision-free navigation in diverse pedestrian scenarios. Our RL framework can completely learn from zero experience via a model-based DRL. The robot and humans are first projected onto a map, which is subsequently decoded into low-dimensional latent state. A predictive dynamic model in the latent space is jointly created to efficiently optimize the navigation policy. Additionally, we leverage the techniques of system identification, domain randomization, clustering and LiDAR SLAM for practical deployment. Simulation ablations and real implementations demonstrate that our motion planner outperforms state-of-the-art methods, and that the navigation system can be physically implemented in the real world.