28 January 2025 in IEEE Transactions on Automation Science and Engineering
The whole-body control of mobile manipulators for the 6-DOF trajectory following task is the basis of many continuous tasks. However, traditional control strategies rely on accurate models and expert knowledge for solving the trajectory following task. Deep reinforcement learning (DRL) provides a promising model-free solution, but it is sample-inefficient. To this end, we propose Trajectory Following Hindsight Experience Replay (TF-HER), a sample-efficient DRL algorithm for the whole-body coupled trajectory following task with dense rewards. TF-HER builds a multi-trajectory state space, and relabels the low-reward data to generate informative high-reward experiences. Also, the distributional shift caused by the relabeling is corrected by estimating the density ratio of relabeled experiences. Extensive demonstrations on both nonholonomic and holonomic bases in simulation validate that our algorithm can accelerate the model convergence and significantly improve the sample efficiency. Furthermore, we present real-world experiments to demonstrate the effectiveness of our approach. The code is available: https://github.com/IRMV-Manipulation-Group/TF-HER.
8 April 2025 in IEEE/ASME Transactions on Mechatronics
This study presents a method for mobile manipulators to transport heavy and large objects by utilizing environmental support, thereby extending the capability of mobile manipulators to manipulate objects that exceed their payload capacity. This is a challenging task, as the agent must achieve whole-body coordination while satisfying multiple constraints, including payload limits, base collision avoidance, and joint limits for torque, velocity, and position. We propose a hierarchical deep reinforcement learning with model guidance to address the whole-body contact-rich mobile manipulation task. The system includes two main components: 1) high-level trajectory correction of end-effector based on dynamic movement primitives in operational space and 2) low-level redundancy resolution guided by a model-based safety filter, which filters out unsafe action and generates safe action by quadratic programming method. The proposed hierarchical structure can ensure the convergence of end-effector trajectory while solving the nonstationary transition problem during parallel training. Experiments in simulation and real-world show the system's effectiveness in achieving faster convergence, greater stability, and higher performance in mobile manipulation tasks compared to conventional methods.
29 April 2025 in IEEE/ASME Transactions on Mechatronics
Visual servoing is established as a theoretically reliable scheme for achieving high-precision robotic control. However, various image and physical constraints inevitably limit the application of visual servoing methods. This manuscript proposes a controller based on quadratic programming with proven local stability, explicitly handling some dominant constraints. Other difficult-to-model constraints, such as collisions, are addressed using an efficient sampling-based bidirectional planning framework designed for the visual servoing problem. Considering constraints, the proposed method explores the permissible image space by progressively expanding two search trees and ultimately generates feasible image trajectories. Subsequently, the quadratic programming-based local controller is extended to the tracking problem, aiming to track the planned trajectory and guide the manipulator to reach target features. The effectiveness of the proposed methods is demonstrated through various simulated and real experiments with a seven-degree-of-freedom manipulator. Comprehensive comparative experiments against several related methods highlight the performance of the proposed method.