14 February 2024 in IEEE Robotics and Automation Letters
Adversarial Inverse Reinforcement Learning (AIRL) has gained popularity as an alternative to supervised imitation learning, addressing the distributional bias issue of the latter. However, it still faces significant challenges in long-horizon tasks due to the lack of effective exploration. In our letter, we demonstrate that standard AIRL strategies end exploration prematurely during online reinforcement learning and fail to learn the entire task due to their inability to fully conform to the expert distribution, which is particularly detrimental to real-world robots. To address these challenges, we introduce the SC-AIRL approach. It decomposes long-horizon tasks into logical subtasks which reduces the agent's need for rich exploration. SC-AIRL utilizes expert demonstrations for performing multiple subtasks and shares a single critic and identical reward function across different subtask trainings. Additionally, we incorporate a human intervention mechanism during the subtask learning process to keep exploration from ending prematurely. Our experiments in challenging robot manipulation tasks demonstrate that SC-AIRL outperforms our baselines significantly. Furthermore, we conduct an exploratory experiment and an empirical analysis, emphasizing the potential of the model to manage complex tasks and the advantages of SC-AIRL over the baseline, respectively.
30 April 2025 in IEEE Transactions on Industrial Electronics
Contact force in contact-rich environments is an essential modality for robots to perform general-purpose manipulation tasks, as it provides information to compensate for the deficiencies of visual and proprioceptive data in collision perception, high-precision grasping, and efficient manipulation. In this article, we propose an admittance visuomotor policy framework for continuous, general-purpose, and contact-rich manipulations. During demonstrations, we designed a low-cost, user-friendly teleoperation system with contact interaction, aiming to gather compliant robot demonstrations and accelerate the data collection process. During training and inference, we propose a diffusion-based model to plan action trajectories and desired contact forces from multimodal observation that includes contact force, vision and proprioception. We utilize an admittance controller for compliance action execution. A comparative evaluation with two state-of-the-art methods was conducted on five challenging tasks, each focusing on different action primitives, to demonstrate our framework’s generalization capabilities. Results show our framework achieves the highest success rate and exhibits smoother and more efficient contact compared to other methods, the contact force required to complete each tasks was reduced on average by 48.8%, and the success rate was increased on average by 15.3%. Videos are available at https://ryanjiao.github.io/AdmitDiffPolicy/.