15 August 2023 in IEEE Transactions on Instrumentation and Measurement
Given the problem of exploration and utilization, the high interaction cost of deep reinforcement learning is sometimes unacceptable. Moreover, the model trained by deep reinforcement learning cannot adapt to the complex and changeable assembly environment quickly owing to the weak generalization ability. Thus, the knowledge transfer framework is established for different robot peg-in-hole assemblies in this article to reinforce the generalization ability of the assembly model and the utilization efficiency of data. In the framework, the source domain model is trained through the Proximal Policy Optimization algorithm. The model accurately predicts environmental information and dynamically adjusts the robot’s movements based on the prediction. Combined with the PI controller, the flexible peg-in-hole assembly can be quickly achieved by the robot. The policy fusion between the source and target domain MDP is realized by the Policy Fusion Transfer algorithm designed in this article. The effectiveness of the algorithm is verified and discussed based on the transfer experiments, involving different assembly objects and different assembly environments. The training results demonstrate that the environment is explored by the Policy Fusion Transfer algorithm in a safe range at the initial stage of training by reusing the assembly policy of the source domain and that the faster model convergence speed is achieved. Furthermore, the testing results suggest that the assembly success rate is improved by 26% and the assembly force is reduced by 30N. Moreover, the knowledge transfer between different peg-in-hole assemblies is realized successfully.
2 November 2023 in IEEE Robotics and Automation Letters
Deformable object manipulation has been considered a challenging task for robots for its complex dynamics and the infinite dimensional configuration space. Fabric unfolding manipulation takes on critical significance in the textile industry and household services. Accordingly, enabling robots to possess the above-mentioned skill has been confirmed as a crucial and challenging task. In this study, a general framework is developed for transferring human skills to robots in fabric unfolding manipulation. The developed framework comprises two key components (i.e., behavior cloning to learn human unfolding policy and learning from demonstration to transfer unfolding actions). A mixture density network is introduced, with the aim of addressing the multimodality in human policy. Moreover, task parameter weighting is considered during action generalization to adapt to a wide variety of unfolding scenarios. As revealed by the experimental results of this study, the framework can successfully unfold fabrics of different colors and sizes, and its performance can be comparable to human-level operation. Furthermore, the framework also can be applied to garment unfolding, and experiments suggest that it exhibits generalization.