Industrial robot manipulators are playing a significant role in modern manufacturing industries. Though peg-in-hole assembly is a common industrial task that has been extensively researched, safely solving complex, high-precision assembly in an unstructured environment remains an open problem. Reinforcement-learning (RL) methods have proven to be successful in autonomously solving manipulation tasks. However, RL is still not widely adopted in real robotic systems because working with real hardware entails additional challenges, especially when using position-controlled manipulators. The main contribution of this work is a learning-based method to solve peg-in-hole tasks with hole-position uncertainty. We propose the use of an off-policy, model-free reinforcement-learning method, and we bootstraped the training speed by using several transfer-learning techniques (sim2real) and domain randomization. Our proposed learning framework for position-controlled robots was extensively evaluated in contact-rich insertion tasks in a variety of environments.

Reinforcement Learning (RL) methods have been proven successful in solving manipulation tasks autonomously. However, RL is still not widely adopted on real robotic systems because working with real hardware entails additional challenges, especially when using rigid position-controlled manipulators. These challenges include the need for a robust controller to avoid undesired behavior, that risk damaging the robot and its environment, and constant supervision from a human operator. The main contributions of this work are, first, we propose a fail-safe mechanism for safely training an RL agent on manipulation tasks using a rigid robot. Second, to enable a position-controlled manipulator to perform contact-rich manipulation tasks, we implemented two different force control schemes based on standard force feedback controllers; one is a modified parallel position-force control, and the other one is an impedance control. Third, we empirically study both control schemes when used as the action representation of an RL agent. We evaluate the trade-off between control complexity and performance by comparing several versions of the control schemes, each with a different number of force control parameters. The proposed methods are validated both on simulation and a real robot, a UR3 e-series robotic arm when executing contact-rich manipulation tasks.

Complex assembly tasks involve nonlinear and low-clearance insertion trajectories with varying contact forces at different stages. For a robot to solve these tasks, it requires a precise and adaptive controller which conventional force control methods cannot provide. Imitation learning is a promising method for learning controllers that can solve the nonlinear trajectories from human demonstrations without needing to explicitly program them into the robot. However, the force profiles obtain from human demonstration via tele-operation tend to be sub-optimal for complex assembly tasks, thus it is undesirable to imitate such force profiles. Reinforcement learning learns adaptive control policies through interactions with the environment but struggles with low sample efficiency and equipment tear and wear in the physical world. To address these problems, we present a combined learning-based framework to solve complex robotic assembly tasks from human demonstrations via hybrid trajectory learning and force learning. The main contribution of this work is the development of a framework that combines imitation learning, to learn the nominal motion trajectory, with a reinforcement learning-based force control scheme to learn an optimal force control policy, which can satisfy the nominal trajectory while adapting to the force requirement of the assembly task. To further improve the imitation learning part, we develop a hierarchical architecture, following the idea of goal-conditioned imitation learning, to generate the trajectory learning policy on the skill level offline. Through experimental validations, we corroborate that the proposed learning-based framework can generate high-quality trajectories and find suitable force control policies which adapt to the tasks' force requirements more efficiently.

Factory automation robot systems often depend on specially-made jigs that precisely position each part, which increases the system's cost and limits flexibility. We propose a method to determine the 3D pose of an object with high precision and confidence, using only parallel robotic grippers and no parts-specific jigs. Our method automatically generates a sequence of actions that ensures that the real-world position of the physical object matches the system's assumed pose to sub-mm precision. Furthermore, we propose the use of “extrinsic” actions, which use gravity, the environment and the gripper geometry to significantly reduce or even eliminate the uncertainty about an object's pose. We show in simulated and real-robot experiments that our method outperforms our previous work, at success rates over 95%.

Towards the automation of assembly tasks using industrial robot manipulators, improving the robotic grasping is essential. In this paper, we employed a reinforcement learning method based on the policy search algorithm, call Guided Policy Search, to learn policies for the grasping problem. The goal was to evaluate if policies trained solely using sets of primitive shaped objects, can still achieve the task of grasping objects of more complex shapes. The results show that even using simple shaped objects; the method can learn policies that generalize to more complex shapes. Additionally, a robustness test was conducted to show that the visual component of the policy helps to guide the system when there is an error in the estimation of the target object pose.