Deep Reinforcement Learning (RL) is a promising approach for adaptive robot control, but its current application to robotics is currently hindered by high sample requirements. To alleviate this issue, we propose to exploit the symmetries present in robotic tasks. Intuitively, symmetries from observed trajectories define transformations that leave the space of feasible RL trajectories invariant and can be used to generate new feasible trajectories, which could be used for training. Based on this data augmentation idea, we formulate a general framework, called Invariant Transform Experience Replay that we present with two techniques: (i) Kaleidoscope Experience Replay exploits reflectional symmetries and (ii) Goal-augmented Experience Replay which takes advantage of lax goal definitions. In the Fetch tasks from OpenAI Gym, our experimental results show significant increases in learning rates and success rates. Particularly, we attain a 13, 3, and 5 times speedup in the pushing, sliding, and pick-and-place tasks respectively in the multi-goal setting. Performance gains are also observed in similar tasks with obstacles and we successfully deployed a trained policy on a real Baxter robot. Our work demonstrates that invariant transformations on RL trajectories are a promising methodology to speedup learning in deep RL.

Continuum robotic manipulators are increasingly adopted in minimal invasive surgery. However, their nonlinear behavior is challenging to model accurately, especially when subject to external interaction, potentially leading to poor control performance. In this letter, we investigate the feasibility of adopting a model-free multiagent reinforcement learning (RL), namely multiagent deep Q network (MADQN), to control a 2-degree of freedom (DoF) cable-driven continuum surgical manipulator. The control of the robot is formulated as a one DoF, one agent problem in the MADQN framework to improve the learning efficiency. Combined with a shielding scheme that enables dynamic variation of the action set boundary, MADQN leads to efficient and importantly safer control of the robot. Shielded MADQN enabled the robot to perform point and trajectory tracking with submillimeter root mean square errors under external loads, soft obstacles, and rigid collision, which are common interaction scenarios encountered by surgical manipulators. The controller was further proven to be effective in a miniature continuum robot with high structural nonlinearitiy, achieving trajectory tracking with submillimeter accuracy under external payload.

Graphical Abstract This paper proposes a novel robotic gripper used for assembly tasks that can adaptively grasp objects with different shapes. The proposed hand has a combined structure between two kinds of shape adaptive mechanisms where one is the granular jamming and the other is a multi-finger mechanism driven by a single wire. Due to the effect of the two shape adaptive mechanisms, the pose of a grasped object does not change during an assembly operation. The proposed hand has four fingers where two are the active ones and the other two are the passive ones. The pose of the grasped object can be uniquely determined since the passive fingers are used to orient an object placed on a table before the active fingers are closed to grasp it. Assembly experiments of some kinds of parts are shown to validate the effectiveness of our proposed gripper.

We propose a deep learning based method for fast and responsive human-robot handovers that generate robot motion according to human motion observations. Our method learns an offline human-robot interaction model through a Recurrent Neural Network with Long Short-Term Memory units (LSTM-RNN). The robot uses the learned network to respond appropriately to novel online human motions. Our method is tested both on pre-recorded data and real-world human-robot handover experiments. Our method achieves robot motion accuracies that outperform the baseline. In addition, our method demonstrates a strong ability to adapt to changes in velocity of human motions.

Robot introspection is expected to greatly aid longer-term autonomy of autonomous manipulation systems. By equipping robots with abilities that allow them to assess the quality of their sensory data, robots can detect and classify anomalies and recover appropriately from common anomalies. This work builds on our previous Sense-Plan-Act-Introspect-Recover (SPAIR) system. We introduce an improved anomaly detector that exploits latent states to monitor anomaly occurrence when robots collaborate with humans in shared workspaces, but also present a multiclass classifier that is activated with anomaly detection. Both implementations are derived from Bayesian non-parametric methods with strong modeling capabilities for learning and inference of multivariate time series with complex and uncertain behavior patterns. In particular, we explore the use of a hierarchical Dirichlet stochastic process prior to learning a Hidden Markov Model (HMM) with a switching vector auto-regressive observation model (sHDP-VAR-HMM). The detector uses a dynamic log-likelihood threshold that varies by latent state for anomaly detection and the anomaly classifier is implemented by calculating the cumulative log-likelihood of testing observation based on trained models. The purpose of our work is to equip the robot with anomaly detection and anomaly classification for the full set of skills associated with a given manipulation task. We consider a human–robot cooperation task to verify our work and measure the robustness and accuracy of each skill. Our improved detector succeeded in detecting 136 common anomalies and 368 nominal executions with a total accuracy of 91.0%. An overall anomaly classification accuracy of 97.1% is derived by performing the anomaly classification on an anomaly dataset that consists of 7 kinds of detected anomalies from a total of 136 anomalies samples.

Robotic failure is all too common in unstructured robot tasks. Despite well-designed controllers, robots often fail due to unexpected events. Robots under a sense-plan-act paradigm do not have an additional loop to check their actions. In this work, we present a principled methodology to bootstrap online robot introspection for contact tasks. In effect, we seek to enable the robot to recognize and expect its behavior, else detect anomalies. We postulated that noisy wrench data inherently contains patterns that can be effectively represented by a vocabulary. The vocabulary is obtained by segmenting and encoding data. And when wrench information represents a sequence of sub-tasks, the vocabulary represents a set of words or sentence and provides a unique identifier. The grammar, which can also include unexpected events, was classified both offline and online for simulated and real robot experiments. Multi-class Support Vector Machines (SVMs) were used offline, while online probabilistic SVMs were used to give temporal confidence to the introspection result. Our work's contribution is the presentation of a generalizable online semantic scheme that enables a robot to understand its high-level state whether nominal or anomalous. It is shown to work in offline and online scenarios for a particularly challenging contact task: snap assemblies. We perform the snap assembly in one-arm simulated and real one-arm experiments and a simulated two-arm experiment. The data set itself is also fully available online and provides a valuable resource by itself for this type of contact task. Our verification mechanism can be used by high-level planners or reasoning systems to enable intelligent failure recovery or determine the next most optimal manipulation skill to be used. Supplemental information, code, data, and other supporting documentation can be found at [1].