Tactile sensing is a key sensor modality for robots interacting with their surroundings. These sensors provide a rich and diverse set of data signals that contain detailed information collected from contacts between the robot and its environment. The data are however not limited to individual contacts and can be used to extract a wide range of information about the objects in the environment as well as the actions of the robot during the interactions. In this article, we provide an overview of tactile information and its applications in robotics. We present a hierarchy consisting of raw, contact, object, and action levels to structure the tactile information, with higher-level information often building upon lower-level information. We discuss different types of information that can be extracted at each level of the hierarchy. The article also includes an overview of different types of robot applications and the types of tactile information that they employ. Finally we end the article with a discussion for future tactile applications which are still beyond the current capabilities of robots.
Shared control, which permits a human operator and an autonomous controller to share the control of a telerobotic system, can reduce the operator's workload and/or improve performances during the execution of tasks. Due to the great benefits of combining the human intelligence with the higher power/precision abilities of robots, the shared control architecture occupies a wide spectrum among telerobotic systems. Although various shared control strategies have been proposed, a systematic overview to tease out the relation among different strategies is still absent. This survey, therefore, aims to provide a big picture for existing shared control strategies. To achieve this, we propose a categorization method and classify the shared control strategies into 3 categories: Semi-Autonomous control (SAC), State-Guidance Shared Control (SGSC), and State-Fusion Shared Control (SFSC), according to the different sharing ways between human operators and autonomous controllers. The typical scenarios in using each category are listed and the advantages/disadvantages and open issues of each category are discussed. Then, based on the overview of the existing strategies, new trends in shared control strategies, including the “autonomy from learning” and the “autonomy-levels adaptation,” are summarized and discussed.
31 December 2021 in IEEE Transactions on Automation Science and Engineering
To accurately pour drinks into various containers is an essential skill for service robots. However, drink pouring is a dynamic process and difficult to model. Traditional deep imitation learning techniques for implementing autonomous robotic pouring have an inherent black-box effect and require a large amount of demonstration data for model training. To address these issues, an Explainable Hierarchical Imitation Learning (EHIL) method is proposed in this paper such that a robot can learn high-level general knowledge and execute low-level actions across multiple drink pouring scenarios. Moreover, with the EHIL method, a logical graph can be constructed for task execution, through which the decision-making process for action generation can be made explainable to users and the causes of failure can be traced out. Based on the logical graph, the framework is manipulable to achieve different targets while the adaptability to unseen scenarios can be achieved in an explainable manner. A series of experiments have been conducted to verify the effectiveness of the proposed method. Results indicate that EHIL outperforms the traditional behavior cloning method in terms of success rate, adaptability, manipulability, and explainability. Note to Practitioners—Pouring liquids is a common activity in people’s daily lives and all wet-lab industries. Drink pouring dynamic control is difficult to model, while the accurate perception of flow is challenging. To enable the robot to learn under unknown dynamics via observing the human demonstration, deep imitation learning can be used. To address the limitations of traditional deep neural networks, an Explainable Hierarchical Imitation Learning (EHIL) method is proposed in this paper. The proposed method enables the robot to learn a sequence of reasonable pouring phases for performing the task rather than simply execute the task via traditional behavior cloning. In this way, explainability and safety can be ensured. Manipulability can be achieved by reconstructing the logical graph. The target of this research is to obtain pouring dynamics via the learning method and realize the precise and quick pouring of drink from the source containers to various targeted containers with reliable performance, adaptability, manipulability, and explainability.
3 September 2021 in IEEE Robotics and Automation Letters
In robotic teleoperation, an operator usually needs to be trained for long hours. One of the factors leading to this steep learning curve is the quality of telepresence. This letter proposes a novel and user-friendly telepresence interface for manipulation. Visual information from different views is merged and presented to the operator in an intuitive way to facilitate the task based on state-of-the-art virtual reality technology. Besides rendering the scene, a virtual robot is also rendered in the immersive view so that the robot is visible even when it is occluded. We performed a series of user studies to evaluate this interface. The results show that the proposed interface can achieve a better task performance compared to a standard approach in both the manipulation efficiency and the users' preferences.
3 February 2021 in IEEE Robotics and Automation Letters
Humans use their limbs to perform various movements to interact with an external environment. Thanks to limb's variable and adaptive stiffness, humans can adapt their movements to the external unstable dynamics. The underlying adaptive mechanism has been investigated, employing a simple planar device perturbed by external 2D force patterns. In this work, we will employ a more advanced, compliant robot arm to extend previous work to a more realistic 3D-setting. We study the adaptive mechanism and use machine learning to capture the human adaptation behavior. In order to model human's stiffness adaptive skill, we give human subjects the task to reach for a target by moving a handle assembled on the end-effector of a compliant robotic arm. The arm is force controlled and the human is required to navigate the handle inside a non-visible, virtual maze and explore it only through robot force feedback when contacting maze virtual walls. By sampling the hand's position and force data, a computational model based on a combination of model predictive control and nonlinear regression is used to predict participants' successful trials. Our study shows that participants selectively increased the stiffness within the axis direction of uncertainty to compensate for instability caused by a divergent external force field. The learned controller was able to successfully mimic this behavior. When it is deployed on the robot for the navigation task, the robot arm successfully adapt to the unstable dynamics in the virtual maze, in a similar manner as observed in the participants’ adaptation skill.
22 July 2020 in Industrial Robot: the international journal of robotics research and application
Purpose: The purpose of this paper is to propose a new joint friction model, which can accurately model the real friction, especially in cases with sudden changes in the motion direction. The identification and sensor-less control algorithm are investigated to verify the validity of this model. Design/methodology/approach: The proposed friction model is nonlinear and it considers the angular displacement and angular velocity of the joint as a secondary compensation for identification. In the present study, the authors design a pipeline – including a manually designed excitation trajectory, a weighted least squares algorithm for identifying the dynamic parameters and a hand guiding controller for the arm’s direct teaching. Findings: Compared with the conventional joint friction model, the proposed method can effectively predict friction factors during the dynamic motion of the arm. Then friction parameters are quantitatively obtained and compared with the proposed friction model and the conventional friction model indirectly. It is found that the average root mean square error of predicted six joints in the proposed method decreases by more than 54%. The arm’s force control with the full torque using the estimated dynamic parameters is qualitatively studied. It is concluded that a light-weight industrial robot can be dragged smoothly by the hand guiding. Practical implications: In the present study, a systematic pipeline is proposed for identifying and controlling an industrial arm. The whole procedure has been verified in a commercial six DOF industrial arm. Based on the conducted experiment, it is found that the proposed approach is more accurate in comparison with conventional methods. A hand-guiding demo also illustrates that the proposed approach can provide the industrial arm with the full torque compensation. This essential functionality is widely required in many industrial arms such as kinaesthetic teaching. Originality/value: First, a new friction model is proposed. Based on this model, identifying the dynamic parameter is carried out to obtain a set of model parameters of an industrial arm. Finally, a smooth hand guiding control is demonstrated based on the proposed dynamic model.