While most of the existing works on grasp pose detection have assumed a complete 3D object model, this paper proposes a grasp pose detection method for unknown deformable objects, based on visual information. The proposed method is comprised two parts; (1) pix2stiffness estimation, which generates a stiffness map that indicates the object's stiffness for each pixel in an image using generative adversarial networks (GAN), and (2) grasp pose detection, which adapts a stiffness map to maximally reduce the object's deformation and avoid any possible damage. We demonstrate the validity of the proposed method and evaluate the estimation accuracy via simulations, and in a real environment. We also verify that the proposed approach can plan how to grasp an object using a few 3D models of objects. GRAPHICAL ABSTRACT
In this study, we introduce a method for estimating deformability from the appearance of an object and a technique for detecting the grasp pose using a single-depth image. Deformability, defined as a score, provides an approximate measure of the ease with which an object can be deformed, based on the intuitive human assumption that appearance correlates with this property. Unlike previous approaches that synthesize the entire image in one step and struggle with accuracy in complex scenarios, such as cluttered environments with multiple objects, our method advances the field by employing an encoder-decoder model for simultaneous deformability estimation and instance segmentation from depth images, as well as by enhancing grasp pose detection for deformable objects under specific conditions using the outcomes of our deformability assessments. We demonstrated the efficacy of our integrated approach in diverse challenging settings, including the manipulation of deformable hollow objects in cluttered scenes, interaction with partially deformable objects, and displacement of nearby deformable objects in real-world environments.
20 June 2023 in Journal of Robotics and Mechatronics
This study investigated a pregrasp strategy for hooking and caging ring-shaped objects. Through-hole features enable the robot hand to hook an object with holes by inserting its finger into one of the holes. Compared to directly grasping the ring, an inserting motion is more convenient to allow the uncertainty of positioning errors and avoid collisions between the hand and the object. Instead of recognizing the exact shape of the object, we only detected its ring-shaped feature as a through-hole to be inserted and estimated its approximate center position and orientation from the point cloud of the object. The estimated geometric properties enabled the approaching motion of the robotic gripper to complete insertion. The proposed perception and motion-planning method was demonstrated for rigid and deformable objects with holes.