Deformable object manipulation (DOM) is an emerging research problem in robotics. The ability to manipulate deformable objects endows robots with higher autonomy and promises new applications in the industrial, services, and health-care sectors. However, compared to rigid object manipulation, the manipulation of deformable objects is considerably more complex and is still an open research problem. Addressing DOM challenges demands breakthroughs in almost all aspects of robotics: hardware design, sensing, (deformation) modeling, planning, and control. In this article, we review recent advances and highlight the main challenges when considering deformation in each subfield. A particular focus of our article lies in the discussions of these challenges and proposing future directions of research.

In this paper, we present a new feedback method to automatically servo-control the 3-D shape of soft objects with robotic manipulators. The soft object manipulation problem has recently received a great deal of attention from robotics researchers because of its potential applications in, e.g., food industry, home robots, medical robotics, and manufacturing. A major complication to automatically control the shape of an object is the estimation of its deformation properties, which determines how the manipulator's motion actively transforms into deformations. Note that these properties are rarely known beforehand, and its offline parametric identification is difficult and/or impractical to conduct in many applications. To cope with this issue, we developed a new algorithm that computes in real time the unknown deformation parameters of a soft object; this algorithm provides a valuable adaptive behavior to the deformation controller, something we cannot achieve with traditional fixed-model approaches. In contrast with most controllers in the literature, our new method can explicitly servo-control 3-D deformations (and not just 2-D image projections) in an entirely model-free way. To validate the proposed adaptive controller, we present a detailed experimental study with robotic manipulators.

This paper addresses the design of a vision-based method to automatically deform soft objects into desired two-dimensional shapes with robot manipulators. The method presents an innovative feedback representation of the object’s shape (based on a truncated Fourier series) and effectively exploits it to guide the soft object manipulation task. A new model calibration scheme that iteratively approximates a local deformation model from vision and motion sensory feedback is derived; this estimation method allows us to manipulate objects with unknown deformation properties. Pseudocode algorithms are presented to facilitate the implementation of the controller. Numerical simulations and experiments are reported to validate this new approach.

This letter presents a point cloud based robotic system for arc welding. Using hand gesture controls, the system scans partial point cloud views of workpiece and reconstructs them into a complete 3D model by a linear iterative closest point algorithm. Then, a bilateral filter is extended to denoise the workpiece model and preserve important geometrical information. To extract the welding seam from the model, a novel intensity-based algorithm is proposed that detects edge points and generates a smooth 6-DOF welding path. The methods are tested on multiple workpieces with different joint types and poses. Experimental results prove the robustness and efficiency of this robotic system on automatic path planning for welding applications.

Despite the recent progress in physically interactive and surgical robotics, the active deformation of compliant objects remains an open problem. The main obstacle to its implementation comes from the difficulty to identify or estimate the object's deformation model. In this paper, we propose a novel vision-based deformation controller for robot manipulators interacting with unknown elastic objects. We derive a new dynamic-state feedback velocity control law using the passivity-based framework. Our method exploits visual feedback to estimate the deformation Jacobian matrix in real time, avoiding any model identification steps. We prove that even in the presence of inexact estimations, the closed-loop dynamical system ensures input-to-state stability (i.e., full dissipativity) with respect to external disturbances. An experimental study with several deformation tasks is presented to validate the theory.

In this paper, we address the active deformation control of compliant objects by robot manipulators. The control of deformations is needed to automate several important tasks, for example, the manipulation of soft tissues, shaping of food materials, or needle insertion. Note that in many of these applications, the object’s deformation properties are not known. To cope with this issue, in this paper we present two new visual servoing approaches to explicitly servo-control elastic deformations. The novelty of our kinematic controllers lies in its uncalibrated behavior; our adaptive methods do not require the prior identification of the object’s deformation model and the camera’s intrinsic/extrinsic parameters. This feature provides a way to automatically control deformations in a model-free manner. The experimental results that we report validate the feasibility of our controllers.