We present a survey of recent work on robot manipulation and sensing of deformable objects, a field with relevant applications in diverse industries such as medicine (e.g. surgical assistance), food handling, manufacturing, and domestic chores (e.g. folding clothes). We classify the reviewed approaches into four categories based on the type of object they manipulate. Furthermore, within this object classification, we divide the approaches based on the particular task they perform on the deformable object. Finally, we conclude this survey with a discussion of the current state-of-the-art approaches and propose future directions within the proposed classification.

Vision feedback control loop techniques are efficient for a large class of applications, but they come up against difficulties when the initial and desired robot positions are distant. Classical approaches are based on the regulation to zero of an error function computed from the current measurement and a constant desired one. By using such an approach, it is not obvious how to introduce any constraint in the realized trajectories or to ensure the convergence for all the initial configurations. In this paper, we propose a new approach to resolve these difficulties by coupling path planning in image space and image-based control. Constraints such that the object remains in the camera field of view or the robot avoids its joint limits can be taken into account at the task planning level. Furthermore, by using this approach, current measurements always remain close to their desired value, and a control by image-based servoing ensures robustness with respect to modeling errors. The proposed method is based on the potential field approach and is applied whether the object shape and dimensions are known or not, and when the calibration parameters of the camera are well or badly estimated. Finally, real-time experimental results using an eye-in-hand robotic system are presented and confirm the validity of our approach.

This study is focused on developing a new dexterous soft robotic gripper with three fingers and an active palm capable of performing in-hand manipulation purposes. This innovative design meets all the dexterous manipulation requirements without any increase in mechanical complexity. In each finger, the bending position can be modified and controlled by moving a stiff rod inserted inside the center hole of the finger. In this way, the effective length of the manipulation can be changed. As a result, these reconfigurable fingers provide a more accessible workspace than conventional soft grippers. Besides, a large diversity of the fingers shape configurations results in more dexterity and in-hand manipulation capability. Workspace analysis is accomplished to characterize the advantages of the proposed design. The effectiveness of this soft robotic gripper is validated by different in-hand manipulation experimental tests, including rotation, regrasping, and rolling. The results suggest a promising solution to bridge the design gap between hard and soft robots for dexterous manipulation tasks. The hybrid design carries advantages of these two classes, such as reconfigurability, position, and shape control from hard robots, with large degrees of freedom (DOFs), complex deformations, and lightweight from soft robots. Like human manipulation, the palm plays a major role in stable grasping, especially for enhancing the in-hand manipulation capability. Therefore, we also investigate two types of vacuum palms (suction cup and granular particles) to guarantee a wide range of object manipulation tasks that cannot be completely performed by previously suggested soft grippers.

Omnidirectional cameras have a wide field of view and are thus used in many robotic vision tasks. An omnidirectional view may be acquired by a fisheye camera which provides a full image compared to catadioptric visual sensors and do not increase the size and the weakness of the imaging system with respect to perspective cameras. We prove that the unified model for catadioptric systems can model fisheye cameras with distortions directly included in its parameters. This unified projection model consists on a projection onto a virtual unitary sphere, followed by a perspective projection onto an image plane. The validity of this assumption is discussed and compared with other existing models. Calibration and partial Euclidean reconstruction results help to confirm the validity of our approach. Finally, an application to the visual servoing of a mobile robot is presented and experimented.