1 August 2024 in IEEE Transactions on Automation Science and Engineering
Robotic arm target grasping by vision support is a commonly used method in grasping tasks and is usually used for multi-target complex scenes. Where vision support is generally used to identify the targets and to get their positions, categories and sizes. Most robotic arm grasping tasks using target recognition methods as visual inspection ignore the relationship between target objects such as the occlusion problem between objects. This limits the targets to be grasped and makes the crawling task inefficient. We propose Grasping with Occlusion-Aware aLly (GOAL) method based on binocular stereo-vision. Firstly, occlusion relationships in the view are directly inferred and targets are segmented as well as localized. Subsequently, multi-target grasping pose estimation is performed to obtain effective grasping positions. Ultimately, validation is conducted on a high-resolution dataset using the EPSON robotic arm. Note to Practitioners —This research significantly advances the field by addressing occlusion challenges in robotic grasping, offering effective methods, a valuable dataset, and practical insights. The proposed Grasping with Occlusion-Aware aLly (GOAL) method was validated on a high-resolution dataset using the EPSON robotic arm, showcasing its applicability and efficiency in real-world scenarios. This work provides valuable contributions to practitioners in the field of robotic manipulation and grasping tasks.
26 April 2023 in Research Square
Robotic arms are currently in the spotlight of the industry of future, but their efficiency faces huge challenges. The efficient grasping of the robotic arm, replacing human work, requires visual support. In this paper, we first propose to augment end-to-end deep learning gasping with a object detection model in order to improve the efficiency of grasp pose prediction. The accurate positon of the object is difficult to obtain in the depth image due to the absent of the label in point cloud in an open environment. In our work, the detection information is fused with the depth image to obtain accurate 3D mask of the point cloud, guiding the classical GraspNet to generate more accurate grippers. The detection-driven 3D mask method allows also to design a priority scheme increasing the adaptability of grasping scenarios. The proposed grasping method is validated on multiple benchmark datasets achieving state-of-the-art performances.