26 April 2024 in IEEE/ASME Transactions on Mechatronics
Six-DoF grasp detection has attracted increasing attention due to its high flexibility in manipulation tasks. Recent learning-based methods typically use point sampling or filtering to extract a small subset of scene points for grasp pose computation. However, the sampling method disregards the fact that each scene point contributes differently to the grasp, while the filtering method nearly destroys the geometrical structure of the scene. Concurrently, indirectly learning point features by inputting point coordinates to the network is insufficient for detecting grasp poses since this task heavily depends on the geometrical structure of the local point clouds. In order to solve the above-mentioned problems, we design a novel interest point selection algorithm in the grasp scene, and a novel explicit geometrical feature extraction algorithm for local point clouds in the six-DoF grasp pose detection task. We include the basic grasp points, nearby object points and background points as interest points in calculating the grasp poses. Moreover, we add a module that predicts grasp collision within the network to eliminate the time-consuming collision detection in postprocessing. The proposed network is trained and evaluated on the GraspNet1Billion dataset, and achieves state-of-the-art performance. We also conduct an ablation study to validate the proposed algorithms and the network. Furthermore, we successfully implement our network on a UR5e robotic arm to demonstrate its practical applicability in both single-object and multiobject grasp scenarios.
1 October 2026 in IEEE/ASME Transactions on Mechatronics
Image-based visual servoing guides a robotic system to its desired place through ongoing feedback from visual signals. However, interference such as object occlusion and camera drift may cause visual signal loss, ultimately leading to servo failure. This article proposes a novel continuous image-based visual servoing (CIBVS) approach to address the potential visual signal loss. A geometric relationship is established between the positions of natural features and servo features. In each iteration, natural features are coarsely matched between frames, and mismatched pairs are eliminated by the proposed refining score based on the geometric relationship. When visual signal loss occurs, CIBVS leverages the natural features and the geometric relationship to estimate the lost servo features through optimization, ensuring continuous and stable visual feedback generation. In simulation, CIBVS was evaluated across three scenarios and 750 test cases, achieving an enhanced success rate and estimation accuracy. In the real world, CIBVS was integrated into various robotic systems, including manipulators, drones, and ground vehicles, to achieve multiple tasks. Experimental results demonstrated that, in scenarios with visual signal loss, CIBVS achieved significant improvements over extended-Kalman-filter-based IBVS and visual-odometry-based IBVS in servoing success rate, estimation accuracy, and convergence speed.