5 January 2022 in IEEE Robotics and Automation Letters
Grasping with anthropomorphic robotic hands involves much more hand-object interactions compared to parallel-jaw grippers. Modeling hand-object interactions is essential to the study of multi-finger hand dextrous manipulation. This work presents DVGG, an efficient grasp generation network that takes single-view observation as input and predicts high-quality grasp configurations for unknown objects. In general, our generative model consists of three components: 1) Point cloud completion for the target object based on the partial observation; 2) Diverse sets of grasps generation given the complete point cloud; 3) Iterative grasp pose refinement for physically plausible grasp optimization. To train our model, we build a large-scale grasping dataset that contains about 300 common object models with 1.5 M annotated grasps in simulation. Experiments in simulation show that our model can predict robust grasp poses with a wide variety and high success rate. Real robot platform experiments demonstrate that the model trained on our dataset performs well in the real world. Remarkably, our method achieves a grasp success rate of 70.7% for novel objects in the real robot platform, which is a significant improvement over the baseline methods.
28 June 2024 in IEEE Transactions on Robotics
This article investigates the challenge of enabling multi-finger hands to perform human-like functional grasping for various intentions. However, accomplishing functional grasping in real robot hands present many challenges, including handling generalization ability for kinematically diverse robot hands, generating intention-conditioned grasps for a large variety of objects, and incomplete perception from a single-view camera. In this work, we first propose a six-step functional grasp synthesis algorithm based on fine-grained contact modeling. With the fine-grained contact-based optimization and learned dense shape correspondence, the algorithm is adaptable to various objects of the same category and a wide range of multi-finger hands using few demonstrations. Secondly, over 10K functional grasps are synthesized to train our neural network, named DexFG-Net, which generates intention-conditioned grasps based on reconstructed object. Extensive experiments in the simulation and physical grasps indicate that the grasp synthesis algorithm can produce human-like functional grasp with robust stability and functionality, and the DexFG-Net can generate plausible and human-like intention-conditioned grasping postures for anthropomorphic hands. Project page and video demonstration is available at https://v-wewei.github.io/sr_dexgrasp .
31 August 2022 in IEEE Transactions on Cognitive and Developmental Systems
Human-robot interaction plays an important role in robots serving human production and life. Object handover between humans and robotics is one of the fundamental problems of human-robot interaction. The majority of current work uses parallel-jaw grippers as the end-effector device, which limits the ability of the robot to grab miscellaneous objects from human and manipulate them subsequently. In this paper, we present a framework for human-to-robot dexterous handover using an anthropomorphic hand. The framework takes images captured by two cameras to complete handover scene understanding, grasp configurations prediction, and handover execution. To enable the robot to generalize to diverse delivered objects with miscellaneous shapes and sizes, we propose an anthropomorphic hand grasp network (AHG-Net), an end-to-end network that takes the single-view point clouds of the object as input and predicts the suitable anthropomorphic hand configurations with 5 different grasp taxonomies. To train our model, we build a large-scale dataset with 1M hand grasp annotations from 5K single-view point clouds of 200 objects. We implement a handover system using a UR5 robot arm and HIT-DLR II anthropomorphic robot hand based on our presented framework, which can not only adapt to different human givers but generalize to diverse novel objects with various shapes and sizes. The generalizability, reliability, and robustness of our method are demonstrated on 15 different novel objects with arbitrary handover poses from frontal and lateral positions, a system ablation study, a grasp planner comparison, and a user study on 6 participants delivering 15 objects from two benchmark sets.
28 November 2024 in IEEE Transactions on Robotics
Human-robot object handovers are essential for robots to effectively serve human needs in various domains of human–robot interaction and collaboration, yet remain a significant challenge. Remarkable progress has been made by parallel-jaw gripper robots in grasp generation and motion planning for handovers, while few studies address this issue using anthropomorphic hands, necessitating the ability to handle higher collision probabilities and lower approaching space under occluded situations. In this article, we present a reactive human-to-robot dexterous handover framework for anthropomorphic hands. The closed-loop framework employs an effective collision detection and grasp selection approach to ensure safe and smooth motion in unstructured environments. We implement a handover system using a UR5 robot arm and a Schunk SVH Hand based on the presented framework, which can react to human motion during the handover process, generalize to diverse objects with different 6-DoF poses, and execute suitable grasp configurations. The generalizability, reliability, and efficiency of our method are demonstrated through the handover of 30 novel objects, a system ablation study for submodule evaluation, and a user study assessment involving eight participants.
5 July 2021 in IEEE Transactions on Instrumentation and Measurement
Tubular solder joint detection is an important and challenging issue in the industry, due to the illegible objects, rarely collected datasets and requiring high-precision and real-time performance for positioning and angle estimation. In this article, we propose an Attention integrated and Cross-spatial feature fused Rotation Network (ACR-Net) for tubular solder joint detection, which consists of a new feature extraction network named ECA-CSPDarknet44, a cross-spatial feature fusion network (CFFN), and a bin-based rotation detection network (BRDN). The proposed network can efficiently detect oriented tubular solder joints with high-precision and real-time performance. ECA-CSPDarknet44 with attention mechanism was presented, which can adaptively guide the network to learn important features of tubular solder joints, significantly improving the ability of feature extraction. By integrating multiscale global features and multiscale local region features, CFFN can enhance the network's ability to express the characteristics of tubular solder joints. Meanwhile, BRDN is proposed for oriented bounding box regression through a decoupling approach, which regresses target parameters efficiently and accurately with little increasing of model complexity. Finally, we establish a tubular solder joint dataset and conduct sufficient experiments to verify the effectiveness of our method. Our proposed ACR-Net achieves 98.8% mAP with 31.3 frames per second (FPSs) on the dataset, meeting the high-precision and real-time requirements of industrial systems.
8 January 2025 in IEEE Robotics and Automation Letters
Teleoperation offers the possibility of enabling robots to replace humans in operating within hazardous environments. While it provides greater adaptability to unstructured settings than full autonomy, it also imposes significant burdens on human operators, leading to operational errors. To address this challenge, shared control, a key aspect of human-robot collaboration methods, has emerged as a promising alternative. By integrating direct teleoperation with autonomous control, shared control ensures both efficiency and stability. In this letter, we introduce a shared control framework for human-robot collaborative tele-grasping in clutter with five-fingered robotic hands. During teleoperation, the operator's intent to reach the target object is detected in real-time. Upon successful detection, continuous and smooth grasping plans are generated, allowing the robot to seamlessly take over control and achieve natural, collision-free grasping. We validate the proposed framework through fundamental component analysis and experiments on real-world platforms, demonstrating the superior performance of this framework in reducing operator workload and enabling effective grasping in clutter.