Grasping unseen objects has become one of the most critical abilities of robots, where identifying the feasible grasp locations is the key technique. Previously proposed methods generally rely on grid search strategy, which evaluates grasp candidates at regular interval, or directly adopt the deep convolutional network, which requires large dataset and powerful computational ability. In this paper, we propose an edge-based grasp detection strategy, which combines low-level features and a lightweight convolutional neural network (CNN). Specifically, two grip criteria are first introduced to select feasible point pairs and determine the corresponding grasp candidates. Then, a lightweight CNN model is rapidly trained with limited number of samples in order to recognize feasible grasps. Our method does not require additional sensor information and can work well with only RGB image. Comparative experiments on public dataset verify the effectiveness of our efficient grasp search, which is superior to existing grasp search strategies in accuracy and efficiency. Meanwhile, it achieves competitive results with traditional CNN-based methods using less training time. In addition, we also test the proposed approach under real-world robotic grasping scenario.

Most available grasp detection methods tend to directly predict grasp configurations with deep neural networks, where all features are equally extracted and utilized, leading to the relative restriction of truly useful grasping features. Inspired by the observed three-section structure pattern revealed by human-labeled graspable rectangles, we first design a structure prior attention (SPA) module which uses two-dimensional encoding to enhance the local patterns and utilizes self-attention mechanism to reallocate distribution of grasping-specific features. Then, the proposed SPA module is integrated with fundamental feature extraction modules and residual connection to achieve the implicit and explicit feature fusion, which further serves as the building block of our proposed Unet-like grasp detection network. It takes RGBD images as input and outputs image-size feature maps, from which the grasp configurations can be determined. Extensive comparative experiments on the five public datasets prove our method’s superiority to other approaches in detection accuracy, achieving 99.2%, 96.1%, 98.0%, 86.7%, and 92.6% on the Cornell, Jacquard, Clutter, VMRD, and GraspNet datasets. With visual evaluation metrics and user study, the quality maps generated by our method possess more concentrative distribution of high-confidence grasps and clearer discrimination with backgrounds. In addition, its effectiveness is also verified by robotic grasping under real-world scenario, leading to higher success rate.

Although existing grasp detection methods have achieved encouraging performance under well-light conditions, repetitive experiments have found that the detection performance would deteriorate drastically under low-light conditions. Although supplementary information can be provided by additional sensors, such as depth camera, the sparse and weak visual features still hinder the improvement of detection accuracy. In order to address these, we propose a visual enhancement guided grasp detection model (VERGNet) to improve the robustness of robotic grasping in low-light conditions. Firstly, a simultaneous grasp detection and low-light feature enhancement framework is designed, which integrates residual blocks with coordinate attention to re-optimize grasping features. Then, the unsupervised low-light feature enhancement strategy is adopted to reduce the dependence on paired data as well as improve the algorithmic robustness to low-light conditions. Extensive experiments are finally conducted on two newly-constructed low-light grasp datasets and the proposed method achieves 98.9% and 91.2% detection accuracy respectively, which are superior to comparative methods. Besides, the effectiveness in our method has also been validated in real-world low-light imaging scenarios.

The soft fingers and strategic grasping skills enable the human hands to grasp objects in a stable manner. This letter is to model human grasping skills and transfer the learned skills to robots to improve grasping quality and success rate. First, we designed a wearable tool-like parallel hand exoskeleton equipped with optical tactile sensors to acquire multimodal information, including hand positions and postures, the relative distance of the exoskeleton claws, and tactile images. Using the demonstration data, we summarized three characteristics observed from human demonstrations, involving varying-speed actions, grasping effect read from tactile images and grasping strategies for different positions. The characteristics were then utilized in the robot skill modelling to achieve a more human-like grasp. Since no force sensors are fixed to the claws, we introduced a new variable, called “grasp depth”, to represent the grasping effect on the object. The robot grasping strategy diagram is constructed as follows: First, grasp quality is predicted using a linear array network (LAN) and global visual images as inputs. The conditions such as grasp width, depth, position, and angle are also predicted. Second, with the grasp width and depth of the object determined, dynamic movement primitives (DMPs) are employed to mimic human grasp actions with varying velocities. To further enhance grasp quality, a final action adjustment based on tactile detection is performed during the near-grasp time. The proposed strategy was validated through experiments conducted with a Franka robot with a self-designed gripper. The results demonstrate that robot grasping test achieved an increase in the grasping success rate from 82% to 96%, compared to the results obtained by pure LAN and constant grasp depth testing.

In order to achieve higher accuracy, the complexity of grasp detection network increases accordingly with complicated model structures and tremendous parameters. Although various light-weight strategies are adopted, directly designing the compact network can be sub-optimal and difficult to strike the balance between accuracy and model size. To solve this problem, we explore a more efficient grasp detection model from two aspects: elaborately designing a light-weight network and performing knowledge distillation on the designed network. Specifically, based on the designed light-weight backbone, the features from RGB and D images with unequal effective grasping information rates are fully utilized and the information compensation strategies are adopted to make the model small enough while maintaining its accuracy. Then, the grasping features contained in the large teacher model are adaptively and effectively learned by our proposed method via knowledge distillation. Experimental results indicate that the proposed method is able to achieve comparable performance (98.9%, 93.1%, 82.3%, and 90.0% on Cornell, Jacquard, GraspNet, and MultiObj datasets, respectively) with more complicate models while reducing the parameters from MBs to KBs. Real-world robotic grasping experiment in an embedded AI computing device also prove the effectiveness of this approach.

The vision-based grasp detection method is an important technique used to research the grasping task of robots. Unfortunately, the performances of these research methods in practice are worse than the state-of-the-art accuracy on public datasets, because shifts in data distribution owing to real-world conditions are common in the real world and neural network-based methods are sensitive to small data changes. These disturbances mainly change the image texture, causing the performance of grasp detection methods to decline sharply. However, the evaluation metric of existing models does not reflect the actual robustness of the method. Therefore, we propose a new solution. First, referring to the existing research on image classification methods, we propose a benchmark to verify the realistic robustness of the grasp detection model. Second, to improve the robustness of the model, we randomly transfer texture knowledge from other images to provide variable texture information for network training. This forces the model to rely more on the contour features of the object than on the texture when making decisions; we call this approach “context-free.” We verify the effectiveness of our method for robustness enhancement in various grasp tasks and test the proposed method in a real robot grasping scene.