Robotic grasping is essential for intelligent manufacturing. This article presents a novel anchor-free grasp detector based on fully convolutional network for detecting multiple valid grasps from RGB-D images in real time. Grasp detection is formulated as a closest horizontal or vertical rectangle regression task and a grasp angle classification task. By directing predicting grasps at feature points, our method eliminates the predefined anchors that commonly used in prior methods, and thus anchor-related hyperparameters and complex computations are avoided. For suppressing ambiguous and low-quality training samples, a new sample assignment strategy that combines center-sampling and regression weights is proposed. Our method achieves a state-of-the-art accuracy of 99.4% on Cornell and 96.2% on Jacquard dataset, and real-time speed of 104 frames per second, with approximately 2× fewer parameters and 8× less training time compared to previous one-stage detector. Moreover, an efficient multiscale feature fusion module is integrated to improve the performance of multigrasp detection by 25%. In real-world robotic grasping of novel objects, our method achieves a grasp success rate of 91.3% for single object and 83.3% for multiple objects with only 26 ms used for the whole planning. The results demonstrate that our method is robust for potential industrial applications.
Aiming at stroke patients’ hand rehabilitation training, we present a hand exoskeleton with both active and passive control modes for neural rehabilitation. The exoskeleton control system is designed as a human–robot interaction control system based on field-programmable gate array (FPGA) and Android mobile terminal with good portability and openness. Passive rehabilitation pattern based on proportional derivative (PD) inverse dynamic control method and active rehabilitation pattern based on impedance method, are established respectively. By the comparison of the threshold value and the force on the fingertip of the exoskeleton from the sensor, the automatic switch between active and passive rehabilitation mode is accomplished. The hand model is built in Android environment that can synchronize the movement of the hand. It can also induce patients to participate in rehabilitation training actively. To verify the proposed control approach, we set up and conduct an experiment to do the passive rehabilitation mode, active rehabilitation mode, and active plus passive mode experimental researches. The experiment results effectively verify the feasibility of the exoskeleton system fulfilling the proposed control strategy.
28 December 2020 in IEEE Transactions on Multimedia
3D Hand pose estimation from a single depth image is an essential topic in computer vision and human-computer interaction. Although the rising of deep learning boosts the accuracy a lot, the problem is still hard to solve due to the complex structure of the human hand. Two existing types of methods with deep learning, i.e. the regression-based and detection-based methods, either lose spatial information of the hand structure or lack direct supervision of the joint coordinates. In this paper, we propose a novel Differentiable Spatial Regression method which combines the advantages of these two types of methods to overcome each other's shortcomings. Our method uses spatial-form representation (SFR) to maintain spatial information and differentiable decoder to establish a direct supervision. Following the procedure suggested by our method, a particular model named SRNet is designed which uses a combination of 2D heatmaps and local offset maps as SFRs. Two modules named Plane Regression and Depth Regression are designed as differentiable decoder to regress plane coordinates and depth coordinates respectively. Ablation study demonstrates the superiority of our method over the two combined methods since the differentiable decoder leads to better SFRs learned by the network itself other than human design. Extensive experiments on four public datasets demonstrate that SRNet is comparable with the state-of-the-art models.
29 January 2021 in IEEE Transactions on Cognitive and Developmental Systems
Introducing interactive system into portable robots for hand rehabilitation has always been a crucial topic. Moreover, hand rehabilitation with force cognition can make patients participate actively and improve rehabilitation effect. In this paper, we design a portable robotic device with interactive system for patients to rehabilitate with force cognition. Firstly, an exoskeleton glove is designed with a compact mechanical structure which is controlled by a real-time feedback system. The portable device allows patients to rehabilitate not only in hospital. Next, an interactive system including virtual environment and force cognition is introduced to detect the hand motion and collision. At last, clinical tests of our portable device is carried out with 9 subjects after tendon injury to show the effective assistance with our device.
18 March 2019 in Industrial Robot: the international journal of robotics research and application
Purpose: The purpose of this paper is to propose an error model for serial robot kinematic calibration based on dual quaternions. Design/methodology/approach: The dual quaternions are the combination of dual-number theory and quaternion algebra, which means that they can represent spatial transformation. The dual quaternions can represent the screw displacement in a compact and efficient way, so that they are used for the kinematic analysis of serial robot. The error model proposed in this paper is derived from the forward kinematic equations via using dual quaternion algebra. The full pose measurements are considered to apply the error model to the serial robot by using Leica Geosystems Absolute Tracker (AT960) and tracker machine control (T-MAC) probe. Findings: Two kinematic-parameter identification algorithms are derived from the proposed error model based on dual quaternions, and they can be used for serial robot calibration. The error model uses Denavit–Hartenberg (DH) notation in the kinematic analysis, so that it gives the intuitive geometrical meaning of the kinematic parameters. The absolute tracker system can measure the position and orientation of the end-effector (EE) simultaneously via using T-MAC. Originality/value: The error model formulated by dual quaternion algebra contains all the basic geometrical parameters of serial robot during the kinematic calibration process. The vector of dual quaternion error can be used as an indicator to represent the trend of error change of robot’s EE between the nominal value and the actual value. The accuracy of the EE is improved after nearly 20 measurements in the experiment conduct on robot SDA5F. The simulation and experiment verify the effectiveness of the error model and the calibration algorithms.
Anaplastic thyroid cancer (ATC) comprises approximately 2% of all thyroid cancers, and its median survival rate remains poor because of its resistance to conventional therapy. Vascular endothelial growth factor receptor (VEGFR)-targeted therapeutics-loaded mesoporous silica nanoparticles represent a major advance for angiogenesis imaging and inhibition in lethal cancers. In the present study, we aimed to assess whether 131I-labeled anti-VEGFR2 targeted mesoporous silica nanoparticles would have antitumor efficacy in an ATC tumor-bearing nude mouse model. Using in vitro and in vivo studies, we investigated the increased targeting ability and retention time in the anti-VEGFR2 targeted group using confocal microscopy and a γ counter. The tumor tissue radioactivity of the anti-VEGFR2 targeted group at 24 and 72 h after intratumoral injection was significantly higher than that of the non-targeted groups (all P < 0.05). Moreover, we found that radioactive accumulation was obvious even at 3 week post-injection in the anti-VEGFR2 targeted group via single-photon emission computed tomography/computed tomography, which was not seen at 3 day post-injection in the Na131I group. Meanwhile, compared with the non-targeted group, tumor growth in the targeted group was significantly inhibited, without causing apparent systemic toxic effects. Additionally, the median survival time in the targeted group (41 days) was significantly prolonged compared with that in the non-targeted (34 days) or Na131I (25 days) groups (both P < 0.01). Our data support the view that the as-developed 131I-labeled anti-VEGFR2 targeted mesoporous silica nanoparticles showed promising results in ATC tumor-bearing mouse model and such an approach might represent a novel therapeutic option for ATC.