Touch-free guided hand gesture recognition for human-robot interactions plays an increasingly significant role in teleoperated surgical robot systems. Indeed, despite depth cameras provide more practical information for recognition accuracy enhancement, the instability and computational burden of depth data represent a tricky problem. In this letter, we propose a novel multi-sensor guided hand gesture recognition system for surgical robot teleoperation. A multi-sensor data fusion model is designed for performing interference in the presence of occlusions. A multilayer Recurrent Neural Network (RNN) consisting of a Long Short-Term Memory (LSTM) module and a dropout layer (LSTM-RNN) is proposed for multiple hand gestures classification. Detected hand gestures are used to perform a set of human-robot collaboration tasks on a surgical robot platform. Classification performance and prediction time is compared among the LSTM-RNN model and several traditional Machine Learning (ML) algorithms, such as k-Nearest Neighbor (k-NN) and Support Vector Machines (SVM). Results show that the proposed LSTM-RNN classifier is able to achieve a higher recognition rate and faster inference speed. In addition, the present adaptive data fusion system shows a strong anti-interference capability for hand gesture recognition in real-time.

Although the merits of electromyography (EMG)-based control of powered assistive systems have been certified, the factors that affect the performance of EMG-based human-robot cooperation, which are very important, have received little attention. This study investigates whether a more physiologically appropriate model could improve the performance of human-robot cooperation control for an ankle power-assist exoskeleton robot. To achieve the goal, an EMG-driven Hill-type neuromusculoskeletal model (HNM) and a linear proportional model (LPM) were developed and calibrated through maximum isometric voluntary dorsiflexion (MIVD). The two control models could estimate the real-time ankle joint torque, and HNM is more accurate and can account for the change of the joint angle and muscle dynamics. Then, eight healthy volunteers were recruited to wear the ankle exoskeleton robot and complete a series of sinusoidal tracking tasks in the vertical plane. With the various levels of assist based on the two calibrated models, the subjects were instructed to track the target displayed on the screen as accurately as possible by performing ankle dorsiflexion and plantarflexion. Two measurements, the root mean square error (RMSE) and root mean square jerk (RMSJ), were derived from the assistant torque and kinematic signals to characterize the movement performances, whereas the amplitudes of the recorded EMG signals from the tibialis anterior (TA) and the gastrocnemius (GAS) were obtained to reflect the muscular efforts. The results demonstrated that the muscular effort and smoothness of tracking movements decreased with an increase in the assistant ratio. Compared with LPM, subjects made lower physical efforts and generated smoother movements when using HNM, which implied that a more physiologically appropriate model could enable more natural and human-like human-robot cooperation and has potential value for improvement of human-exoskeleton interaction in future applications.

Although many studies have certified the advantages of human–robot cooperation control with admittance model (AM), robot compliance, and movement smoothness need to be further improved. In this article, a trajectory deformation algorithm (TDA) is developed as a high-level trajectory planner, which can plan subject's desired trajectory based on interaction force during physical human–robot interaction (pHRI). A low-level proportional-derivative (PD) position controller is selected to ensure the lower limb rehabilitation robot can track the desired trajectory. Then, the validity of TDA is verified through simulation and experiment studies. The energy per unit distance (EPUD) and dimensionless squared jerk (DSJ) are chosen as indicators of robot compliance and movement smoothness, respectively. The experimental results demonstrated that both the EPUD and the DSJ values using TDA are smaller than that using the AM, indicating the TDA can improve robot compliance and movement smoothness. Therefore, it may have great potential in fields involving pHRI, such as robot-aided rehabilitation.

Background: Robot-assisted rehabilitation is an advanced new technology in stroke rehabilitation to provide intensive training. Post-stroke motor recovery depends on active rehabilitation by voluntary participation of patient’s paretic motor system as early as possible in order to promote reorganization of brain. However, voluntary residual motor efforts to the affected limb have not been involved enough in most robot-assisted rehabilitation for patients after stroke. The objective of this study is to evaluate the feasibility of robot-assisted rehabilitation using myoelectric control on upper limb motor recovery.Methods: In the present study, an exoskeleton-type rehabilitation robotic system was designed to provide voluntarily controlled assisted torque to the affected wrist. Voluntary intention was involved by using the residual surface electromyography (EMG) from flexor carpi radialis(FCR) and extensor carpi radialis (ECR)on the affected limb to control the mechanical assistance provided by the robotic system during wrist flexion and extension in a 20-session training. The system also applied constant resistant torque to the affected wrist during the training. Sixteen subjects after stroke had been recruited for evaluating the tracking performance and therapeutical effects of myoelectrically controlled robotic system.Results: With the myoelectrically-controlled assistive torque, stroke survivors could reach a larger range of motion with a significant decrease in the EMG signal from the agonist muscles. The stroke survivors could be trained in the unreached range with their voluntary residual EMG on the paretic side. After 20-session rehabilitation training, there was a non-significant increase in the range of motion and a significant decrease in the root mean square error (RMSE) between the actual wrist angle and target angle. Significant improvements also could be found in muscle strength and clinical scales.Conclusions: These results indicate that robot-aided therapy with voluntary participation of patient’s paretic motor system using myoelectric control might have positive effect on upper limb motor recovery.

Various rehabilitation robots have been employed to recover the motor function of stroke patients. To improve the effect of rehabilitation, robots should promote patient participation and provide compliant assistance. This paper proposes an adaptive admittance control scheme (AACS) consisting of an admittance filter, inner position controller, and electromyography (EMG)-driven musculoskeletal model (EDMM). The admittance filter generates the subject's intended motion according to the joint torque estimated by the EDMM. The inner position controller tracks the intended motion, and its parameters are adjusted according to the estimated joint stiffness. Eight healthy subjects were instructed to wear the ankle exoskeleton robot, and they completed a series of sinusoidal tracking tasks involving ankle dorsiflexion and plantarflexion. The robot was controlled by the AACS and a non-adaptive admittance control scheme (NAACS) at four fixed parameter levels. The tracking performance was evaluated using the jerk value, position error, interaction torque, and EMG levels of the tibialis anterior (TA) and gastrocnemius (GAS). For the NAACS, the jerk value and position error increased with the parameter levels, and the interaction torque and EMG levels of the TA tended to decrease. In contrast, the AACS could maintain a moderate jerk value, position error, interaction torque, and TA EMG level. These results demonstrate that the AACS achieves a good tradeoff between accurate tracking and compliant assistance because it can produce a real-time response to stiffness changes in the ankle joint. The AACS can alleviate the conflict between accurate tracking and compliant assistance and has potential for application in robot-assisted rehabilitation.