23 June 2022 in IEEE Robotics and Automation Letters
Lower-limb exoskeleton robots can support hemiplegic patients’ affected limbs and assist in their rehabilitation. In order to set effective control strategies, it is necessary to obtain the user’s motion intention accurately and timeously. These requirements pose many challenges. The surface electromyography (sEMG) signal has long been used for detecting a person’s locomotion mode. However, most traditional myoelectric motion recognition methods need to collect multi-channel sEMG signals (more than eight) and use feature engineering to obtain the desired accuracy. Additionally, traditional methods make decisions when the recognition probability is still uncertain, resulting in misclassification during continuous recognition. In order to overcome these limitations, a dual-purpose autoencoder-guided temporal convolution network (DA-TCN) is proposed, which is used as a motion detection model for a lower-limb exoskeleton robot based on sEMG. In the proposed method, we first obtain four-channel sEMG signals without any feature engineering. Next, the DA-TCN is used to obtain the discriminative deep features of the sEMG signals. Compared with separable features from common TCNs, discriminative features assist in obtaining a more discriminative twins-loss. Finally, the classification result is redetermined by the decoder, based on the reconstruction of deep features, and misclassifications are rejected based on the value of the twins-loss. The performance of the proposed method was evaluated by obtaining data from seven volunteer subjects in seven locomotion modes. The proposed method was able to effectively detect misclassifications and significantly improved the stability of continuous recognition.
9 January 2023 in IEEE Robotics and Automation Letters
Multi-stream signals are increasingly being used in robot-assisted rehabilitation training, where the timely and accurate prediction of a patient's motor intentions is frequently required to provide simultaneous and proportional control strategies. However, existing methods for motion intent prediction typically isolate signals from different modalities, resulting in a loss of signal traceability and interpretability. Therefore, we propose a multi-stream signal-fusion strategy based on knowledge tracing (MSKT). First, we collected surface electromyography, force myography, and vibroarthrography signals from the lower extremity and used different encoders to map these signals into a problem component, concept component, and identity token component. The process of human learning was then simulated using a knowledge-tracing strategy to reconstruct these components into new features. Finally, the reconstructed set of depth features was mapped to joint angles. We validated the performance of the proposed method using motion data from eight participants in different locomotion modes and compared the results to those of a temporal convolution network and long short-term memory networks. The results indicate that the MSKT achieves the lowest root-mean-squared error. This study demonstrates that the proposed knowledge-tracing-based strategy enables accurate continuous estimation for lower-limb exoskeleton rehabilitation robots.
10 July 2025 in IEEE Transactions on Affective Computing
EEG-based emotion recognition holds significant potential in the field of brain-computer interfaces. A key challenge is extracting discriminative spatiotemporal features from electroencephalogram (EEG) signals. Existing studies often rely on domain-specific time-frequency features and analyze temporal dependencies and spatial characteristics separately, neglecting the local-global relationships and the interaction in spatiotemporal dynamics. To address this, we propose a novel network called Multi-scale Inverted Mamba (miMamba), which consists of Multi-Scale Temporal Blocks (MSTB) and Temporal-Spatial Fusion Blocks (TSFB). Specifically, MSTBs are designed to capture both local details and global temporal dependencies across different scale subsequences. The TSFBs, implemented with an inverted Mamba structure, focus on the interaction between dynamic temporal dependencies and spatial characteristics. The primary advantage of miMamba lies in its ability to leverage transformed multi-scale EEG sequences, exploiting the interaction between temporal and spatial features without the need for domain-specific time-frequency feature extraction. Experiments show that using only four EEG channels, miMamba achieves remarkable average recognition accuracies for Valence and Arousal classification: 94.86% on the DEAP dataset, 94.94% on the DREAMER dataset, and 91.36% on the SEED dataset. These results underscore the model's superior performance in multidimensional emotion recognition tasks and its potential for practical applications in resource-constrained affective computing scenarios.
6 January 2025 in IEEE Robotics and Automation Letters
Surface electromyography (sEMG) has demonstrated significant potential in simultaneous and proportional control (SPC). However, existing algorithms for predicting joint angles based on sEMG often suffer from high inference costs or are limited to specific subjects rather than multi-subject scenarios. To address these challenges, we introduced a hierarchical Spiking Attentional Feature Decomposition Network (SAFE-Net). This network initially compresses sEMG signals into neural spiking forms using a Spiking Sparse Attention Encoder (SSAE). Subsequently, the compressed features are decomposed into kinematic and biological features through a Spiking Attentional Feature Decomposition (SAFD) module. Finally, the kinematic and biological features are used to predict joint angles and identify subject identities, respectively. Our validation on two datasets and comparison with two existing methods, Informer and Spikformer, demonstrate that SSAE achieves significant power consumption savings of 39.1% and 37.5% respectively over them in terms of inference costs. Furthermore, SAFE-Net surpasses Informer and Spikformer in recognition accuracy on both datasets. This study underscores the potential of SAFE-Net to advance the field of SPC in lower limb rehabilitation exoskeleton robots.
4 November 2022 in Robotica
Recently, deep learning methods have achieved considerable performance in gesture recognition using surface electromyography signals. However, improving the recognition accuracy in multi-subject gesture recognition remains a challenging problem. In this study, we aimed to improve recognition performance by adding subject-specific prior knowledge to provide guidance for multi-subject gesture recognition. We proposed a time–frequency feature transform suite (TFFT) that takes the maps generated by continuous wavelet transform (CWT) as input. The TFFT can be connected to a neural network to obtain an end-to-end architecture. Thus, we integrated the suite into traditional neural networks, such as convolutional neural networks and long short-term memory, to adjust the intermediate features. The results of comparative experiments showed that the deep learning models with the TFFT suite based on CWT improved the recognition performance of the original architectures without the TFFT suite in gesture recognition tasks. Our proposed TFFT suite has promising applications in multi-subject gesture recognition and prosthetic control.