Convolutional neural network (CNN) has been widely exploited for simultaneous and proportional myoelectric control due to its capability of deriving informative, representative and transferable features from surface electromyography (sEMG). However, muscle contractions have strong temporal dependencies but conventional CNN can only exploit spatial correlations. Considering that long short-term memory neural network (LSTM) is able to capture long-term and non-linear dynamics of time-series data, in this paper we propose a CNN-LSTM hybrid model to fully explore the temporal-spatial information in sEMG. Firstly, CNN is utilized to extract deep features from sEMG spectrum, then these features are processed via LSTM-based sequence regression to estimate wrist kinematics. Six healthy participants are recruited for the participatory collection and motion analysis under various experimental setups. Estimation results in both intra-session and inter-session evaluations illustrate that CNN-LSTM significantly outperforms CNN, LSTM and several representative machine learning approaches, particularly when complex wrist movements are activated.

To develop multi-functionalhuman-machine interfaces that can help disabled people reconstruct lost functions of upper-limbs, machine learning (ML) and deep learning (DL) techniques have been widely implemented to decode human movement intentions from surface electromyography (sEMG) signals. However, due to the high complexity of upper-limb movements and the inherent non-stable characteristics of sEMG, the usability of ML/DL based control schemes is still greatly limited in practical scenarios. To this end, tremendous efforts have been made to improve model robustness, adaptation, and reliability. In this article, we provide a systematic review on recent achievements, mainly from three categories: multi-modal sensing fusion to gain additional information of the user, transfer learning (TL) methods to eliminate domain shift impacts on estimation models, and post-processing approaches to obtain more reliable outcomes. Special attention is given to fusion strategies, deep TL frameworks, and confidence estimation. Research challenges and emerging opportunities, with respect to hardware development, public resources, and decoding strategies, are also analysed to provide perspectives for future developments

Recently, convolutional neural network (CNN) has been widely investigated to decode human intentions using surface Electromyography (sEMG) signals. However, a pre-trained CNN model usually suffers from severe degradation when testing on a new individual, and this is mainly due to domain shift where characteristics of training and testing sEMG data differ substantially. To enhance inter-subject performances of CNN in the wrist kinematics estimation, we propose a novel regression scheme for supervised domain adaptation (SDA), based on which domain shift effects can be effectively reduced. Specifically, a two-stream CNN with shared weights is established to exploit source and target sEMG data simultaneously, such that domain-invariant features can be extracted. To tune CNN weights, both regression losses and a domain discrepancy loss are employed, where the former enable supervised learning and the latter minimizes distribution divergences between two domains. In this study, eight healthy subjects were recruited to perform wrist flexion-extension movements. Experiment results illustrated that the proposed regression SDA outperformed fine-tuning, a state-of-the-art transfer learning method, in both single-single and multiple-single scenarios of kinematics estimation. Unlike fine-tuning which suffers from catastrophic forgetting, regression SDA can maintain much better performances in original domains, which boosts the model reusability among multiple subjects.

Data-driven methods have become increasingly more prominent for musculoskeletal modelling due to their conceptually intuitive simple and fast implementation. However, the performance of a pre-trained data-driven model using the data from specific subject(s) may be seriously degraded when validated using the data from a new subject, hindering the utility of the personalised musculoskeletal model in clinical applications. This paper develops an active physics-informed deep transfer learning framework to enhance the dynamic tracking capability of the musculoskeletal model on the unseen data. The salient advantages of the proposed framework are twofold: 1) For the generic model, physics-based domain knowledge is embedded into the loss function of the data-driven model as soft constraints to penalise/regularise the data-driven model. 2) For the personalised model, the parameters relating to the feature extraction will be directly inherited from the generic model, and only the parameters relating to the subject-specific inference will be fine-tuned by jointly minimising the conventional data prediction loss and the modified physics-based loss. In this paper, we use the synchronous muscle forces and joint kinematics prediction from surface electromyogram (sEMG) as the exemplar to illustrate the proposed framework. Moreover, convolutional neural network (CNN) is employed as the deep neural network to implement the proposed framework, and the physics law between muscle forces and joint kinematics is utilised as the soft constraints. Results of comprehensive experiments on a self-collected dataset from eight healthy subjects indicate the effectiveness and great generalization of the proposed framework.

Although deep learning (DL) techniques have been extensively researched in upper-limb myoelectric control, system robustness in cross-day applications is still very limited. This is largely caused by non-stable and time-varying properties of surface electromyography (sEMG) signals, resulting in domain shift impacts on DL models. To this end, a reconstruction-based method is proposed for domain shift quantification. Herein, a prevalent hybrid framework that combines a convolutional neural network (CNN) and a long short-term memory network (LSTM), i.e. CNN-LSTM, is selected as the backbone. The paring of auto-encoder (AE) and LSTM, abbreviated as LSTM-AE, is proposed to reconstruct CNN features. Based on reconstruction errors (RErrors) of LSTM-AE, domain shift impacts on CNN-LSTM can be quantified. For a thorough investigation, experiments were conducted in both hand gesture classification and wrist kinematics regression, where sEMG data were both collected in multi-days. Experiment results illustrate that, when the estimation accuracy degrades substantially in between-day testing sets, RErrors increase accordingly and can be distinct from those obtained in within-day datasets. According to data analysis, CNN-LSTM classification/regression outcomes are strongly associated with LSTM-AE errors. The average Pearson correlation coefficients could reach -0.986 +/- 0.014 and -0.992 +/- 0.011, respectively.

Convolutional neural networks (CNNs) have been widely utilized to identify hand gestures from surface electromyography (sEMG) signals. However, due to the nonstationary characteristics of sEMG, the classification accuracy usually degrades significantly in the daily living environment involving complex hand movements. To further improve the reliability of a classifier, unconfident classifications are expected to be identified and rejected. In this study, we propose a novel approach to estimate the probability of correctness for each classification. Specifically, a confidence estimation model is established to generate confidence scores (ConfScore) based on posterior probabilities of CNN, and an objective function is designed to train the parameters of this model. In addition, a comprehensive metric that combines the true acceptance rate (TAR) and the true rejection rate (TRR) is proposed to evaluate the rejection performance of ConfScore, so that the tradeoff between system security and control lag could be fully considered. The effectiveness of ConfScore is verified using data from public databases and our online platform. The experimental results illustrate that ConfScore can better reflect the correctness of CNN classifications than traditional confidence features, i.e., maximum posterior probability and entropy of the probability vector. Moreover, the rejection performance is observed to be less sensitive to variations in rejection thresholds.