This manuscript presents a hybrid study of a comprehensive review and a systematic (research) analysis. Myoelectric control is the cornerstone of many assistive technologies used in clinical practice, such as prosthetics and orthoses, and human-computer interaction, such as virtual reality control. Although the classification accuracy of such devices exceeds 90% in a controlled laboratory setting, myoelectric devices still face challenges in robustness to variability of daily living conditions. The intrinsic physiological mechanisms limiting practical implementations of myoelectric devices were explored: the limb position effect and the contraction intensity effect. The degradation of electromyography (EMG) pattern recognition in the presence of these factors was demonstrated on six datasets, where classification performance was 13% and 20% lower than the controlled setting for the limb position and contraction intensity effect, respectively. The experimental designs of limb position and contraction intensity literature were surveyed. Current state-of-the-art training strategies and robust algorithms for both effects were compiled and presented. Recommendations for future limb position effect studies include: the collection protocol providing exemplars of at least 6 positions (four limb positions and three forearm orientations), three-dimensional space experimental designs, transfer learning approaches, and multi-modal sensor configurations. Recommendations for future contraction intensity effect studies include: the collection of dynamic contractions, nonlinear complexity features, and proportional control.

Existing research on myoelectric control systems primarily focuses on extracting discriminative characteristics of the electromyographic (EMG) signal by designing handcrafted features. Recently, however, deep learning techniques have been applied to the challenging task of EMG-based gesture recognition. The adoption of these techniques slowly shifts the focus from feature engineering to feature learning. Nevertheless, the black-box nature of deep learning makes it hard to understand the type of information learned by the network and how it relates to handcrafted features. Additionally, due to the high variability in EMG recordings between participants, deep features tend to generalize poorly across subjects using standard training methods. Consequently, this work introduces a new multi-domain learning algorithm, named ADANN (Adaptive Domain Adversarial Neural Network), which significantly enhances (p = 0.00004) inter-subject classification accuracy by an average of 19.40% compared to standard training. Using ADANN-generated features, this work provides the first topological data analysis of EMG-based gesture recognition for the characterization of the information encoded within a deep network, using handcrafted features as landmarks. This analysis reveals that handcrafted features and the learned features (in the earlier layers) both try to discriminate between all gestures, but do not encode the same information to do so. In the later layers, the learned features are inclined to instead adopt a one-vs.-all strategy for a given class. Furthermore, by using convolutional network visualization techniques, it is revealed that learned features actually tend to ignore the most activated channel during contraction, which is in stark contrast with the prevalence of handcrafted features designed to capture amplitude information. Overall, this work paves the way for hybrid feature sets by providing a clear guideline of complementary information encoded within learned and handcrafted features.

In this work, we present a hardware-software solution to improve the robustness of hand gesture recognition to confounding factors in myoelectric control. The solution includes a novel, full-circumference, flexible, 64-channel high-density electromyography (HD-EMG) sensor called EMaGer. The stretchable, wearable sensor adapts to different forearm sizes while maintaining uniform electrode density around the limb. Leveraging this uniformity, we propose novel array barrel-shifting data augmentation (ABSDA) approach used with a convolutional neural network (CNN), and an anti-aliased CNN (AA-CNN), that provides shift invariance around the limb for improved classification robustness to electrode movement, forearm orientation, and inter-session variability. Signals are sampled from a 4x16 HD-EMG array of electrodes at a frequency of 1 kHz and 16-bit resolution. Using data from 12 able-bodied participants, the approach is tested in response to sensor rotation, forearm rotation, and inter-session scenarios. The proposed ABSDA-CNN method improves inter-session accuracy by 25.67% on average across users for 6 gesture classes compared to conventional CNN classification. A comparison with other devices shows that this benefit is enabled by the unique design of the EMaGer array. The AA-CNN yields improvements of up to 63.05% accuracy over non-augmented methods when tested with electrode displacements ranging from -45 $^\circ$ to +45 $^\circ$ around the limb. Overall, this paper demonstrates the benefits of co-designing sensor systems, processing methods, and inference algorithms to leverage synergistic and interdependent properties to solve state-of-the-art problems.

Myoelectric control has been used predominantly in the field of prosthetics, but is an increasingly promising hands-free input modality for emerging consumer markets such as mixed reality. Developing robust machine learning-enabled EMG control systems, however, has historically required substantial domain expertise. This has presented a significant barrier to entry for researchers, impeded progress in EMG-based interaction design, and contributed to the perception that such systems lack the robustness and intuitiveness required for real-world use. To overcome these challenges, we present LibEMG, an open-source Python library for performing offline EMG analyses and developing online EMG-based interactions. By abstracting the challenges and nuances surrounding myoelectric control, including hardware interfacing, data acquisition, feature extraction/selection, classification, post-processing, and evaluation, we eliminate many of the significant barriers limiting the exploration of this technology. Combining expertise from the prosthetics and human-computer interaction communities into a shared library, extensive examples, and documentation, we provide researchers with an accessible tool to accelerate research and improve reproducibility in myoelectric control. In doing so, we aim to facilitate the exploration of this technology, particularly outside prosthesis control, to unlock its potential as a widely applicable hands-free input modality.

Discrete myoelectric control-based gesture recognition has recently gained interest as a possible input modality for many emerging ubiquitous computing applications. Unlike the continuous control commonly employed in powered prostheses, discrete systems seek to recognize the dynamic sequences associated with gestures to generate event-based inputs. More akin to those used in general-purpose human-computer interaction, these could include, for example, a flick of the wrist to dismiss a phone call or a double tap of the index finger and thumb to silence an alarm. Moelectric control systems have been shown to achieve near-perfect classification accuracy, but in highly constrained offline settings. Real-world, online systems are subject to ‘confounding factors’ (i.e. factors that hinder the real-world robustness of myoelectric control that are not accounted for during typical offline analyses), which inevitably degrade system performance, limiting their practical use. Although these factors have been widely studied in continuous prosthesis control, there has been little exploration of their impacts on discrete myoelectric control systems for emerging applications and use cases. Correspondingly, this work examines, for the first time, three confounding factors and their effect on the robustness of discrete myoelectric control: (1) limb position variability, (2) cross-day use, and a newly identified confound faced by discrete systems (3) gesture elicitation speed. Results from four different discrete myoelectric control architectures: (1) Majority Vote LDA, (2) Dynamic Time Warping, (3) an LSTM network trained with Cross Entropy, and (4) an LSTM network trained with Contrastive Learning, show that classification accuracy is significantly degraded ( p < 0.05) as a result of each of these confounds. This work establishes that confounding factors are a critical barrier that must be addressed to enable the real-world adoption of discrete myoelectric control for robust and reliable gesture recognition.

In the field of EMG-based force modeling, the ability to generalize models across individuals could play a significant role in its adoption across a range of applications, including assistive devices, robotic and rehabilitation devices. However, current studies have predominately focused on intra-subject modeling, largely neglecting the burden of end-user data acquisition. In this work, we propose the use of transfer learning (TL) to generalize force modeling to a new user by first establishing a baseline model trained using other users’ data, and then adapting to the end-user using a small amount of new data (only ${10}\%$ , ${20}\%$ , and ${40}\%$ of the new user data). Using a deep multimodal convolutional neural network, consisting of two CNN models, one with high-density (HD) EMG and one with motion data recorded by an Inertial Measurement Unit (IMU), our proposed TL technique significantly improved force modeling compared to leave-one-subject-out (LOSO) and even intra-subject scenarios. The TL approach increased the average R squared values of the force modeling task by 60.81%, 190.53%, and 199.79% compared to the LOSO case, and by 13.4%, 36.88%, and 45.51% compared to the intra-subject case for isotonic, isokinetic and dynamic conditions, respectively. These results show that it is possible to adapt to a new user with minimal data while improving performance significantly compared to the intra-subject scenario. We also show that TL can be used to generalize on a new experimental condition for a new user.