Noninvasive neuroimaging and increased user engagement improve EEG-based neural decoding and facilitate real-time 2D robotic device control.

A decade ago, a group of researchers from academia and industry identified a dichotomy between the industrial and academic state-of-the-art in upper-limb prosthesis control, a widely used bio-robotics application. They proposed that four key technical challenges, if addressed, could bridge this gap and translate academic research into clinically and commercially viable products. These challenges are unintuitive control schemes, lack of sensory feedback, poor robustness and single sensor modality. Here, we provide a perspective review on the research effort that occurred in the last decade, aiming at addressing these challenges. In addition, we discuss three research areas essential to the recent development in upper-limb prosthetic control research but were not envisioned in the review 10 years ago: deep learning methods, surface electromyogram decomposition and open-source databases. To conclude the review, we provide an outlook into the near future of the research and development in upper-limb prosthetic control and beyond.

Brain-computer interface (BCI) provides a novel technology for patients and healthy human subjects to control a robotic arm. Currently, BCI control of a robotic arm to complete the reaching and grasping tasks in an unstructured environment is still challenging because the current BCI technology does not meet the requirement of manipulating a multi-degree robotic arm accurately and robustly. BCI based on steady-state visual evoked potential (SSVEP) could output a high information transfer rate; however, the conventional SSVEP paradigm failed to control a robotic arm to move continuously and accurately because the users have to switch their gaze between the flickering stimuli and the target frequently. This study proposed a novel SSVEP paradigm in which the flickering stimuli were attached to the robotic arm's gripper and moved with it. First, an offline experiment was designed to investigate the effects of moving flickering stimuli on the SSVEP's responses and decoding accuracy. After that, contrast experiments were conducted, and twelve subjects were recruited to participate in a robotic arm control experiment using both the paradigm one (P1, with moving flickering stimuli) and the paradigm two (P2, conventional fixed flickering stimuli) using a block randomization design to balance their sequences. Double blinks were used to trigger the grasping action asynchronously whenever the subjects were confident that the position of the robotic arm's gripper was accurate enough. Experimental results showed that the paradigm P1 with moving flickering stimuli provided a much better control performance than the conventional paradigm P2 in completing a reaching and grasping task in an unstructured environment. Subjects' subjective feedback scored by a NASA-TLX mental workload scale also corroborated the BCI control performance. The results of this study suggest that the proposed control interface based on SSVEP BCI provides a better solution for robotic arm control to complete the accurate reaching and grasping tasks.

Ultrasound can non-invasively detect muscle deformations and has great potential applications in prosthetic hand control. Traditional ultrasound equipment was usually too bulky to be applied in wearable scenarios. This research presented a compact ultrasound device that could be integrated into a prosthetic hand socket. The miniaturized ultrasound system included four A-mode ultrasound transducers for sensing musculature deformations, a signal excitation/acquisition module, and a prosthetic hand control module. The size of the ultrasound system was 65*75*25mm, weighing only 85g. For the first time, we integrated the ultrasound system into a prosthetic hand socket to evaluate its performance in practical prosthetic hand control. We designed an experiment requiring twenty subjects to perform six commonly used gestures. The performance of decoding ultrasound signals was analyzed offline using four classification algorithms and then was assessed in online control. The average values of online classification accuracy with and without wearing the physical prosthetic were 91.5 $\pm 6.4\%$ and 96.5 $\pm 3.0\%$ , respectively. We found that wearing the prosthetic hand influenced the ultrasound gestures classification accuracy, but remarkable online classification performance could still be maintained. These experimental results demonstrated the efficacy of the designed integrated ultrasound system for practical use, paving the way for an effective HMI system that could be widely used in prosthetic hand control.

Objective: While noninvasive electroenceph-alography (EEG) based brain-computer interfacing (BCI) has been successfully demonstrated in two-dimensional (2-D) control tasks, little work has been published regarding its extension to practical three-dimensional (3-D) control. Methods: In this study, we developed a new BCI approach for 3-D control by combining a novel form of endogenous visuospatial attentional modulation, defined as overt spatial attention (OSA), and motor imagery (MI). Results: OSA modulation was shown to provide comparable control to conventional MI modulation in both 1-D and 2-D tasks. Furthermore, this paper provides evidence for the functional independence of traditional MI and OSA, as well as an investigation into the simultaneous use of both. Using this newly proposed BCI paradigm, 16 participants successfully completed a 3-D eight-target control task. Nine of these subjects further demonstrated robust 3-D control in a 12-target task, significantly outperforming the information transfer rate achieved in the 1-D and 2-D control tasks (29.7 ± 1.6 b/min). Conclusion: These results strongly support the hypothesis that noninvasive EEG-based BCI can provide robust 3-D control through endogenous neural modulation in broader populations with limited training. Significance: Through the combination of the two strategies (MI and OSA), a substantial portion of the recruited subjects were capable of robustly controlling a virtual cursor in 3-D space. The proposed novel approach could broaden the dimensionality of BCI control and shorten the training time.

Objective: BCI decoding accuracy plays a crucial role in practical applications. With accurate feedback, BCI-based therapy determines beneficial neural plasticity in stroke patients. In this study, we aimed at improving sensorimotor rhythm (SMR)-based BCI performance by integrating motor tasks with tactile stimulation. Methods: Eleven stroke patients were recruited for three experimental conditions, i.e., motor attempt (MA) condition, tactile stimulation (TS) condition, and tactile stimulation-assisted motor attempt (TS-MA) condition. Tactile stimulation was delivered to the paretic hand wrist during both task and idle states using a DC vibrator. Results: We observed that the TS-MA condition achieved greater motor-related cortical activation (MRCA) in alpha-beta band when compared with both TS and MA conditions. Consequently, online BCI decoding accuracies between task and idle states were significantly improved from 74.5% in the MA condition to 85.1% in the TS-MA condition (p < 0.001), whereas the accuracy in the TS condition was 54.6% (approaching to the chance level of 50%). Conclusion: This finding demonstrates that sensory afferent from peripheral nerves benefits the neural process of sensorimotor cortex in stroke patients. With appropriate sensory stimulation, MRCA is enhanced and corresponding brain patterns are more discriminative. Significance: This novel SMR-BCI paradigm shows great promise to facilitate the practical application of BCI-based stroke rehabilitation.