Reinforcement learning (RL)-based brain machine interfaces (BMIs) enable the user to learn from the environment through interactions to complete the task without desired signals, which is promising for clinical applications. Previous studies exploited Q-learning techniques to discriminate neural states into simple directional actions providing the trial initial timing. However, the movements in BMI applications can be quite complicated, and the action timing explicitly shows the intention when to move. The rich actions and the corresponding neural states form a large state-action space, imposing generalization difficulty on Q-learning. In this paper, we propose to adopt attention-gated reinforcement learning (AGREL) as a new learning scheme for BMIs to adaptively decode high-dimensional neural activities into seven distinct movements (directional moves, holdings and resting) due to the efficient weight-updating. We apply AGREL on neural data recorded from M1 of a monkey to directly predict a seven-action set in a time sequence to reconstruct the trajectory of a center-out task. Compared to Q-learning techniques, AGREL could improve the target acquisition rate to 90.16% in average with faster convergence and more stability to follow neural activity over multiple days, indicating the potential to achieve better online decoding performance for more complicated BMI tasks.

This article proposes an adaptive control for a flexible manipulator (FM) under the influence of distributed disturbances, unknown dead zones, and input quantization. First, the hybrid effect of the unknown dead zone and input quantization is formulated and represented based on some essential transformations. Then, an adaptive robust quantized control with online updating laws is developed to address the uncertainty of the dead zone, ensure robustness and angle position, and dampen the vibration in the FM system. Subsequently, the Lyapunov theoretical analysis is employed to ensure the bounded stability of the system. Finally, numerical simulations and experiments with a Quanser platform are given to further verify the feasibility and superiority of the designed scheme.

Neuroprosthesis enables the brain control on the external devices purely using neural activity for paralyzed people. Supervised learning decoders recalibrate or re-fit the discrepancy between the desired target and decoder’s output, where the correction may over-dominate the user’s intention. Reinforcement learning decoder allows users to actively adjust their brain patterns through trial and error, which better represents the subject’s motive. The computational challenge is to quickly establish new state-action mapping before the subject becomes frustrated. Recently proposed quantized attention-gated kernel reinforcement learning (QAGKRL) explores the optimal nonlinear neural-action mapping in the Reproducing Kernel Hilbert Space (RKHS). However, considering all past data in RKHS is less efficient and sensitive to detect the new neural patterns emerging in brain control. In this paper, we propose a clustering-based kernel RL algorithm. New neural patterns emerge and are clustered to represent the novel knowledge in brain control. The current neural data only activate the nearest subspace in RKHS for more efficient decoding. The dynamic clustering makes our algorithm more sensitive to new brain patterns. We test our algorithm on both the synthetic and real-world spike data. Compared with QAGKRL, our algorithm can achieve a quicker knowledge adaptation in brain control with less computational complexity.

Active rehabilitation involves patient’s voluntary thoughts as the control signals of restore device to assist stroke rehabilitation. Although restoration of hand opening stands importantly in patient’s daily life, it is difficult to distinguish the voluntary finger extension from thumb adduction and finger flexion using stroke patients’ electroencephalography (EMG) on single muscle activity. We propose to implement corticomuscular coherence analysis on electroencephalography (EEG) and EMG signals on Extensor Digitorum to extract their intention involved in hand opening. EEG and EMG signals of 8 subjects are simultaneously collected when executing 4 hand movement tasks (finger extension, thumb adduction, finger flexion, and rest). We explore the spatial and temporal distribution of the coherence and observe statistically significant corticomuscular coherence appearing at left motor cortical area and different patterns within beta frequency range for 4 movement tasks. Linear discriminate analysis is applied on the coherence pattern to distinguish finger extension from thumb adduction, finger flexion, and rest. The classification results are greater than those by EEG only. The results indicate the possibility to detect voluntary hand opening based on coherence analysis between single muscle EMG signal and single EEG channel located in motor cortical area, which potentially helps active hand rehabilitation for stroke patients.