Objective: Computer-aided estimation of brain connectivity aims to reveal information propagation in brain automatically, which has great potential in clinical applications, e.g., epilepsy foci diagnosis. Granger causality is an effective tool for directional connection analysis in multivariate time series. However, most existing methods based on Granger causality assume fixed time lags in information transmission, while the propagation delay between brain signals is usually changing constantly. Methods: We propose a Granger causality estimator based on the recurrent neural network, called RNN-GC, to deal with the multivariate brain connectivity detection problem. Our model takes input of time-series signals with arbitrary length of transmission time lags and learns the information flow from the data using the gated RNN model, i.e., long short-term memory (LSTM). The LSTM model can sequentially update the gates in memory cells to determine how many preceding points should be considered for prediction. Therefore, the LSTM-based RNN-GC estimator works well on varying-length time lags and shows effectiveness even on very long transmission delays. Results: Experiments are carried out in comparison with other methods using both simulation data and epileptic electroencephalography signals. The RNN-GC estimator achieves superior performance in brain connectivity estimation and shows robustness in modeling multivariate connections with varying-length time lags. Conclusion: The RNN-GC method is capable of modeling nonlinear and varying-length lagged information transmission and effective in directional brain connectivity estimation. Significance: The proposed method is promising to serve as a robust brain connection analysis tool in clinical applications.

Brain-computer interface (BCI) is a direct communication pathway between brain and external devices, and BCI-based prosthetic devices are promising to provide new rehabilitation options for people with motor disabilities. Electrocorticography (ECoG) signals contain rich information correlated with motor activities, and have great potential in hand gesture decoding. However, most existing decoders use long time windows, thus ignore the temporal dynamics within the period. In this study, we propose to use recurrent neural networks (RNNs) to exploit the temporal information in ECoG signals for robust hand gesture decoding. With RNN's high nonlinearity modeling ability, our method can effectively capture the temporal information in ECoG time series for robust gesture recognition. In the experiments, we decode three hand gestures using ECoG signals of two participants, and achieve an accuracy of 90%. Specially, we investigate the possibility of recognizing the gestures in a time interval as short as possible after motion onsets. Our method rapidly recognizes gestures within 0.5 s after motion onsets with an accuracy of about 80%. Experimental results also indicate that the temporal dynamics is especially informative for effective and rapid decoding of hand gestures.

Objective . Brain-computer interface (BCI) has demonstrated its effectiveness in epilepsy treatment and control. In a BCI-aided epilepsy treatment system, therapic electrical stimulus is delivered in response to the prediction of upcoming seizure onsets, therefore timely and accurate seizure prediction algorithm plays an important role. However, unlike typical signatures such as slow or sharp waves in ictal periods, the signal patterns in preictal periods are usually subtle, and highly individual-dependent. How to extract effective and robust preictal features is still a challenging problem. Approach . Most recently, graph convolutional neural network (GCNN) has demonstrated the strength in the electroencephalogram (EEG) and intracranial electroencephalogram (iEEG) signal modeling, due to its advantages in describing complex relationships among different EEG/iEEG regions. However, current GCNN models are not suitable for seizure prediction. The effectiveness of GCNNs highly relies on prior graphs that describe the underlying relationships in EEG regions. However, due to the complex mechanism of seizure evolution, the underlying relationship in the preictal period can be diverse in different patients, making it almost impossible to build a proper prior graph in general. To deal with this problem, we propose a novel approach to automatically learn a patient-specific graph in a data-driven way, which is called the joint graph structure and representation learning network (JGRN). JGRN constructs a global-local graph convolutional neural network which jointly learns the graph structures and connection weights in a task-related learning process in iEEG signals, thus the learned graph and feature representations can be optimized toward the objective of seizure prediction. Main results . Experimental results show that our JGRN outperforms CNN and GCNN models remarkably, and the improvement is more obvious when preictal features are subtle. Significance . The proposed approach promises to achieve robust seizure prediction performance and to have the potential to be extended to general problems in brain-computer interfaces.

Objective: Brain-computer interfaces (BCI) that enables people with severe motor disabilities to use their brain signals for direct control of objects have attracted increased interest in rehabilitation. To date, no study has investigated feasibility of the BCI framework incorporating both intracortical and scalp signals. Methods: Concurrent local field potential (LFP) from the hand-knob area and scalp EEG were recorded in a paraplegic patient undergoing a spike-based close-loop neurorehabilitation training. Based upon multimodal spatio-spectral feature extraction and Naïve Bayes classification, we developed, for the first time, a novel LFP-EEG-BCI for motor intention decoding. A transfer learning (TL) approach was employed to further improve the feasibility. The performance of the proposed LFP-EEG-BCI for four-class upper-limb motor intention decoding was assessed. Results: Using a decision fusion strategy, we showed that the LFP-EEG-BCI significantly (p $< $ 0.05) outperformed single modal BCI (LFP-BCI and EEG-BCI) in terms of decoding accuracy with the best performance achieved using regularized common spatial pattern features. Interrogation of feature characteristics revealed discriminative spatial and spectral patterns, which may lead to new insights for better understanding of brain dynamics during different motor imagery tasks and promote development of efficient decoding algorithms. Moreover, we showed that similar classification performance could be obtained with few training trials, therefore highlighting the efficacy of TL. Conclusion: The present findings demonstrated the superiority of the novel LFP-EEG-BCI in motor intention decoding. Significance: This work introduced a novel LFP-EEG-BCI that may lead to new directions for developing practical neurorehabilitation systems with high detection accuracy and multi-paradigm feasibility in clinical applications.

How the human motor cortex (MC) orchestrates sophisticated sequences of fine movements such as handwriting remains a puzzle. Here we investigate this question through Utah array recordings from human MC during attempted handwriting of Chinese characters (n = 306, each consisting of 6.3 ± 2.0 strokes). We find that MC activity evolves through a sequence of states corresponding to the writing of stroke fragments during complicated handwriting. The directional tuning curve of MC neurons remains stable within states, but its gain or preferred direction strongly varies across states. By building models that can automatically infer the neural states and implement state-dependent directional tuning, we can significantly better explain the firing pattern of individual neurons and reconstruct recognizable handwriting trajectories with 69% improvement compared with baseline models. Our findings unveil that skilled and sophisticated movements are encoded through state-specific neural configurations.

Objective: Brain-machine interfaces (BMIs) aim to provide direct brain control of devices such as prostheses and computer cursors, which have demonstrated great potential for motor restoration. One major limitation of current BMIs lies in the unstable performance due to the variability of neural signals, especially in online control, which seriously hinders the clinical availability of BMIs. Method: We propose a dynamic ensemble Bayesian filter (DyEnsemble) to deal with the neural variability in online BMI control. Unlike most existing approaches using fixed models, DyEnsemble learns a pool of models that contains diverse abilities in describing the neural functions. In each time slot, it dynamically weights and assembles the models according to the neural signals in a Bayesian framework. In this way, DyEnsemble copes with variability in signals and improves the robustness of online control. Results: Online BMI experiments with a human participant demonstrate that, compared with the velocity Kalman filter, DyEnsemble significantly improves the control accuracy (increases the success rate by 13.9% in the random target pursuit task) and robustness (performs more stably over different experiment days). Conclusion: Experimental results demonstrate the superiority of DyEnsemble in online BMI control. Significance: DyEnsemble frames a novel and flexible dynamic decoding framework for robust BMIs, beneficial to various neural decoding applications.