Successful applications of brain-computer interface (BCI) approaches to motor imagery (MI) are still limited. In this paper, we propose a classification framework for MI electroencephalogram (EEG) signals that combines a convolutional neural network (CNN) architecture with a variational autoencoder (VAE) for classification. The decoder of the VAE generates a Gaussian distribution, so it can be used to fit the Gaussian distribution of EEG signals. A new representation of input was developed by combining the time, frequency, and channel information from the EEG signal, and the CNN-VAE method was designed and optimized accordingly for this form of input. In this network, the classification of the extracted CNN features is performed via the deep network VAE. Our framework, with an average kappa value of 0.564, outperforms the best classification method in the literature for BCI Competition IV dataset 2b with a 3% improvement. Furthermore, using our own dataset, the CNN-VAE framework also yields the best performance for both three-electrode and five-electrode EEGs and achieves the best average kappa values 0.568 and 0.603, respectively. Our results show that the proposed CNN-VAE method raises performance to the current state of the art.
10 April 2025 in IEEE Transactions on Neural Networks and Learning Systems
Integrating prior knowledge of neurophysiology into neural network architecture enhances the performance of emotion decoding. While numerous techniques emphasize learning spatial and short-term temporal patterns, there has been a limited emphasis on capturing the vital long-term contextual information associated with emotional cognitive processes. In order to address this discrepancy, we introduce a novel transformer model called emotion transformer (EmT). EmT is designed to excel in both generalized cross-subject electroencephalography (EEG) emotion classification and regression tasks. In EmT, EEG signals are transformed into a temporal graph format, creating a sequence of EEG feature graphs using a temporal graph construction (TGC) module. A novel residual multiview pyramid graph convolutional neural network (RMPG) module is then proposed to learn dynamic graph representations for each EEG feature graph within the series, and the learned representations of each graph are fused into one token. Furthermore, we design a temporal contextual transformer (TCT) module with two types of token mixers to learn the temporal contextual information. Finally, the task-specific output (TSO) module generates the desired outputs. Experiments on four publicly available datasets show that EmT achieves higher results than the baseline methods for both EEG emotion classification and regression tasks. The code is available at https://github.com/yi-ding-cs/EmT.
26 September 2022 in IEEE Transactions on Neural Systems and Rehabilitation Engineering
Although brain-computer interface (BCI) shows promising prospects to help post-stroke patients recover their motor function, its decoding accuracy is still highly dependent on feature extraction methods. Most current feature extractors in BCI are classification-based methods, yet very few works from literature use metric learning based methods to learn representations for BCI. To circumvent this shortage, we propose a deep metric learning based method, Weighted Convolutional Siamese Network (WCSN) to learn representations from electroencephalogram (EEG) signal. This approach can enhance the decoding accuracy by learning a low dimensional embedding to extract distance-based representations from pair-wise EEG data. To enhance training efficiency and algorithm performance, a temporal-spectral distance weighted sampling method is proposed to select more informative input samples. In addition, an adaptive training strategy is adopted to address the session-to-session non-stationarity by progressively updating the subject-specific model. The proposed method is applied on both upper limb and lower limb neurorehabilitation datasets acquired from 33 stroke patients, with a total of 358 sessions. Results indicate that using k-Nearest Neighbor as the classification algorithm, the proposed method yielded 72.8% and 66.0% accuracies for the two datasets respectively, significantly better than the other state-of-the-arts ( ${p} < {0.05}$ ). Without losing generality, we also evaluated the proposed method on two publicly available datasets acquired from healthy subjects, wherein the proposed algorithm demonstrated superior performance at most cases as well. Our results support, for the first time, the use of a metric learning based feature extractor to learn representations from non-stationary EEG signals for BCI-assisted post-stroke rehabilitation.
21 September 2021 in IEEE Internet of Things Journal
The brain–computer interface (BCI) technology improves the communication efficiency between people and Internet of Things (IoT) devices. BCI based on the steady-state visual evoked potential (SSVEP-BCI) is the preferred scheme for controlling devices because of its convenient operation, low training requirement, and high information transmission rate (ITR). Most signal acquisition devices for BCIs are used for medical diagnosis and scientific research and utilize multiple channels and wet electrodes to obtain high-quality signals. However, the practicability, wearability, and cost of the signal acquisition devices for real-life applications need to be considered, resulting in new requirements for the acquisition mode, the number of electrodes, power consumption, and signal processing methods. This article presents a wearable low-power collaborative sensing system based on a time mask window canonical correlation analysis method (TMW-CCA). An 8-array spring dry electrode signal acquisition device based on a flexible circuit board is designed to address the shortcomings of traditional wet electrode acquisition devices, such as high-power consumption, discomfort, and being unsuitable for long-time use. The proposed TMW-CCA method, which uses a dry electrode sensor to evaluate the time domain’s signal quality dynamically, exhibits 12.5% higher steady-state visual evoked potential recognition accuracy and 40% lower average power consumption (only 740 mW) than the benchmark.
Covert speech involves imagining speaking without audible sound or any movements. Decoding covert speech from electroencephalogram (EEG) is challenging due to a limited understanding of neural pronunciation mapping and the low signal-to-noise ratio of the signal. In this study, we developed a large-scale multi-utterance speech EEG dataset from 57 right-handed native English-speaking subjects, each performing covert and overt speech tasks by repeating the same word in five utterances within a ten-second duration. Given the spatio-temporal nature of the neural activation process during speech pronunciation, we developed a Functional Areas Spatio-temporal Transformer (FAST), an effective framework for converting EEG signals into tokens and utilizing transformer architecture for sequence encoding. Our results reveal distinct and interpretable speech neural features by the visualization of FAST-generated activation maps across frontal and temporal brain regions with each word being covertly spoken, providing new insights into the discriminative features of the neural representation of covert speech. This is the first report of such a study, which provides interpretable evidence for speech decoding from EEG. The code for this work has been made public at https://github.com/Jiang-Muyun/FAST