16 December 2022 in IEEE Transactions on Neural Systems and Rehabilitation Engineering
Due to the limited perceptual field, convolutional neural networks (CNN) only extract local temporal features and may fail to capture long-term dependencies for EEG decoding. In this paper, we propose a compact Convolutional Transformer, named EEG Conformer, to encapsulate local and global features in a unified EEG classification framework. Specifically, the convolution module learns the low-level local features throughout the one-dimensional temporal and spatial convolution layers. The self-attention module is straightforwardly connected to extract the global correlation within the local temporal features. Subsequently, the simple classifier module based on fully-connected layers is followed to predict the categories for EEG signals. To enhance interpretability, we also devise a visualization strategy to project the class activation mapping onto the brain topography. Finally, we have conducted extensive experiments to evaluate our method on three public datasets in EEG-based motor imagery and emotion recognition paradigms. The experimental results show that our method achieves state-of-the-art performance and has great potential to be a new baseline for general EEG decoding. The code has been released in https://github.com/eeyhsong/EEG-Conformer.
12 June 2023 in IEEE Transactions on Neural Systems and Rehabilitation Engineering
Due to the individual difference, EEG signals from other subjects (source) can hardly be used to decode the mental intentions of the target subject. Although transfer learning methods have shown promising results, they still suffer from poor feature representation or neglect long-range dependencies. In light of these limitations, we propose Global Adaptive Transformer (GAT), an domain adaptation method to utilize source data for cross-subject enhancement. Our method uses parallel convolution to capture temporal and spatial features first. Then, we employ a novel attention-based adaptor that implicitly transfers source features to the target domain, emphasizing the global correlation of EEG features. We also use a discriminator to explicitly drive the reduction of marginal distribution discrepancy by learning against the feature extractor and the adaptor. Besides, an adaptive center loss is designed to align the conditional distribution. With the aligned source and target features, a classifier can be optimized to decode EEG signals. Experiments on two widely used EEG datasets demonstrate that our method outperforms state-of-the-art methods, primarily due to the effectiveness of the adaptor. These results indicate that GAT has good potential to enhance the practicality of BCI.
9 October 2024 in Expert Systems with Applications
17 March 2021 in Journal of Neural Engineering
Objective . Motor imagery electroencephalography (EEG) decoding is a vital technology for the brain–computer interface (BCI) systems and has been widely studied in recent years. However, the original EEG signals usually contain a lot of class-independent information, and the existing motor imagery EEG decoding methods are easily interfered by this irrelevant information, which greatly limits the decoding accuracy of these methods. Approach . To overcome the interference of the class-independent information, a motor imagery EEG decoding method based on feature separation is proposed in this paper. Furthermore, a feature separation network based on adversarial learning (FSNAL) is designed for the feature separation of the original EEG samples. First, the class-related features and class-independent features are separated by the proposed FSNAL framework, and then motor imagery EEG decoding is performed only according to the class-related features to avoid the adverse effects of class-independent features. Main results . To validate the effectiveness of the proposed motor imagery EEG decoding method, we conduct some experiments on two public EEG datasets (the BCI competition IV 2a and 2b datasets). The experimental results comparison between our method and some state-of-the-art methods demonstrates that our motor imagery EEG decoding method outperforms all the compared methods on the two experimental datasets. Significance . Our motor imagery EEG decoding method can alleviate the interference of class-independent features, and it has great application potential for improving the performance of motor imagery BCI systems in the near future.
26 May 2025 in IEEE Transactions on Neural Networks and Learning Systems
Electroencephalography (EEG), known for its convenient noninvasive acquisition but moderate signal-to-noise ratio, has recently gained much attention due to the potential to decode image information. However, previous works have not delivered sufficient evidence of this task, primarily limited by performance and biological plausibility. In this work, we first introduce a self-supervised framework to demonstrate the feasibility of recognizing images from EEG signals. Contrastive learning is leveraged to align the representations of EEG responses with image stimuli. Then, language descriptions of the stimuli generated by large language models (LLMs) help guide learning core semantic information. With the framework, we attain significantly above-chance results on the THINGS-EEG2 dataset, achieving a top-1 accuracy of 19.7% and a top-5 accuracy of 51.5% in challenging 200-way zero-shot tasks. Furthermore, we conduct thorough experiments to resolve the human visual responses with EEG from temporal, spatial, spectral, and semantic perspectives. These results provide evidence of feasibility and plausibility regarding EEG-based image recognition, substantiated by comparative studies with the THINGS-Magnetoencephalography (MEG) dataset. The findings offer valuable insights for neural decoding and real-world applications of brain-computer interfaces (BCIs), such as health care and robot control. The code is available at https://github.com/eeyhsong/NICE-LLM.
1 August 2021 in Journal of Neural Engineering
Objective. The original motor imagery electroencephalography (MI-EEG) data contains not only temporal features but also a large number of spatial features related to the distribution of electrodes on the brain. However, in the process of MI-EEG decoding, most of the current convolutional neural network (CNN) based methods do not make the utmost of the spatial features related to electrode distribution. Approach. In this study, we adopt a concise 3D representation for the MI-EEG data to take full advantage of the spatial features and propose a two-branch 3D CNN (TB-3D CNN) for the 3D representation of MI-EEG data. First, the spatial and temporal features of the input 3D samples are extracted by the spatial and temporal feature learning branches, respectively, to avoid the mutual interference between the temporal and spatial features. Then, the central loss is introduced into the TB-3D CNN framework to further improve the MI-EEG decoding accuracy. And a 3D data augmentation method based on the cyclic translation of time dimension is proposed for the 3D representation method to alleviate the overfitting problem. Main results. Some experiments are conducted on the famous BCI competition IV 2a dataset to evaluate the effectiveness of the proposed MI-EEG decoding method. The experimental results comparison with some state-of-the-art methods demonstrates that the average accuracy of our method is 4.42% higher than that of the best of the comparative methods. Significance. The proposed MI-EEG decoding method has great promise to improve the performance of motor imagery brain-computer interface system.