Electroencephalography (EEG) signals classification is essential for the brain-computer interface (BCI). Recently, energy-efficient spiking neural networks (SNNs) have shown great potential in EEG analysis due to their ability to capture the complex dynamic properties of biological neurons while also processing stimulus information through precisely timed spike trains. However, most existing methods do not effectively mine the specific spatial topology of EEG channels and temporal dependencies of the encoded EEG spikes. Moreover, most are designed for specific BCI tasks and lack some generality. Hence, this study presents a novel SNN model with the customized spike-based adaptive graph convolution and long short-term memory (LSTM), termed SGLNet, for EEG-based BCIs. Specifically, we first adopt a learnable spike encoder to convert the raw EEG signals into spike trains. Then, we tailor the concepts of the multi-head adaptive graph convolution to SNN so that it can make good use of the intrinsic spatial topology information among distinct EEG channels. Finally, we design the spike-based LSTM units to further capture the temporal dependencies of the spikes. We evaluate our proposed model on two publicly available datasets from two representative fields of BCI, notably emotion recognition, and motor imagery decoding. The empirical evaluations demonstrate that SGLNet consistently outperforms existing state-of-the-art EEG classification algorithms. This work provides a new perspective for exploring high-performance SNNs for future BCIs with rich spatiotemporal dynamics.

Fatigue driving is a significant cause of road traffic accidents and associated casualties. Automatic assessment of driver drowsiness by monitoring electroencephalography (EEG) signals offer a more objective way to improve driving safety. However, most existing measures are based on multi-channel EEG signals, which are more difficult to apply in practical scenarios as it usually lacks better portability and comfort. In addition, due to the relatively parsimonious and non-stationary characteristics, it is still challenging to effectively accomplish drowsiness recognition by exploiting single-channel EEG signals alone. To this end, we propose a novel temporal-frequential attentional convolutional neural network (TFAC-Net) to take full advantage of spectral-temporal features for single-channel EEG driver drowsiness recognition. Specifically, to capture the potentially valuable information contained in single-channel EEG, the continuous wavelet transform is first employed to generate a corresponding spectral-temporal representation. Then, the temporal-frequential attention mechanism is adopted to reveal critical time-frequency regions in terms of the driver’s mental state. Finally, an adaptive feature fusion module is considered to recalibrate and integrate the most relevant feature channels for final prediction. Extensive experimental results on a widely used public EEG driving dataset demonstrate that the TFAC-Net approach is superior to the state-of-the-art methods, and could discover some discriminative temporal-frequential regions. Moreover, this study also sheds light on the development of portable EEG devices and practical driver drowsiness recognition.

Sleep staging is a vital process for evaluating sleep quality and diagnosing sleep-related diseases. Most of the existing automatic sleep staging methods focus on time-domain information and often ignore the transformation relationship between sleep stages. To deal with the above problems, we propose a Temporal-Spectral fused and Attention-based deep neural Network model (TSA-Net) for automatic sleep staging, using a single-channel electroencephalogram (EEG) signal. The TSA-Net is composed of a two-stream feature extractor, feature context learning, and conditional random field (CRF). Specifically, the two-stream feature extractor module can automatically extract and fuse EEG features from time and frequency domains, considering that both temporal and spectral features can provide abundant distinguishing information for sleep staging. Subsequently, the feature context learning module learns the dependencies between features using the multi-head self-attention mechanism and outputs a preliminary sleep stage. Finally, the CRF module further applies transition rules to improve classification performance. We evaluate our model on two public datasets, Sleep-EDF-20 and Sleep-EDF-78. In terms of accuracy, the TSA-Net achieves 86.64% and 82.21% on the Fpz-Cz channel, respectively. The experimental results illustrate that our TSA-Net can optimize the performance of sleep staging and achieve better staging performance than state-of-the-art methods.

Emotion recognition based on electroencephalography (EEG) has attracted significant attention and achieved considerable advances in the fields of affective computing and human-computer interaction. However, most existing studies ignore the coupling and complementarity of complex spatiotemporal patterns in EEG signals. Moreover, how to exploit and fuse crucial discriminative aspects in high redundancy and low signal-to-noise ratio EEG signals remains a great challenge for emotion recognition. In this paper, we propose a novel attention-based spatial-temporal dual-stream fusion network, named ASTDF-Net, for EEG-based emotion recognition. Specifically, ASTDF-Net comprises three main stages: first, the collaborative embedding module is designed to learn a joint latent subspace to capture the coupling of complicated spatiotemporal information in EEG signals. Second, stacked parallel spatial and temporal attention streams are employed to extract the most essential discriminative features and filter out redundant task-irrelevant factors. Finally, the hybrid attention-based feature fusion module is proposed to integrate significant features discovered from the dual-stream structure to take full advantage of the complementarity of the diverse characteristics. Extensive experiments on two publicly available emotion recognition datasets indicate that our proposed approach consistently outperforms state-of-the-art methods.

Regarding cognitive workload recognition, electroencephalography (EEG) signals are nonstationary across time and vary from different subjects, thus hindering the cross-subject recognition performance. Though subject calibration or collecting massive training data may ease the above problem, it is generally time-consuming and expensive. In this paper, we propose a Deep Domain Adaptation (DDA) scheme for EEG-based cross-subject cognitive workload recognition, using the knowledge from the existing subjects (source domain) to improve the recognition performance of a new subject (target domain). Precisely, the proposed DDA method composes four modules, EEG features extractor, label classifier, feature distribution alignment, and domain discriminator. The model starts with the EEG feature extractor to learn the shallow feature representation for both domains. The label classifier further learns the deep representation and trains the classifier supervised. Finally, the feature distribution alignment matches the shallow feature distribution discrepancy, and the domain discriminator matches the deep distribution discrepancy by the adversarial training with the feature extractor. It not only learns domain-invariant features but also achieves robust domain-adaptive cross-subject recognition results in an end-to-end training framework. We conduct experiments to classify low and high workloads on a self-designed EEG dataset and one public EEG dataset. Experimental results demonstrate that our DDA scheme significantly outperforms the baselines and other state-of-the-art methods with improvements of 2% to 9% in terms of recognition accuracy.

Speech imagery-based Brain-computer interface (BCI) using Electroencephalogram (EEG) signal is a promising area of research for individuals with severe speech production disorders. Recent advances in deep learning (DL) have led to significant improvements in this domain. However, there is a lack of comprehensive review that covers the application of DL methods for decoding imagined speech via EEG. In this paper, we survey speech imagery and DL literature to address critical questions regarding preferred paradigms, preprocessing necessity, optimal input formulations, and current trends in DL-based techniques. Specifically, we first search major databases across science and engineering disciplines for relevant studies. Then, we analyze the DL-based techniques applied in speech imagery decoding from five main perspectives: dataset, preprocessing, input formulation, DL architecture, and performance evaluation. Moreover, we summarize the key findings of this work and propose a set of practical recommendations. Finally, we highlight the practical challenges of DL-based imagined speech decoding and suggest future research directions.