In the rapid serial visual presentation (RSVP) classification task, the data from the target and non-target classes are incredibly imbalanced. These class imbalance problems (CIPs) can hinder the classifier from achieving better performance, especially in deep learning. This paper proposed a novel data augmentation method called balanced Wasserstein generative adversarial network with gradient penalty (BWGAN-GP) to generate RSVP minority class data. The model learned useful features from majority classes and used them to generate minority-class artificial EEG data. It combines generative adversarial network (GAN) with autoencoder initialization strategy enables this method to learn an accurate class-conditioning in the latent space to drive the generation process towards the minority class. We used RSVP datasets from nine subjects to evaluate the classification performance of our proposed generated model and compare them with those of other methods. The average AUC obtained with BWGAN-GP on EEGNet was 94.43%, an increase of 3.7% over the original data. We also used different amounts of original data to investigate the effect of the generated EEG data on the calibration phase. Only 60% of original data were needed to achieve acceptable classification performance. These results show that the BWGAN-GP could effectively alleviate CIPs in the RSVP task and obtain the best performance when the two classes of data are balanced. The findings suggest that data augmentation techniques could generate artificial EEG to reduce calibration time in other brain-computer interfaces (BCI) paradigms similar to RSVP.

Objective. Motor Imagery Brain-Computer Interface (MI-BCI) is an active Brain-Computer Interface (BCI) paradigm focusing on the identification of motor intention, which is one of the most important non-invasive BCI paradigms. In MI-BCI studies, deep learning-based methods (especially lightweight networks) have attracted more attention in recent years, but the decoding performance still needs further improving. Approach. To solve this problem, we designed a filter bank structure with sinc-convolutional layers for spatio-temporal feature extraction of MI-electroencephalography in four motor rhythms. The Channel Self-Attention method was introduced for feature selection based on both global and local information, so as to build a model called Filter Bank Sinc-convolutional Network with Channel Self-Attention for high performance MI-decoding. Also, we proposed a data augmentation method based on multivariate empirical mode decomposition to improve the generalization capability of the model. Main results. We performed an intra-subject evaluation experiment on unseen data of three open MI datasets. The proposed method achieved mean accuracy of 78.20% (4-class scenario) on BCI Competition IV IIa, 87.34% (2-class scenario) on BCI Competition IV IIb, and 72.03% (2-class scenario) on Open Brain Machine Interface (OpenBMI) dataset, which are significantly higher than those of compared deep learning-based methods by at least 3.05% ( p = 0.0469), 3.18% ( p = 0.0371), and 2.27% ( p = 0.0024) respectively. Significance. This work provides a new option for deep learning-based MI decoding, which can be employed for building BCI systems for motor rehabilitation.

Exploring the effective signal features of electroencephalogram (EEG) signals is an important issue in the research of brain–computer interface (BCI), and the results can reveal the motor intentions that trigger electrical changes in the brain, which has broad research prospects for feature extraction from EEG data. In contrast to previous EEG decoding methods that are based solely on a convolutional neural network, the traditional convolutional classification algorithm is optimized by combining a transformer mechanism with a constructed end-to-end EEG signal decoding algorithm based on swarm intelligence theory and virtual adversarial training. The use of a self-attention mechanism is studied to expand the receptive field of EEG signals to global dependence and train the neural network by optimizing the global parameters in the model. The proposed model is evaluated on a real-world public dataset and achieves the highest average accuracy of 63.56% in cross-subject experiments, which is significantly higher than that found for recently published algorithms. Additionally, good performance is achieved in decoding motor intentions. The experimental results show that the proposed classification framework promotes the global connection and optimization of EEG signals, which can be further applied to other BCI tasks.

Objective. Achieving high precision rapid serial visual presentation (RSVP) task often requires many electrode channels to obtain more information. However, the more channels may contain more redundant information and also lead to its limited practical applications. Therefore, it is necessary to reduce the number of channels to enhance the classification performance and users experience. Furthermore, cross-subject generalization has always been one of major challenges in electroencephalography channel reduction, especially in the RSVP paradigm. Most search-based channel selection method presented in the literature are single-objective methods, the classification accuracy (ACC) is usually chosen as the only criterion. Approach. In this article, the idea of multi-objective optimization was introduced into the RSVP channel selection to minimize two objectives: classification error and the number of channels. By combining a multi-objective evolutionary algorithm for solving large-scale sparse problems and hierarchical discriminant component analysis (HDCA), a novel channel selection method for RSVP was proposed. After that, the cross-subject generalization validation through the proposed channel selection method. Main results. The proposed method achieved an average ACC of 95.41% in a public dataset, which is 3.49% higher than HDCA. The ACC was increased by 2.73% and 2.52%, respectively. Besides, the cross-subject generalization models in channel selection, namely special-16 and special-32, on untrained subjects show that the classification performance is better than the Hoffmann empirical channels. Significance. The proposed channel selection method could reduce the calibration time in the experimental preparation phase and obtain a better accuracy, which is promising application in the RSVP scenario that requires low-density electrodes.

Objective: The Rapid Serial Visual Presentation (RSVP) paradigm facilitates target identification in a rapid picture stream, which is applied extensively in military target surveillance and police monitoring. Most researchers concentrate on the single target RSVP-BCI whereas the study of dual-target is scarcely conducted, limiting RSVP application considerably. Methods: This paper proposed a novel classification model named Common Representation Extraction-Targeted Stacked Convolutional Autoencoder (CRE-TSCAE) to detect two targets with one nontarget in RSVP tasks. CRE generated a common representation for each target class to reduce variability from different trials of the same class and distinguish the difference between two targets better. TSCAE aimed to control uncertainty in the training process while requiring less target training data. The model learned a compact and discriminative feature through the training from several learning tasks so as to distinguish each class effectively. Results: It was validated on the World Robot Contest 2021 and 2022 ERP datasets. Experimental results showed that CRE-TSCAE outperformed the state-of-the-art RSVP decoding algorithms and the Average ACC was 71.25%, improving 6.5% at least over the rest. Conclusion: It demonstrated that CRE-TSCAE showed a strong ability to extract discriminative latent features in detecting the differences among two targets with nontarget, which guaranteed increased classification accuracy. Significance: CRE-TSCAE provided an innovative and effective classification model for dual-target RSVP-BCI tasks and some insights into the neurophysiological distinction between different targets.

Objective: Electroencephalography (EEG)-based Rapid Serial Visual Presentation (RSVP) has steadily gained attention since 2015 as a paradigm to enhance image target detection in brain-computer interfaces (BCIs) used with healthy individuals. Methods: We reviewed the literature using Scopus and Web of Science as primary databases, covering publications from 2015 to 2024. After literature screening and filtering, a total of 86 papers on RSVP-BCI studies were analyzed over this decade-long period. The research categorizes RSVP into three dimensions: public datasets, paradigm encoding, and decoding methods, while exploring eight mode combinations involving target types, subject groups, and different modalities. Results: Our literature search revealed a scarcity of studies addressing diverse target types across different subject groups or modality combinations, indicating a promising direction for future RSVP-BCI development. Future efforts should prioritize inclusivity across all age groups, the design of user-friendly stimulus interfaces, and the development of advanced algorithms, with the goal of creating a more widely accessible RSVP-BCI system. Conclusion: We have provided a comprehensive review of advances over the past decade in RSVP-based target detection, including datasets, encoding design, decoding methods and potential applications. Significance: The present work aims to articulate prospective trajectories for the continued advancement of the RSVP community.