25 September 2024 in IEEE Transactions on Biomedical Engineering
Objective: Attention regulation is an essential ability in daily life that affects learning and work efficiency and is closely related to mental health. The effectiveness of brain-computer interface (BCI) systems in attention regulation has been proven, but most of these systems rely on bulky and expensive equipment and are still in the experimental stage. This study proposes a wearable BCI system for real-time attention regulation and cognitive monitoring. Methods: The BCI system integrates a wearable single-channel electroencephalogram (EEG) headband with wireless data streaming for real-time analysis. Twenty healthy subjects participated in the long-term attention regulation experiment and were evenly divided into an experimental group and a control group based on the presence of real-time neurofeedback. The neurofeedback is represented by output value of attention, which calculated from single-channel EEG data. Before and after the regulation sessions, baseline assessments were conducted for each subject, incorporating multi-channel EEG data analysis and cognitive behavioral evaluations, to verify the effectiveness of system for attention regulation. Results: The online experimental results indicate that the average attention level in the experimental group increased from 0.625 to 0.812, while no significant improvement was observed in the control group. Further comparative analysis revealed the reasons for the enhancement of attention regulation ability in terms of both brain network patterns and cognitive performance. Significance: The proposed wearable BCI system is effective at improving attention regulation ability and could be a promising tool for assisting people with attention disorders.
10 October 2022 in IEEE Transactions on Cognitive and Developmental Systems
In this study, a hybrid brain-computer interface (BCI) system combining P300 potential and emotion patterns was proposed to improve the performance of awareness detection. Two video clips were flashed randomly to evoke the P300 potential, while a laughing or crying video clip was used to induce the corresponding emotion pattern. The subjects were asked to concentrate on the laughing or crying video clip cued by the instruction and to count the flashes of the corresponding video clip. Two layers of classification were developed. In the first layer, P300 detection and emotion recognition were performed separately using two support vector machine (SVM) classifiers. Specifically, the activation, spatial and connection patterns were fused in emotion recognition. In the second layer, the SVM scores of P300 detection and emotion recognition were fed into another SVM classifier to determine which video clip the subjects responded to. Six healthy subjects and eight patients with disorders of consciousness (DOC) were involved in the command-following experiment. The results showed that the accuracy of the hybrid BCI system was better than those of the single-modality systems. Furthermore, three patients were able to perform tasks (66%-72%) using our hybrid BCI, which indicated their residual awareness and emotion-related abilities.
18 May 2022 in IEEE Transactions on Neural Systems and Rehabilitation Engineering
Behavioral assessment of sound localization in the Coma Recovery Scale-Revised (CRS-R) poses a significant challenge due to motor disability in patients with disorders of consciousness (DOC). Brain-computer interfaces (BCIs), which can directly detect brain activities related to external stimuli, may thus provide an approach to assess DOC patients without the need for any physical behavior. In this study, a novel audiovisual BCI system was developed to simulate sound localization evaluation in CRS-R. Specifically, there were two alternatively flashed buttons on the left and right sides of the graphical user interface, one of which was randomly chosen as the target. The auditory stimuli of bell sounds were simultaneously presented by the ipsilateral loudspeaker during the flashing of the target button, which prompted patients to selectively attend to the target button. The recorded electroencephalography data were analyzed in real time to detect event-related potentials evoked by the target and further to determine whether the target was attended to or not. A significant BCI accuracy for a patient implied that he/she had sound localization. Among eighteen patients, eleven and four showed sound localization in the BCI and CRS-R, respectively. Furthermore, all patients showing sound localization in the CRS-R were among those detected by our BCI. The other seven patients who had no sound localization behavior in CRS-R were identified by the BCI assessment, and three of them showed improvements in the second CRS-R assessment after the BCI experiment. Thus, the proposed BCI system is promising for assisting the assessment of sound localization and improving the clinical diagnosis of DOC patients.
19 June 2025 in IEEE Transactions on Affective Computing
Emotion recognition from EEG signals offers significant advantages in affective computing, as EEG more accurately reflects internal emotional states than other modalities, such as facial expressions or peripheral physiological signals. Modeling and capturing subtle affective changes over time is crucial for real-world applications to achieve better human-computer interaction. However, training such models usually requires segment-level emotion labels, which are costly and may not be feasible. Assigning the overall label to all EEG segments within a trial can lead to inaccurate model training and degraded performance, as emotions evolve continuously. This highlights the need for models capable of learning from trial-wise emotion labels while capturing temporal dynamics of emotional responses within each segment because trial-wise post-stimulus labels are more accessible. To this end, we propose EmotionMIL, an end-to-end EEG-based emotion recognition framework that leverages recent advances in deep multiple instance learning (MIL). This framework enables robust emotion recognition from weakly labeled EEG signals and identifies the most prominent emotional responses. EmotionMIL captures the temporal dynamics of emotions using a retentive self-attention mechanism, which adaptively assigns weights to EEG segments based on their relevance in predicting the overall emotion label. A pseudo-bag augmentation strategy is also introduced to enhance the model's generalization ability by generating additional pseudo-bags from the original ones. Evaluated on three benchmark datasets—DEAP, DREAMER, and SEED—EmotionMIL outperforms state-of-the-art non-MIL and MIL models in both subject-dependent and subject-independent tasks, achieving superior accuracy and F1-score. Ablation study further validates the model design, while visualization results demonstrate that EmotionMIL effectively identifies both spatial EEG patterns and temporal emotional dynamics. These findings underscore EmotionMIL's potential for robust, interpretable emotion recognition, paving the way for real-world applications in emotion-aware systems. The code is available at https://github.com/yuty2009/emotionmil.
15 April 2025 in IEEE Transactions on Biomedical Engineering
Deep learning-based methods for automatic sleep staging offer an efficient and objective alternative to costly manual scoring. However, their reliance on extensive labeled datasets and the challenge of generalization to new subjects and datasets limit their widespread adoption. Self-supervised learning (SSL) has emerged as a promising solution to address these issues by learning transferable representations from unlabeled data. This study highlights the effectiveness of SSL in automated sleep staging, utilizing a customized SSL approach to train a multi-view sleep staging model. This model includes a temporal view feature encoder for raw EEG signals and a spectral view feature encoder for time-frequency features. During pretraining, we incorporate a cross-view contrastive loss in addition to a contrastive loss for each view to learn complementary features and ensure consistency between views, enhancing the transferability and robustness of learned features. A dynamic weighting algorithm balances the learning speed of different loss components. Subsequently, these feature encoders, combined with a sequence encoder and a linear classifier, enable sleep staging after finetuning with labeled data. Evaluation on three publicly available datasets demonstrates that finetuning the entire SSL-pretrained model achieves competitive accuracy with state-of-the-art methods—86.4%, 83.8%, and 85.5% on SleepEDF-20, SleepEDF-78, and MASS datasets, respectively. Notably, our framework achieves near-equivalent performance with only 5% of the labeled data compared to full-label supervised training, showcasing SSL's potential to enhance automated sleep staging efficiency.