Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) stand as state-of-the-art techniques for non-invasive functional neuroimaging. On a unimodal basis, EEG has poor spatial resolution while presenting high temporal resolution. In contrast, fNIRS offers better spatial resolution, though it is constrained by its poor temporal resolution. One important merit shared by the EEG and fNIRS is that both modalities have favorable portability and could be integrated into a compatible experimental setup, providing a compelling ground for the development of a multimodal fNIRS–EEG integration analysis approach. Despite a growing number of studies using concurrent fNIRS-EEG designs reported in recent years, the methodological reference of past studies remains unclear. To fill this knowledge gap, this review critically summarizes the status of analysis methods currently used in concurrent fNIRS–EEG studies, providing an up-to-date overview and guideline for future projects to conduct concurrent fNIRS–EEG studies. A literature search was conducted using PubMed and Web of Science through 31 August 2021. After screening and qualification assessment, 92 studies involving concurrent fNIRS–EEG data recordings and analyses were included in the final methodological review. Specifically, three methodological categories of concurrent fNIRS–EEG data analyses, including EEG-informed fNIRS analyses, fNIRS-informed EEG analyses, and parallel fNIRS–EEG analyses, were identified and explained with detailed description. Finally, we highlighted current challenges and potential directions in concurrent fNIRS–EEG data analyses in future research.

Motor Imagery (MI) paradigm is critical in neural rehabilitation and gaming. Advances in brain-computer interface (BCI) technology have facilitated the detection of MI from electroencephalogram (EEG). Previous studies have proposed various EEG-based classification algorithms to identify the MI, however, the performance of prior models was limited due to the cross-subject heterogeneity in EEG data and the shortage of EEG data for training. Therefore, inspired by generative adversarial network (GAN), this study aims to propose an improved domain adaption network based on Wasserstein distance, which utilizes existing labeled data from multiple subjects (source domain) to improve the performance of MI classification on a single subject (target domain). Specifically, our proposed framework consists of three components, including a feature extractor, a domain discriminator, and a classifier. The feature extractor employs an attention mechanism and a variance layer to improve the discrimination of features extracted from different MI classes. Next, the domain discriminator adopts the Wasserstein matrix to measure the distance between source domain and target domain, and aligns the data distributions of source and target domain via adversarial learning strategy. Finally, the classifier uses the knowledge acquired from the source domain to predict the labels in the target domain. The proposed EEG-based MI classification framework was evaluated by two open-source datasets, the BCI Competition IV Datasets 2a and 2b. Our results demonstrated that the proposed framework could enhance the performance of EEG-based MI detection, achieving better classification results compared with several state-of-the-art algorithms. In conclusion, this study is promising in helping the neural rehabilitation of different neuropsychiatric diseases.

Emotion plays crucial roles in human life. Recently, emotion classification from electroencephalogram (EEG) signal has attracted attention by researchers due to the rapid development of brain computer interface (BCI) techniques and machine learning algorithms. However, recent studies on emotion classification show resource utilization because they use the fully-supervised learning methods. Therefore, in this study, we applied the self-supervised learning methods to improve the efficiency of resources usage. We employed a self-supervised approach to train deep multi-task convolutional neural network (CNN) for EEG-based emotion classification. First, six signal transformations were performed on unlabeled EEG data to construct the pretext task. Second, a multi-task CNN was used to perform signal transformation recognition on the transformed signals together with the original signals. After the signal transformation recognition network was trained, the convolutional layer network was frozen and the fully connected layer was reconstructed as emotion recognition network. Finally, the EEG data with affective labels were used to train the emotion recognition network to clarify the emotion. In this paper, we conduct extensive experiments from the data scaling perspective using the SEED, DEAP affective dataset. Results showed that the self-supervised learning methods can learn the internal representation of data and save computation time compared to the fully-supervised learning methods. In conclusion, our study suggests that the self-supervised machine learning model can improve the performance of emotion classification compared to the conventional fully supervised model.

Brain-Computer Interface (BCI) techniques hold a great promise for neuroprosthetic applications. A desirable BCI system should be portable, minimally invasive, and feature high classification accuracy and efficiency. As two commonly used noninvasive brain imaging modalities, Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) BCI system have often been incorporated in the development of hybrid BCI systems, largely due to their complimentary properties. In this study, we aimed to investigate whether the early temporal information extracted from singular EEG and fNIRS channels on each hemisphere can be used to enhance the accuracy and efficiency of a hybrid EEG-fNIRS BCI system. Eleven healthy volunteers were recruited and underwent simultaneous EEG-fNIRS recording during a motor execution task that included left and right hand movements. Singular EEG and fNIRS channels corresponding to the motor cortices of each hemisphere were selected using a general linear model. Early temporal information was extracted from the EEG channel (0-1s) along with initial hemodynamic dip information from fNIRS (0-2s) for classification using a support vector machine (SVM). Results demonstrated a lofty classification accuracy using a minimal number of channels and features derived from early temporal information. In conclusion, a hybrid EEG-fNIRS BCI system can achieve higher classification accuracy (91.02%±4.08%) and efficiency by integrating their complimentary properties, compared to using EEG (85.64%±7.4%) or fNIRS alone (85.55%±10.72%). Such a hybrid system can also achieve minimal response lag in application by focusing on rapidly-evolving brain dynamics.

Emerging evidence indicates that cognitive deficits in Alzheimer's disease (AD) are associated with disruptions in brain network. Exploring alterations in the AD brain network is therefore of great importance for understanding and treating the disease. This study employs an integrative functional near-infrared spectroscopy (fNIRS) - electroencephalography (EEG) analysis approach to explore dynamic, regional alterations in the AD-linked brain network. FNIRS and EEG data were simultaneously recorded from 14 participants (8 healthy controls and 6 patients with mild AD) during a digit verbal span task (DVST). FNIRS-based spatial constraints were used as priors for EEG source localization. Graph-based indices were then calculated from the reconstructed EEG sources to assess regional differences between the groups. Results show that patients with mild AD revealed weaker and suppressed cortical connectivity in the high alpha band and in beta band to the orbitofrontal and parietal regions. AD-induced brain networks, compared to the networks of age-matched healthy controls, were mainly characterized by lower degree, clustering coefficient at the frontal pole and medial orbitofrontal across all frequency ranges. Additionally, the AD group also consistently showed higher index values for these graph-based indices at the superior temporal sulcus. These findings not only validate the feasibility of utilizing the proposed integrated EEG-fNIRS analysis to better understand the spatiotemporal dynamics of brain activity, but also contribute to the development of network-based approaches for understanding the mechanisms that underlie the progression of AD.