Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) stand as state-of-the-art techniques for non-invasive functional neuroimaging. On a unimodal basis, EEG has poor spatial resolution while presenting high temporal resolution. In contrast, fNIRS offers better spatial resolution, though it is constrained by its poor temporal resolution. One important merit shared by the EEG and fNIRS is that both modalities have favorable portability and could be integrated into a compatible experimental setup, providing a compelling ground for the development of a multimodal fNIRS–EEG integration analysis approach. Despite a growing number of studies using concurrent fNIRS-EEG designs reported in recent years, the methodological reference of past studies remains unclear. To fill this knowledge gap, this review critically summarizes the status of analysis methods currently used in concurrent fNIRS–EEG studies, providing an up-to-date overview and guideline for future projects to conduct concurrent fNIRS–EEG studies. A literature search was conducted using PubMed and Web of Science through 31 August 2021. After screening and qualification assessment, 92 studies involving concurrent fNIRS–EEG data recordings and analyses were included in the final methodological review. Specifically, three methodological categories of concurrent fNIRS–EEG data analyses, including EEG-informed fNIRS analyses, fNIRS-informed EEG analyses, and parallel fNIRS–EEG analyses, were identified and explained with detailed description. Finally, we highlighted current challenges and potential directions in concurrent fNIRS–EEG data analyses in future research.

Emotion recognition is important in the application of brain-computer interface (BCI). Building a robust emotion recognition model across subjects and sessions is critical in emotion based BCI systems. Electroencephalogram (EEG) is a widely used tool to recognize different emotion states. However, EEG has disadvantages such as small amplitude, low signal-to-noise ratio, and non-stationary properties, resulting in large differences across subjects. To solve these problems, this paper proposes a new emotion recognition method based on a multi-source associate domain adaptation network, considering both domain invariant and domain-specific features. First, separate branches were constructed for multiple source domains, assuming that different EEG data shared the same low-level features. Secondly, the domain specific features were extracted by using the one-to-one associate domain adaptation. Then, the weighted scores of specific sources were obtained according to the distribution distance, and multiple source classifiers were deduced with the corresponding weighted scores. Finally, EEG emotion recognition experiments were conducted on different datasets, including SEED, DEAP, and SEED-IV dataset. Results indicated that, in the cross-subject experiment, the average accuracy in SEED dataset was 86.16%, DEAP dataset was 65.59%, and SEED-IV was 59.29%. In the cross-session experiment, the accuracies of SEED and SEED-IV datasets were 91.10% and 66.68%, respectively. Our proposed method has achieved better classification results compared to state-of-the-art domain adaptation methods.

Motor imagery (MI)-based electroencephalography (EEG) stands as a prominent paradigm in the brain–computer interface (BCI) field, which is frequently applied in neural rehabilitation and gaming due to its accessibility and reliability. Despite extensive research dedicated to MI EEG classification algorithms, a notable deficiency still remains: their performance is often optimal only in subject-specific or dataset-specific scenarios, which undermines their generalization capability, hence restricting BCI systems' practical utility in real-world contexts. To address this limitation, this study introduces a cutting-edge approach: a discriminative adversarial network based on spatial-temporal-graph fusion (STG-DAN). This innovation aims to learn features that are not only class-discriminative but also domain-invariant. Specifically, the feature extraction module guarantees the feature discriminativeness by amalgamating spatial-temporal and graph-related features, while the domain alignment module focuses on both global domain and local subdomain. The two modules are incorporated into one adversarial learning framework to facilitate the acquisition of domain-invariant features. Evaluations on two publicly accessible datasets, BCI competition IV 2a and OpenBMI, affirm the superiority of our proposed model (averaged accuracy = 62.94% and 73.01% for the two datasets in cross-subject circumstance, respectively). In cross-dataset circumstances, it also outperforms several state-of-the-art algorithms, attesting to the potency of STG-DAN.

Motor control deficits are very common in stroke survivors and often lead to disability. Current clinical measures for profiling motor control impairments are largely subjective and lack precise interpretation in a “control” perspective. This study aims to provide an accurate interpretation and assessment of the underlying “motor control” deficits caused by stroke, using a recently developed novel technique, i.e., the functional brain controllability analysis. The electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) were simultaneously recorded from 16 stroke patients and 11 healthy subjects during a hand-clenching task. A high spatiotemporal resolution fNIRS-informed EEG source imaging approach was then employed to estimate the cortical activity and construct the functional brain network. Subsequently, network control theory was applied to evaluate the modal controllability of some key motor regions, including primary motor cortex (M1), premotor cortex (PMC), and supplementary motor cortex (SMA), and also the executive control network (ECN). Results indicated that the modal controllability of ECN in stroke patients was significantly lower than healthy subjects (p = 0.03). Besides, the modal controllability of SMA in stroke patients was also significant smaller than healthy subjects (p = 0.02). Finally, the baseline modal controllability of M1 was found to be significantly correlated with the baseline FM-UL clinical scores (r = 0.58, p = 0.01). In conclusion, our results provide a new perspective to better understand the motor control deficits caused by stroke. We expect such an analytical methodology can be extended to investigate the other neurological or psychiatric diseases caused by cognitive control or motor control impairment.

Perceiving and modulating emotions is vital for cognitive function and is often impaired in neuropsychiatric conditions. Current tools for evaluating emotional dysregulation suffer from subjectivity and lack of precision, especially when it comes to understanding emotion from a regulatory or control-based perspective. To address these limitations, this study leverages an advanced methodology known as functional brain controllability analysis. We simultaneously recorded electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) data from 17 healthy subjects engaged in emotion processing and regulation tasks. We then employed a novel EEG/fMRI integration technique to reconstruct cortical activity in a high spatiotemporal resolution manner. Subsequently, we conducted functional brain controllability analysis to explore the neural network control patterns underlying different emotion conditions. Our findings demonstrated that the dorsolateral and ventrolateral prefrontal cortex exhibited increased controllability during the processing and regulation of negative emotions compared to processing of neutral emotion. Besides, the anterior cingulate cortex was notably more active in managing negative emotion than in either controlling neutral emotion or regulating negative emotion. Finally, the posterior parietal cortex emerged as a central network controller for the regulation of negative emotion. This study offers valuable insights into the cortical control mechanisms that support emotion perception and regulation.

Objective: The changes in neural drive to muscles associated with modulation of inter-muscular coordination in the upper extremity have not yet been investigated. Such information could help elucidate the neural mechanisms behind motor skill learning. Methods: Six young, neurologically healthy participants underwent a six-week training protocol to decouple two synergist elbow flexor muscles as a newly learned motor skill in the isometric force generation in upward and medial directions. Concurrent electroencephalography and surface electromyography from twelve upper extremity muscles were recorded in two conditions (As-Trained & Habitual) across two assessments (Week 0 vs. Week 6). Changes to inter-muscular connectivity (IMC), functional muscle networks, cortico-muscular connectivity (CMC), cortico-cortical connectivity (CCC) as well as functional brain network controllability (FBNC) associated with the modulation of inter-muscular coordination patterns were assessed to provide a perspective on the neural mechanisms for the newly learned motor skills. Results: Significant decreases in elbow flexor IMC, CMC, and increases in CCC were observed. No significant changes were observed for FBNC. Conclusion: The results of this study suggest that modulating the inter-muscular coordination of the elbow flexor muscle synergy during isometric force generation is associated with multiple yet distinct changes in functional connectivity across the central and peripheral perspectives. Significance: Understanding the neural mechanisms of modulating inter-muscular coordination patterns can help inform motor rehabilitation regimens.