Previous studies reported mental stress as one of the major contributing factors leading to various diseases such as heart attack, depression and stroke. An accurate stress assessment method may thus be of importance to clinical intervention and disease prevention. We propose a joint independent component analysis (jICA) based approach to fuse simultaneous measurement of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) on the prefrontal cortex (PFC) as a means of stress assessment. For the purpose of this study, stress was induced by using an established mental arithmetic task under time pressure with negative feedback. The induction of mental stress was confirmed by salivary alpha amylase test. Experiment results showed that the proposed fusion of EEG and fNIRS measurements improves the classification accuracy of mental stress by +3.4% compared to EEG alone and +11% compared to fNIRS alone. Similar improvements were also observed in sensitivity and specificity of proposed approach over unimodal EEG/fNIRS. Our study suggests that combination of EEG (frontal alpha rhythm) and fNIRS (concentration change of oxygenated hemoglobin) could be a potential means to assess mental stress objectively.

One of the common symptoms of autism,^{1, 2} Asperger syndrome,^{1, 3} schizophrenia, and depression is a lack of social skill. To alleviate these mental disorders on the basis of a neuroscience approach, it is important to investigate the relationship between social skills and brain function.⁴ Social skills, in general, include adaptability of one's own behavior to a social context, reading the atmosphere or others’ intentions, communication skills, and ethical sense.^{5, 6} Most of these social skills are acquired through interactions with others. To study the social-brain function, it is therefore meaningful to investigate interacting brains of multiple participants.