A growing number of affective computing researches recently developed a computer system that can recognize an emotional state of the human user to establish affective human-computer interactions. Various measures have been used to estimate emotional states, including self-report, startle response, behavioral response, autonomic measurement, and neurophysiologic measurement. Among them, inferring emotional states from electroencephalography (EEG) has received considerable attention as EEG could directly reflect emotional states with relatively low costs and simplicity. Yet, EEG-based emotional state estimation requires well-designed computational methods to extract information from complex and noisy multichannel EEG data. In this paper, we review the computational methods that have been developed to deduct EEG indices of emotion, to extract emotion-related features, or to classify EEG signals into one of many emotional states. We also propose using sequential Bayesian inference to estimate the continuous emotional state in real time. We present current challenges for building an EEG-based emotion recognition system and suggest some future directions.

A1 Functional advantages of cell-type heterogeneity in neural circuits

There has been a long-standing demand for noninvasive neuroimaging methods that can detect neuronal activity at both high temporal and high spatial resolution. We present a two-dimensional fast line-scan approach that enables direct imaging of neuronal activity with millisecond precision while retaining the high spatial resolution of magnetic resonance imaging (MRI). This approach was demonstrated through in vivo mouse brain imaging at 9.4 tesla during electrical whisker-pad stimulation. In vivo spike recording and optogenetics confirmed the high correlation of the observed MRI signal with neural activity. It also captured the sequential and laminar-specific propagation of neuronal activity along the thalamocortical pathway. This high-resolution, direct imaging of neuronal activity will open up new avenues in brain science by providing a deeper understanding of the brain’s functional organization, including the temporospatial dynamics of neural networks.

In fast and transient somatosensory processing, the relative timing of the selected spikes is more important than the spike frequency because the ensemble of the first spikes in the spike trains encodes the dynamic tactile information. Here, inspired by the functional effectiveness of the selected spikes, we propose an artificial dynamic sensory system based on position-encoded spike spectrum. We use a mixed ion-electron conductor to generate a potential spike signal. We design artificial receptors that have different ion relaxation times (τ); thus, a sequence of the spikes from the receptors creates a spike spectrum, providing the spatial information (position and motion trace) and the temporal information (speed and dynamic contact area). The artificial receptors can be incorporated by as much as 132/square centimeters by using only two global signal addressing lines for sensor operation. Structural simplicity of the device opens the possibility of scalable fabrication with dense receptor integration. With computational decoding of the position-encoded spike spectrum, the artificial sensory system can recognize complicated dynamic motions in real time. The high-resolution spatiotemporal tactile perception in the ionic artificial sensory system enables the real-time dynamic robotic manipulation.

It has become increasingly important to monitor drivers’ negative emotions during driving to prevent accidents. Despite drivers’ anxiety being critical for safe driving, there is a lack of systematic approaches to detect anxiety in driving situations. This study employed multimodal biosignals, including electroencephalography (EEG), photoplethysmography (PPG), electrodermal activity (EDA) and pupil size to estimate anxiety under various driving situations. Thirty-one drivers, with at least one year of driving experience, watched a set of thirty black box videos including anxiety-invoking events, and another set of thirty videos without them, while their biosignals were measured. Then, they self-reported anxiety-invoked time points in each video, from which features of each biosignal were extracted. The logistic regression (LR) method classified single biosignals to detect anxiety. Furthermore, in the order of PPG, EDA, pupil, and EEG (easiest to hardest accessibility), LR classified accumulated multimodal signals. Classification using EEG alone showed the highest accuracy of 77.01%, while other biosignals led to a classification with accuracy no higher than the chance level. This study exhibited the feasibility of utilizing biosignals to detect anxiety invoked by driving situations, demonstrating benefits of EEG over other biosignals.