People express emotions through different modalities. Utilization of both verbal and nonverbal communication channels allows to create a system in which the emotional state is expressed more clearly and therefore easier to understand. Expanding the focus to several expression forms can facilitate research on emotion recognition as well as human–machine interaction. This article presents analysis of audiovisual information to recognize human emotions. A cross-corpus evaluation is done using three different databases as the training set (SAVEE, eNTERFACE’05 and RML) and AFEW (database simulating real-world conditions) as a testing set. Emotional speech is represented by commonly known audio and spectral features as well as MFCC coefficients. The SVM algorithm has been used for classification. In case of facial expression, faces in key frames are found using Viola–Jones face recognition algorithm and facial image emotion classification done by CNN (AlexNet). Multimodal emotion recognition is based on decision-level fusion. The performance of emotion recognition algorithm is compared with the validation of human decision makers.

This study was aimed to compare two electroencephalogram (EEG) analysis methods, spectral asymmetry index (SASI) and Higuchi’s fractal dimension (HFD), for detection of depression. Linear SASI method is based on evaluation of the balance of powers in two EEG frequency bands in one channel selected higher and lower than the alpha band spectrum maximum. Nonlinear HFD method calculates fractal dimension directly in the time domain. The resting EEG signals of 17 depressive patients and 17 control subjects were used as a database for calculations. SASI values were positive for depressive and negative for control group (P<0.05). SASI provided the true detection rate of 88% in the depressive and 82% in the control group. The calculated HFD values detected a small (3%) increase with depression (P<0.05). HFD provided the true detection rate of 94% in the depressive group and 76% in the control group. The rate of correct indication in the both groups was 85% using SASI or HFD. Statistically significant variations were not revealed between hemispheres (P>0.05). The results indicated that the linear EEG analysis method SASI and the nonlinear HFD method both demonstrated a good sensitivity for detection of characteristic features of depression in a single-channel EEG.

Depression is a public health issue that severely affects one’s well being and can cause negative social and economic effects to society. To raise awareness of these problems, this research aims at determining whether the long-lasting effects of depression can be determined from electroencephalographic (EEG) signals. The article contains an accuracy comparison for SVM, LDA, NB, kNN, and D3 binary classifiers, which were trained using linear (relative band power, alpha power variability, spectral asymmetry index) and nonlinear (Higuchi fractal dimension, Lempel–Ziv complexity, detrended fluctuation analysis) EEG features. The age- and gender-matched dataset consisted of 10 healthy subjects and 10 subjects diagnosed with depression at some point in their lifetime. Most of the proposed feature selection and classifier combinations achieved accuracy in the range of 80% to 95%, and all the models were evaluated using a 10-fold cross-validation. The results showed that the motioned EEG features used in classifying ongoing depression also work for classifying the long-lasting effects of depression.