The objective of this paper is to develop a novel emotion recognition system from electroencephalogram (EEG) signals using effective connectivity and deep learning methods. Emotion recognition is an important task for various applications such as human-computer interaction and, mental health diagnosis. The paper aims to improve the accuracy and robustness of emotion recognition by combining different effective connectivity (EC) methods and pre-trained convolutional neural networks (CNNs), as well as long short-term memory (LSTM). EC methods measure information flow in the brain during emotional states using EEG signals. We used three EC methods: transfer entropy (TE), partial directed coherence (PDC), and direct directed transfer function (dDTF). We estimated a fused image from these methods for each five-second window of 32-channel EEG signals. Then, we applied six pre-trained CNNs to classify the images into four emotion classes based on the two-dimensional valence-arousal model. We used the leave-one-subject-out cross-validation strategy to evaluate the classification results. We also used an ensemble model to select the best results from the best pre-trained CNNs using the majority voting approach. Moreover, we combined the CNNs with LSTM to improve recognition performance. We achieved the average accuracy and F-score of 98.76%, 98.86%, 98.66 and 98.88% for classifying emotions using DEAP and MAHNOB-HCI datasets, respectively. Our results show that fused images can increase the accuracy and that an ensemble and combination of pre-trained CNNs and LSTM can achieve high accuracy for automated emotion recognition. Our model outperformed other state-of-the-art systems using the same datasets for four-class emotion classification.

Background/Objectives: Predicting individual response to depression therapy prior to treatment initiation remains a critical clinical challenge, as the response rate to both selective serotonin reuptake inhibitors (SSRIs) and repetitive transcranial magnetic stimulation (rTMS) is approximately 50%, leaving treatment selection largely trial-based. This study presents a computer-aided decision (CAD) framework that predicts depression therapy outcomes from pre-treatment electroencephalogram (EEG) signals using advanced time-frequency representations and pretrained convolutional neural networks (CNNs). Methods: EEG signals from 30 SSRI patients and 46 rTMS patients are transformed into time-frequency images using Continuous Wavelet Transform (CWT), Variational Mode Decomposition (VMD), and their pixel-level fusion. Four pretrained CNN architectures, including ResNet-18, MobileNet-V3, EfficientNet-B0, and TinyViT-Hybrid, are fine-tuned and evaluated under both image-independent and subject-independent 6-fold cross-validation (CV). Results: Results reveal a clear therapy-specific pattern: CWT-based representations yield superior discrimination for SSRI outcome prediction, with ResNet-18 achieving 99.43% image-level accuracy, while VMD-based representations are statistically superior for rTMS outcome prediction, with ResNet-18 reaching 98.77%. Pixel-level fusion of CWT and VMD does not consistently improve performance over the best individual representation in either therapy context. Pairwise Wilcoxon signed-rank tests confirm a two-tier architectural hierarchy in which ResNet-18 and TinyViT-Hybrid significantly outperform MobileNet-V3 and EfficientNet-B0 across all conditions, while remaining statistically indistinguishable from each other. At the subject level, the framework achieves 82.50% and 83.53% accuracy for SSRI and rTMS, respectively, under strict subject-independent evaluation. Per-channel analysis reveals occipital dominance for SSRI under CWT and frontotemporal dominance for rTMS under VMD, consistent with known neurophysiological mechanisms. Conclusions: These findings demonstrate that the choice of time-frequency representation is therapy-specific and at least as important as architectural complexity, and that competitive performance can be achieved without recurrent or attention layers by combining well-designed spectral images with a simple pretrained residual network.