The dynamic uncertain relationship among each brain region is a necessary factor that limits EEG-based emotion recognition. It is a thought-provoking problem to availably employ time-varying spatial and temporal characteristics from multi-channel electroencephalogram (EEG) signals. Although deep learning has made remarkable achievements in emotion recognition, the biological topological information among brain regions does not fully exploit, which is vital for EEG-based emotion recognition. In response to this problem, we design a hybrid model called ST-GCLSTM, which comprises a spatial-graph convolutional network (SGCN) module and an attention-enhanced bi-directional Long Short-Term Memory (LSTM) module. The main advantage of ST-GCLSTM is that it can consider the biological topology information of each brain region to extract representative spatial-temporal features from multiple EEG channels. Specifically, we construct two layers SGCN by introducing adjacency matrices to adaptively learn the intrinsic connection among different EEG channels. Moreover, an attention-enhanced mechanism is placed into a bi-directional LSTM module to extract the crucial spatial-temporal features from sequential EEG data, and then these features serve as the input layer of the classifier to learn discriminative emotion-related features. Extensive experiments on the DEAP, SEED, and SEED-IV datasets demonstrate the effectiveness of the proposed ST-GCLSTM model, revealing that our model had an absolute performance improvement over state-of-the-art strategies.
Emotion recognition has attracted a lot of attention in recent years. It is widely used in health care, teaching, human-computer interaction, and other fields. Various human emotional features are often used to recognize different emotions. Currently, there is more and more research on multimodal emotion recognition based on the fusion of multiple features. This paper proposes a deep learning model of multimodal emotion recognition based on the fusion of electroencephalogram (EEG) signals and facial expressions to achieve an excellent classification effect. First, the pre-trained convolution neural network (CNN) is used to extract the facial features from the facial expressions. Next, the attention mechanism is introduced to extract more critical facial frame features. Then, we apply CNNs to extract spatial features from original EEG signals, which use a local convolution kernel and a global convolution kernel to learn the features of left and right hemispheres channels and all EEG channels. After feature-level fusion, the fusion features of the facial expression features and EEG features are fed into the classifier for emotion recognition. This paper conducted experiments on the DEAP and MAHNOB-HCI datasets to evaluate the performance of the proposed model. The accuracy of valence dimension classification is 96.63%, and arousal dimension classification is 97.15% on the DEAP dataset, while 96.69% and 96.26% on the MAHNOB-HCI dataset. The experimental results show that the proposed model can effectively carry out emotion recognition.
27 December 2022 in Signal, Image and Video Processing
The electroencephalogram (EEG) signal has become a highly effective decoding target for emotion recognition and has garnered significant attention from researchers. Its spatial topological and time-dependent characteristics make it crucial to explore both spatial information and temporal information for accurate emotion recognition. However, existing studies often focus on either spatial or temporal aspects of EEG signals, neglecting the joint consideration of both perspectives. To this end, this article proposes a hybrid network consisting of a dynamic graph convolution (DGC) module and temporal self-attention representation (TSAR) module, which concurrently incorporates the representative knowledge of spatial topology and temporal context into the EEG emotion recognition task. Specifically, the DGC module is designed to capture the spatial functional relationships within the brain by dynamically updating the adjacency matrix during the model training process. Simultaneously, the TSAR module is introduced to emphasize more valuable time segments and extract global temporal features from EEG signals. To fully exploit the interactivity between spatial and temporal information, the hierarchical cross-attention fusion (H-CAF) module is incorporated to fuse the complementary information from spatial and temporal features. Extensive experimental results on the DEAP, SEED, and SEED-IV datasets demonstrate that the proposed method outperforms other state-of-the-art methods.
22 July 2025 in IEEE Transactions on Industrial Informatics
In this article, we propose Img2CAD, the first approach to our knowledge that uses 2-D image inputs to generate computer-aided design (CAD) models with editable parameters. Unlike existing artificial intelligence (AI) methods for 3-D model generation using text or image inputs often rely on mesh-based representations, which are incompatible with CAD tools and lack editability and fine control, Img2CAD enables seamless integration between AI-based 3-D reconstruction and CAD software. We have identified an innovative intermediate representation called structured visual geometry, characterized by vectorized wireframes extracted from objects. This representation significantly enhances the performance of generating conditioned CAD models. In addition, we introduce two new datasets to further support research in this area: a big cad model dataset (ABC)-mono, the largest known dataset comprising over 200 000 3-D CAD models with rendered images, and KOCAD, the first dataset featuring real-world captured objects alongside their ground truth CAD models, supporting further research in conditioned CAD model generation.