As a key field in music information retrieval, music emotion recognition is indeed a challenging task. To enhance the accuracy of music emotion classification and recognition, this paper uses the idea of inception structure to use different receptive fields to extract features of different dimensions and perform compression, expansion, and recompression operations to mine more effective features and connect the timing signals in the residual network to the GRU module to extract timing features. A one-dimensional (1D) residual Convolutional Neural Network (CNN) with an improved Inception module and Gate Recurrent Unit (GRU) was presented and tested on the Soundtrack dataset. Fast Fourier Transform (FFT) was used to process the samples experimentally and determine their spectral characteristics. Compared with the shallow learning methods such as support vector machine and random forest and the deep learning method based on Visual Geometry Group (VGG) CNN proposed by Sarkar et al., the proposed deep learning method of the 1D CNN with the Inception-GRU residual structure demonstrated better performance in music emotion recognition and classification tasks, achieving an accuracy of 84%.
7 March 2024 in IEEE Transactions on Fuzzy Systems
Conducting and interacting with an orchestra is a multimodal process that integrates channels such as music, visual cues, posture, and gestures to convey artistic intent accurately. For robots, discerning human emotions from these channels can enhance human-machine interactions. Currently, gesture recognition systems in orchestras focus more on rhythm, speed, and dynamics, while studying emotional factors in orchestra conducting music requires more profound research. We introduced the Facial Expression and Orchestra Gesture Emotion (FEGE) dataset, consisting of eight different emotions for recognition. This paper introduces a Fuzzy Multimodal Fusion Network (FMFN) based on fuzzy logic, which operates in multi-feature spaces and is designed for emotion recognition in bimodal tasks involving facial expressions and orchestra-conducting gestures. The network maps facial expressions and gestures into a multi-feature space through bimodal processing, learns unique and shared representations, and decodes them using classifiers optimized by FMFN parameters. Finally, it processes data uncertainty and fuzziness using a fuzzy logic system, improving the classification decision process to enhance the robustness and adaptability of emotion recognition tasks in bimodal visual modalities. Experimental results on the FEGE dataset confirmed the effectiveness of our network. The proposed bimodal fusion network achieved an accuracy of 89.16% in bimodal emotion recognition, which is approximately a 21% improvement over single-modal recognition results. This approach can also be better applied to human-machine interaction systems, particularly in orchestra conducting training, aiming to enhance the most critical emotional factors conveyed during the conducting process, thus elevating the depth of artistic intent.