With the popularization of better network access and the penetration of personal smartphones in today’s world, the explosion of multi-modal data, particularly opinionated video messages, has created urgent demands and immense opportunities for Multi-Modal Sentiment Analysis (MSA). Deep learning with the attention mechanism has served as the foundation technique for most state-of-the-art MSA models due to its ability to learn complex inter- and intra-relationships among different modalities embedded in video messages, both temporally and spatially. However, modal fusion is still a major challenge due to the vast feature space created by the interactions among different data modalities. To address the modal fusion challenge, we propose an MSA algorithm based on deep learning and the attention mechanism, namely the Mixture of Attention Variants for Modal Fusion (MAVMF). The MAVMF algorithm includes a two-stage process: in stage one, self-attention is applied to effectively extract image and text features, and the dependency relationships in the context of video discourse are captured by a bidirectional gated recurrent neural module; in stage two, four multi-modal attention variants are leveraged to learn the emotional contributions of important features from different modalities. Our proposed approach is end-to-end and has been shown to achieve a superior performance to the state-of-the-art algorithms when tested with two largest public datasets, CMU-MOSI and CMU-MOSEI.

Recommender systems have gained widespread adoption but still suffer from challenges, such as limited feature diversity and difficulties in capturing complex user–item interactions. In this article, we propose a Knowledge Graph Enhanced recommender system based on a Variational Graph Autoencoder (KG-EVGAE). KG-EVGAE first applies a path augmentation strategy to enrich the KG with potentially missing relations, thereby enhancing entity connectivity and revealing latent structural semantics. Based on the augmented KG, convolutional neural networks are used to extract interaction features, while an attention mechanism is employed on the user social network to adaptively learn user features. These features are fused through multiple feature fusion strategies and fed into an improved VGAE to learn expressive latent embeddings for recommendation. Thus, KG-EVGAE not only mitigates the challenges of missing features and diversity insufficiency, but also captures complex interactions and dependencies between nodes and edges more effectively. Extensive experiments conducted on four standard datasets demonstrate the superiority of KG-EVGAE in recommendation tasks.

Audio classification plays an essential role in sentiment analysis and emotion recognition, especially for analyzing customer attitudes in marketing phone calls. Efficiently categorizing customer purchasing propensity from large volumes of audio data remains challenging. In this work, we propose a novel Multi-Segment Multi-Task Fusion Network (MSMT-FN) that is uniquely designed for addressing this business demand. Evaluations conducted on our proprietary MarketCalls dataset, as well as established benchmarks (CMU-MOSI, CMU-MOSEI, and MELD), show MSMT-FN consistently outperforms or matches state-of-the-art methods. Additionally, our newly curated MarketCalls dataset will be available upon request, and the code base is made accessible at GitHub Repository MSMT-FN, to facilitate further research and advancements in audio classification domain.