Multimodal sentiment analysis (MSA) seeks to predict subjective human sentiments by utilizing information from multiple modalities. It has been applied in diverse scenarios. Recent studies suggest that MSA benefits from integrating diverse modalities, emphasizing the fusion of multimodal information at different levels and the joint learning of modality-consistent and modality-inconsistent representations. In this work, we put forward a cross-modality dual-attention fusion network (CMDAF) that incorporates a cross-modality dual-attention fusion module. This module facilitates the integration of representations across modalities, enabling effective fusion from shallow to deep levels. A Transformer-based projection module transforms the fused representations into modality-consistent and modality-inconsistent forms, further enhanced through multi-scale contrast learning. Experiments on three MSA datasets (MOSI, MOSEI, and CH-SIMS) demonstrate the superiority of CMDAF, achieving state-of-the-art performance on most metrics.

Jiao-Liao Mandarin, a distinguished dialect in China, reflects the linguistic features and cultural heritage of the Jiao-Liao region. However, the labor-intensive and costly nature of manual transcription limits the scale of transcribed corpora, posing challenges for speech recognition. We present JLMS25, a transcribed corpus for Jiao-Liao Mandarin, alongside a novel multi-dialect knowledge transfer (MDKT) framework for low-resource speech recognition. By leveraging phonetic and linguistic knowledge from neighboring dialects, the MDKT framework improves recognition in resource-constrained settings. It comprises an acoustic feature extractor, a dialect feature extractor, and two modules—WFAdapter (weight decomposition adapter) and AttAdapter (attention-based adapter)—to enhance adaptability and mitigate overfitting. The training involves a three-phase strategy: multi-dialect AID-ASR multi-task learning in phase one, freezing the dialect feature extractor in phase two, and fine-tuning only the adapters in phase three. Experiments on the Jiao-Liao Mandarin subset of the KeSpeech dataset and JLMS25 dataset show that MDKT outperforms full-parameter fine-tuning, reducing Character Error Rate (CER) by 5.4% and 7.7% and Word Error Rate (WER) by 6.1% and 10.8%, respectively.