Runchuan Ye

Runchuan Ye

Auto-generated profile, based on publicly available information

Runchuan Ye

Shenzhen International Graduate School, Tsinghua University, Shenzhen, China

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
28 October 2024
Verified
Conference Paper
Open access
YXYaoxun Xu
Yaoxun Xu
YZYixuan Zhou
Yixuan Zhou
YCYunrui Cai
Yunrui Cai
JXJingran Xie
Jingran Xie
RYRunchuan Ye
Runchuan Ye
ZWZhiyong Wu
Zhiyong Wu
This paper addresses the challenges in MER 2024 by focusing on the Open Vocabulary (OV) task, which extends beyond traditional fixed label space for multimodal emotion recognition. The study emphasizes the use of Large Language Models (LLMs) to interpret and extract emotional information from multimodal inputs, complemented by speech transcription, speech emotion description, and video clues. The paper explores the integration of these features into a prompt fed into a pre-trained LLaMA3-8B model, utilizing prompt engineering to achieve satisfactory results without fine-tuning. This approach bridges the gap between speech, video and text data, leveraging the full potential of LLMs for open-ended emotion recognition tasks and introducing a solution to the field.

28 October 2024
Verified
Conference Paper
Open access
YCYunrui Cai
Yunrui Cai
RYRunchuan Ye
Runchuan Ye
JXJingran Xie
Jingran Xie
YZYixuan Zhou
Yixuan Zhou
YXYaoxun Xu
Yaoxun Xu
ZWZhiyong Wu
Zhiyong Wu
Multimodal emotion recognition (MER) plays a crucial role in user sentiment analysis and enhancing human-computer interaction experiences. However, in real-world scenarios, noise interference such as environmental noise and image blurring is widespread, impacting the model's ability to recognize emotions. Therefore, improving the robustness of MER models to handle complex and variable noise interference is a significant challenge. To address this challenge, we propose a robust representation learning method based on contrastive learning and Mixup data augmentation strategies to stabilize model performance on noisy data. Specifically, we first perform Mixup data augmentation in the representation space of each modality to broaden class decision boundaries and enhance model generalization. Meanwhile, through a contrastive learning strategy, we align the representations of clean and noisy data to improve model robustness. We evaluate our model's performance on the test set of the MER2024 MER-NOISE track. Our model achieves a weighted average F-score score of 82.71%, which is 3.09% higher than the MER2024 Baseline model. This result demonstrates that our method effectively enhances model robustness and generalization.