Yifei Zhang

Dr. Yifei Zhang

Northeastern University, Shenyang, Liaoning, China
orcid

Research Focus & Metrics

Academic Career Length

11Years (2016-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
2 November 2020 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
XYXiaocui Yang
Xiaocui Yang
SFShi Feng
Shi Feng
DWDaling Wang
Daling Wang
YZYifei Zhang
Yifei Zhang
Compared with single-modal content, multimodal data can express users feelings and sentiments more vividly and interestingly. Therefore, multimodal sentiment analysis has become a popular research topic. However, most existing methods either learn modal sentiment feature independently, without considering their correlations, or they simply integrate multimodal features. In addition, most publicly available multimodal datasets are labeled by sentiment polarities, while the emotions expressed by users are specific. Based on this observation, in this paper, we build a large-scale image-text emotion dataset (i.e., labeled by different emotions), called TumEmo, with more than 190,000 instances from Tumblr1. We further propose a novel multimodal emotion analysis model based on the Multi-view Attentional Network (MVAN), which utilizes a memory network that is continually updated to obtain the deep semantic features of image-text. The model includes three stages: feature mapping, interactive learning, and feature fusion. In the feature mapping stage, we leverage image features from an object viewpoint and a scene viewpoint to capture effective information for multimodal emotion analysis. Then, an interactive learning mechanism is adopted that uses the memory network; this mechanism extracts single-modal emotion features and interactively models the cross-view dependencies between the image and text. In the feature fusion stage, multiple features are deeply fused using a multilayer perceptron and a stacking-pooling module. The experimental results on the MVSA-Single, MVSA-Multiple, and TumEmo datasets show that the proposed MVAN outperforms strong baseline models by large margins. Index TermsMultimodal sentiment analysis; Memory network; Multi-view attention mechanism; Social media

26 October 2023
Verified
Conference Paper
Open access
XYXiaocui Yang
Xiaocui Yang
SFShi Feng
Shi Feng
DWDaling Wang
Daling Wang
YZYifei Zhang
Yifei Zhang
SPSoujanya Poria
Soujanya Poria
Multimodal sentiment analysis has gained significant attention due to the proliferation of multimodal content on social media. However, existing studies in this area rely heavily on large-scale supervised data, which is time-consuming and labor-intensive to collect. Thus, there is a need to address the challenge of few-shot multimodal sentiment analysis. To tackle this problem, we propose a novel method called Multimodal Probabilistic Fusion Prompts (MultiPoint) that leverages diverse cues from different modalities for multimodal sentiment detection in the few-shot scenario. Specifically, we start by introducing a Consistently Distributed Sampling approach called CDS, which ensures that the few-shot dataset has the same category distribution as the full dataset. Unlike previous approaches primarily using prompts based on the text modality, we design unified multimodal prompts to reduce discrepancies between different modalities and dynamically incorporate multimodal demonstrations into the context of each multimodal instance. To enhance the model's robustness, we introduce a probabilistic fusion method to fuse output predictions from multiple diverse prompts for each input. Our extensive experiments on six datasets demonstrate the effectiveness of our approach. First, our method outperforms strong baselines in the multimodal few-shot setting. Furthermore, under the same amount of data (1% of the full dataset), our CDS-based experimental results significantly outperform those based on previously sampled datasets constructed from the same number of instances of each class.

25 January 2026 in Knowledge-Based Systems
Verified
Journal Article
Closed access
YZYiqun Zhang
Yiqun Zhang
XYXiaocui Yang
Xiaocui Yang
XXXingle Xu
Xingle Xu
ZGZeran Gao
Zeran Gao
YHYijie Huang
Yijie Huang
GYGe Yu
Ge Yu

1 January 2024
Verified
Conference Paper
Open access
PDF file
YZYiqun Zhang
Yiqun Zhang
FKFanheng Kong
Fanheng Kong
PWPeidong Wang
Peidong Wang
SSShuang Sun
Shuang Sun
SSSWangLing SWangLing
SWangLing SWangLing
KSKaisong Song
Kaisong Song
Yiqun Zhang, Fanheng Kong, Peidong Wang, Shuang Sun, SWangLing SWangLing, Shi Feng, Daling Wang, Yifei Zhang, Kaisong Song. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

20 June 2021 in Mathematics
Verified
Journal Article
Open access
PDF file
HMHaotian Miao
Haotian Miao
YZYifei Zhang
Yifei Zhang
DWDaling Wang
Daling Wang
SFShi Feng
Shi Feng
With the development of social networks and intelligent terminals, it is becoming more convenient to share and acquire images. The massive growth of the number of social images makes people have higher demands for automatic image processing, especially in the aesthetic and emotional perspective. Both aesthetics assessment and emotion recognition require a higher ability for the computer to simulate high-level visual perception understanding, which belongs to the field of image processing and pattern recognition. However, existing methods often ignore the prior knowledge of images and intrinsic relationships between aesthetic and emotional perspectives. Recently, machine learning and deep learning have become powerful methods for researchers to solve mathematical problems in computing, such as image processing and pattern recognition. Both images and abstract concepts can be converted into numerical matrices and then establish the mapping relations using mathematics on computers. In this work, we propose an end-to-end multi-output deep learning model based on multimodal Graph Convolutional Network (GCN) and co-attention for aesthetic and emotion conjoint analysis. In our model, a stacked multimodal GCN network is proposed to encode the features under the guidance of the correlation matrix, and a co-attention module is designed to help the aesthetics and emotion feature representation learn from each other interactively. Experimental results indicate that our proposed model achieves competitive performance on the IAE dataset. Progressive results on the AVA and ArtPhoto datasets also prove the generalization ability of our model.

26 August 2020 in International Journal of Machine Learning and Cybernetics
Verified
Journal Article
Closed access
PDF file
SFShi Feng
Shi Feng
KSKaisong Song
Kaisong Song
DWDaling Wang
Daling Wang
WGWei Gao
Wei Gao
YZYifei Zhang
Yifei Zhang