Zefan Zhang

Zefan Zhang

Auto-generated profile, based on publicly available information

Zefan Zhang

College of Computer Science and Technology, Key Laboratory of Symbolic Computation and Knowledge Engineering, Ministry of Education, Jilin University, China

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
28 October 2024
Verified
Conference Paper
Open access
ZZZefan Zhang
Zefan Zhang
WZWeiqi Zhang
Weiqi Zhang
YLYanhui Li
Yanhui Li
TBTian Bai
Tian Bai
Multimodal Relation Extraction (MRE) has achieved great improvements. However, modern MRE models are easily affected by irrelevant objects during multimodal alignment which are called error sensitivity issues. The main reason is that visual features are not fully aligned with textual features and the reasoning process may suppress redundant and noisy information at the risk of losing critical information. In light of this, we propose a Caption-Aware Multimodal Relation Extraction Network with Mutual Information Maximization (CAMIM). Specifically, we first generate detailed image captions through the Large Language Model (LLM). Then, the Caption-Aware Module (CAM) hierarchically aligns the fine-grained visual entities and textual entities for reasoning. In addition, for preserving crucial information within different modalities, we leverage a Mutual Information Maximization method to regulate the multimodal reasoning module. Experiments show that our model outperforms the state-of-the-art MRE models on the benchmark dataset MNRE. Further ablation studies prove the pluggable and effective performance of our Caption-Aware Module and Mutual Information Maximization method. Our code is available at https://github.com/zefanZhang-cn/CAMIM.

12 June 2023
Verified
Conference Paper
Open access
ZZZefan Zhang
Zefan Zhang
YJYi Ji
Yi Ji
CLChunping Liu
Chunping Liu

27 October 2025
Verified
Conference Paper
Open access
ZZZefan Zhang
Zefan Zhang
WZWeiqi Zhang
Weiqi Zhang
KSKailong Suo
Kailong Suo
YLYanhui Li
Yanhui Li
TBTian Bai
Tian Bai

11 October 2026 in ACM Transactions on Multimedia Computing, Communications, and Applications
Verified
Journal Article
Open access
ZZZefan Zhang
Zefan Zhang
KZKehua Zhu
Kehua Zhu
SJShijie Jiang
Shijie Jiang
TBTian Bai
Tian Bai
Recently, Multimodal Relation Extraction (MRE) has achieved great improvements. The inherent abstraction and variability of visual content make it challenging to extract relational information in multimodal relation extraction. Beyond identifying explicit visual entities for filtering out noisy information, it is also essential to capture implicit visual background knowledge that aids relation reasoning. Collectively, these factors are critical in alleviating the error sensitivity issues caused by irrelevant visual information. In light of this, we propose an Implicit and Explicit Visual Information Aware Network (IEVAN) for Multimodal Relation Extraction. Specifically, the Caption-Aware Module (CAM) hierarchically aligns the fine-grained visual entities and textual entities for reasoning. The Knowledge-Aware Module (KAM) supplements relation extraction with background knowledge. In addition, for preserving crucial information within different modalities, we leverage a Mutual Information Maximization method to regulate the multimodal reasoning modules. Experiments show that our model outperforms the state-of-the-art MRE models on the benchmark dataset MNRE and MORE. Further ablation studies prove the effective performance of our methods.

29 November 2022 in International Journal of Multimedia Information Retrieval
Verified
Journal Article
Closed access
PDF file
ZZZefan Zhang
Zefan Zhang
TJTainling Jiang
Tainling Jiang
CLChunping Liu
Chunping Liu
YJYi Ji
Yi Ji

12 October 2023 in Journal of Visual Communication and Image Representation
Verified
Journal Article
Closed access
ZZZefan Zhang
Zefan Zhang
SLShun Li
Shun Li
YJYi Ji
Yi Ji
CLChunping Liu
Chunping Liu