Linli Xu

Linli Xu

Auto-generated profile, based on publicly available information

Linli Xu

University of Science and Technology of China, Hefei, China and State Key Laboratory of Cognitive Intelligence, Hefei, China

Research Focus & Metrics

Academic Career Length

16Years (2011-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 January 2024
Verified
Conference Paper
Open access
PDF file
ZGZhujin Gao
Zhujin Gao
JGJunliang Guo
Junliang Guo
XTXu Tan
Xu Tan
YZYongxin Zhu
Yongxin Zhu
FZFang Zhang
Fang Zhang
LXLinli Xu
Linli Xu
Zhujin Gao, Junliang Guo, Xu Tan, Yongxin Zhu, Fang Zhang, Jiang Bian, Linli Xu. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

4 August 2023
Verified
Conference Paper
Open access
PLPengfei Luo
Pengfei Luo
Tong XuTong Xu
Tong Xu
SWShiwei Wu
Shiwei Wu
CZChen Zhu
Chen Zhu
LXLinli Xu
Linli Xu
ECEnhong Chen
Enhong Chen
Multimodal entity linking (MEL) task, which aims at resolving ambiguous mentions to a multimodal knowledge graph, has attracted wide attention in recent years. Though large efforts have been made to explore the complementary effect among multiple modalities, however, they may fail to fully absorb the comprehensive expression of abbreviated textual context and implicit visual indication. Even worse, the inevitable noisy data may cause inconsistency of different modalities during the learning process, which severely degenerates the performance. To address the above issues, in this paper, we propose a novel Multi-GraIned Multimodal InteraCtion Network (MIMIC) framework for solving the MEL task. Specifically, the unified inputs of mentions and entities are first encoded by textual/visual encoders separately, to extract global descriptive features and local detailed features. Then, to derive the similarity matching score for each mention-entity pair, we device three interaction units to comprehensively explore the intra-modal interaction and inter-modal fusion among features of entities and mentions. In particular, three modules, namely the Text-based Global-Local interaction Unit (TGLU), Vision-based DuaL interaction Unit (VDLU) and Cross-Modal Fusion-based interaction Unit (CMFU) are designed to capture and integrate the fine-grained representation lying in abbreviated text and implicit visual cues. Afterwards, we introduce a unit-consistency objective function via contrastive learning to avoid inconsistency and model degradation. Experimental results on three public benchmark datasets demonstrate that our solution outperforms various state-of-the-art baselines, and ablation studies verify the effectiveness of designed modules.

1 January 2023
Verified
Conference Paper
Open access
PDF file
MZMao Zhang
Mao Zhang
YZYongxin Zhu
Yongxin Zhu
ZLZhen Liu
Zhen Liu
ZBZhimin Bao
Zhimin Bao
YWYunfei Wu
Yunfei Wu
LXLinli Xu
Linli Xu
Mao Zhang, Yongxin Zhu, Zhen Liu, Zhimin Bao, Yunfei Wu, Xing Sun, Linli Xu. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

28 October 2024
Verified
Conference Paper
Open access
YWYubo Wang
Yubo Wang
CLChaohu Liu
Chaohu Liu
YQYanqiu Qu
Yanqiu Qu
HCHaoyu Cao
Haoyu Cao
DJDeqiang Jiang
Deqiang Jiang
LXLinli Xu
Linli Xu
Large vision-language models (LVLMs) integrate visual information into large language models, showcasing remarkable multi-modal conversational capabilities. However, the visual modules introduces new challenges in terms of robustness for LVLMs, as attackers can craft adversarial images that are visually clean but may mislead the model to generate incorrect answers. In general, LVLMs rely on vision encoders to transform images into visual tokens, which are crucial for the language models to perceive image contents effectively. Therefore, we are curious about one question: Can LVLMs still generate correct responses when the encoded visual tokens are attacked and disrupting the visual information? To this end, we propose a non-targeted attack method referred to as VT-Attack (Visual Tokens Attack), which constructs adversarial examples from multiple perspectives, with the goal of comprehensively disrupting feature representations and inherent relationships as well as the semantic properties of visual tokens output by image encoders. Using only access to the image encoder in the proposed attack, the generated adversarial examples exhibit transferability across diverse LVLMs utilizing the same image encoder and generality across different tasks. Extensive experiments validate the superior attack performance of the VT-Attack over baseline methods, demonstrating its effectiveness in attacking LVLMs with image encoders, which in turn can provide guidance on the robustness of LVLMs, particularly in terms of the stability of the visual feature space.

1 January 2024
Verified
Conference Paper
Open access
PDF file
HYHaoqiu Yan
Haoqiu Yan
YZYongxin Zhu
Yongxin Zhu
KZKai Zheng
Kai Zheng
BLBing Liu
Bing Liu
HCHaoyu Cao
Haoyu Cao
LXLinli Xu
Linli Xu
Haoqiu Yan, Yongxin Zhu, Kai Zheng, Bing Liu, Haoyu Cao, Deqiang Jiang, Linli Xu. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.