Yuqi Jia

Yuqi Jia

Auto-generated profile, based on publicly available information

Yuqi Jia

Duke University, Durham, NC, USA

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications


23 May 2025 in arXiv
Verified
Preprint
Open access
PDF file
YJYuqi Jia
Yuqi Jia
ZSZedian Shao
Zedian Shao
YLYupei Liu
Yupei Liu
JJJinyuan Jia
Jinyuan Jia
DSDawn Song
Dawn Song
NGNeil Zhenqiang Gong
Neil Zhenqiang Gong
Large Language Models (LLMs) are vulnerable to prompt injection attacks, and several defenses have recently been proposed, often claiming to mitigate these attacks successfully. However, we argue that existing studies lack a principled approach to evaluating these defenses. In this paper, we argue the need to assess defenses across two critical dimensions: (1) effectiveness, measured against both existing and adaptive prompt injection attacks involving diverse target and injected prompts, and (2) general-purpose utility, ensuring that the defense does not compromise the foundational capabilities of the LLM. Our critical evaluation reveals that prior studies have not followed such a comprehensive evaluation methodology. When assessed using this principled approach, we show that existing defenses are not as successful as previously reported. This work provides a foundation for evaluating future defenses and guiding their development. Our code and data are available at: https://github.com/PIEval123/PIEval.

14 November 2026
Verified
Conference Paper
Open access
YLYinuo Liu
Yinuo Liu
XWXilong Wang
Xilong Wang
RXRuohan Xu
Ruohan Xu
YJYuqi Jia
Yuqi Jia
NGNeil Gong
Neil Gong

1 October 2025 in arXiv
Verified
Preprint
Open access
PDF file
YLYinuo Liu
Yinuo Liu
RXRuohan Xu
Ruohan Xu
XWXilong Wang
Xilong Wang
YJYuqi Jia
Yuqi Jia
NGNeil Zhenqiang Gong
Neil Zhenqiang Gong
Multiple prompt injection attacks have been proposed against web agents. At the same time, various methods have been developed to detect general prompt injection attacks, but none have been systematically evaluated for web agents. In this work, we bridge this gap by presenting the first comprehensive benchmark study on detecting prompt injection attacks targeting web agents. We begin by introducing a fine-grained categorization of such attacks based on the threat model. We then construct datasets containing both malicious and benign samples: malicious text segments generated by different attacks, benign text segments from four categories, malicious images produced by attacks, and benign images from two categories. Next, we systematize both text-based and image-based detection methods. Finally, we evaluate their performance across multiple scenarios. Our key findings show that while some detectors can identify attacks that rely on explicit textual instructions or visible image perturbations with moderate to high accuracy, they largely fail against attacks that omit explicit instructions or employ imperceptible perturbations. Our datasets and code are released at: https://github.com/Norrrrrrr-lyn/WAInjectBench.