Zheng Li

Zheng Li

Auto-generated profile, based on publicly available information

Zheng Li

School of Cyber Science and Technology, the State Key Laboratory of Cryptography and Digital Economy Security, and Shandong Key Laboratory of Artificial Intelligence Security, Shandong University, Qingdao, China

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
15 November 2023
Verified
Conference Paper
Open access
ZSZeyang Sha
Zeyang Sha
ZLZheng Li
Zheng Li
NYNing Yu
Ning Yu
YZYang Zhang
Yang Zhang
Text-to-image generation models that generate images based on prompt descriptions have attracted an increasing amount of attention during the past few months. Despite their encouraging performance, these models raise concerns about the misuse of their generated fake images. To tackle this problem, we pioneer a systematic study on the detection and attribution of fake images generated by text-to-image generation models. Concretely, we first build a machine learning classifier to detect the fake images generated by various text-to-image generation models. We then attribute these fake images to their source models, such that model owners can be held responsible for their models' misuse. We further investigate how prompts that generate fake images affect detection and attribution. We conduct extensive experiments on four popular text-to-image generation models, including DALL·E 2, Stable Diffusion, GLIDE, and Latent Diffusion, and two benchmark prompt-image datasets. Empirical results show that (1) fake images generated by various models can be distinguished from real ones, as there exists a common artifact shared by fake images from different models; (2) fake images can be effectively attributed to their source models, as different models leave unique fingerprints in their generated images; (3) prompts with the "person'' topic or a length between 25 and 75 enable models to generate fake images with higher authenticity. All findings contribute to the community's insight into the threats caused by text-to-image generation models. We appeal to the community's consideration of the counterpart solutions, like ours, against the rapidly-evolving fake image generation.

9 December 2019
Verified
Conference Paper
Open access
PDF file
ZLZheng Li
Zheng Li
CHChengyu Hu
Chengyu Hu
YZYang Zhang
Yang Zhang
SGShanqing Guo
Shanqing Guo

2 December 2024
Verified
Conference Paper
Open access
RWRui Wen
Rui Wen
ZLZheng Li
Zheng Li
MBMichael Backes
Michael Backes
YZYang Zhang
Yang Zhang
Adapting Large Language Models (LLMs) to specific tasks introduces concerns about computational efficiency, prompting an exploration of efficient methods such as In-Context Learning (ICL). However, the vulnerability of ICL to privacy attacks under realistic assumptions remains largely unexplored. In this work, we present the first membership inference attack tailored for ICL, relying solely on generated texts without their associated probabilities. We propose four attack strategies tailored to various constrained scenarios and conduct extensive experiments on four popular large language models. Empirical results show that our attacks can accurately determine membership status in most cases, e.g., 95% accuracy advantage against LLaMA, indicating that the associated risks are much higher than those shown by existing probability-based attacks. Additionally, we propose a hybrid attack that synthesizes the strengths of the aforementioned strategies, achieving an accuracy advantage of over 95% in most cases. Furthermore, we investigate three potential defenses targeting data, instruction, and output. Results demonstrate combining defenses from orthogonal dimensions significantly reduces privacy leakage and offers enhanced privacy assurances.

2 December 2024
Verified
Conference Paper
Open access
JZJinghuai Zhang
Jinghuai Zhang
JCJianfeng Chi
Jianfeng Chi
ZLZheng Li
Zheng Li
KCKunlin Cai
Kunlin Cai
YZYang Zhang
Yang Zhang
YTYuan Tian
Yuan Tian
Fine-tuning pre-trained models for downstream tasks has led to a proliferation of open-sourced task-specific models. Recently, Model Merging (MM) has emerged as an effective approach to facilitate knowledge transfer among these independently fine-tuned models. MM directly combines multiple fine-tuned task-specific models into a merged model without additional training, and the resulting model shows enhanced capabilities in multiple tasks. Although MM provides great utility, it may come with security risks because an adversary can exploit MM to affect multiple downstream tasks. However, the security risks of MM have barely been studied. In this paper, we first find that MM, as a new learning paradigm, introduces unique challenges for existing backdoor attacks due to the merging process. To address these challenges, we introduce BadMerging, the first backdoor attack specifically designed for MM. Notably, BadMerging allows an adversary to compromise the entire merged model by contributing as few as one backdoored task-specific model. BadMerging comprises a two-stage attack mechanism and a novel feature-interpolation-based loss to enhance the robustness of embedded backdoors against the changes of different merging parameters. Considering that a merged model may incorporate tasks from different domains, BadMerging can jointly compromise the tasks provided by the adversary (on-task attack) and other contributors (off-task attack) and solve the corresponding unique challenges with novel attack designs. Extensive experiments show that BadMerging achieves remarkable attacks against various MM algorithms. Our ablation study demonstrates that the proposed attack designs can progressively contribute to the attack performance. Finally, we show that prior defense mechanisms fail to defend against our attacks, highlighting the need for more advanced defense. Our code is available at: https://github.com/jzhang538/BadMerging.

2 December 2024
Verified
Conference Paper
Open access
HLHao Li
Hao Li
ZLZheng Li
Zheng Li
SWSiyuan Wu
Siyuan Wu
CHChengrui Hu
Chengrui Hu
YYYutong Ye
Yutong Ye
YZYang Zhang
Yang Zhang
Most existing membership inference attacks (MIAs) utilize metrics (e.g., loss) calculated on the model's final state, while recent advanced attacks leverage metrics computed at various stages, including both intermediate and final stages, throughout the model training. Nevertheless, these attacks often process multiple intermediate states of the metric independently, ignoring their time-dependent patterns. Consequently, they struggle to effectively distinguish between members and non-members who exhibit similar metric values, particularly resulting in a high false-positive rate. In this study, we delve deeper into the new membership signals in the black-box scenario. We identify a new, more integrated membership signal: the Pattern of Metric Sequence, derived from the various stages of model training. We contend that current signals provide only partial perspectives of this new signal: the new one encompasses both the model's multiple intermediate and final states, with a greater emphasis on temporal patterns among them. Building upon this signal, we introduce a novel attack method called Sequential-metric based Membership Inference Attack (SeqMIA). Specifically, we utilize knowledge distillation to obtain a set of distilled models representing various stages of the target model's training. We then assess multiple metrics on these distilled models in chronological order, creating distilled metric sequence. We finally integrate distilled multi-metric sequences as a sequential multiformat and employ an attention-based RNN attack model for inference. Empirical results show SeqMIA outperforms all baselines, especially can achieve an order of magnitude improvement in terms of TPR @ 0.1% FPR. Furthermore, we delve into the reasons why this signal contributes to SeqMIA's high attack performance, and assess various defense mechanisms against SeqMIA.

22 February 2025 in ACM Transactions on Privacy and Security
Verified
Journal Article
Open access
YDYingkai Dong
Yingkai Dong
LWLi Wang
Li Wang
ZLZheng Li
Zheng Li
HLHao Li
Hao Li
PTPeng Tang
Peng Tang
SGShanqing Guo
Shanqing Guo
The prediction module, powered by deep learning models, constitutes a fundamental component of high-level Autonomous Vehicles (AVs). Given the direct influence of the module’s prediction accuracy on AV driving behavior, ensuring its security is paramount. However, limited studies have explored the adversarial robustness of the prediction modules. Furthermore, existing methods still generate adversarial trajectories that deviate significantly from human driving behavior. These deviations can be easily identified as hazardous by AVs’ anomaly detection models and thus cannot effectively evaluate and reflect the robustness of the prediction modules. To bridge this gap, we propose a stealthy and more effective optimization-based attack method. Specifically, we reformulate the optimization problem using Lagrangian relaxation and design a Frenet-based objective function along with a distinct constraint space. We conduct extensive evaluations on 2 popular prediction models and 2 benchmark datasets. Our results show that our attack is highly effective, with over 87% attack success rates, outperforming all baseline attacks. Moreover, our attack method significantly improves the stealthiness of adversarial trajectories while guaranteeing adherence to physical constraints. Our attack is also found robust to noise from upstream modules, transferable across trajectory prediction models, and high realizability. Lastly, to verify its effectiveness in real-world applications, we conduct further simulation evaluations using a production-grade simulator. These simulations reveal that the adversarial trajectory we created could convincingly induce autonomous vehicles (AVs) to initiate hard braking.