Qingyuan Zhong

Qingyuan Zhong

Auto-generated profile, based on publicly available information

Qingyuan Zhong

Sun Yat-Sen University, China

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
27 October 2024
Verified
Conference Paper
Open access
JCJiachi Chen
Jiachi Chen
QZQingyuan Zhong
Qingyuan Zhong
YWYanlin Wang
Yanlin Wang
KNKaiwen Ning
Kaiwen Ning
YLYongkun Liu
Yongkun Liu
ZZZibin Zheng
Zibin Zheng
Warning: Please note that this article contains potential harmful or offensive content. This content is only for the evaluating and analysis of LLMs and does not imply any intention to promote criminal activities. The emergence of Large Language Models (LLMs) has significantly influenced various aspects of software development activities. Despite their benefits, LLMs also pose notable risks, including the potential to generate harmful content and being abused by malicious developers to create malicious code. Several previous studies have focused on the ability of LLMs to resist the generation of harmful content that violates human ethical standards, such as biased or offensive content. However, there is no research evaluating the ability of LLMs to resist malicious code generation. To fill this gap, we propose RMCBench, the first benchmark comprising 473 prompts designed to assess the ability of LLMs to resist malicious code generation. This benchmark employs two scenarios: a text-to-code scenario, where LLMs are prompted with descriptions to generate code, and a code-to-code scenario, where LLMs translate or complete existing malicious code. Based on RMCBench, we conduct an empirical study on the 11 representative LLMs to assess their ability to resist malicious code generation. Our findings indicate that current LLMs have a limited ability to resist malicious code generation with an average refusal rate of 40.36% in text-to-code scenario and 11.52% in code-to-code scenario. The average refusal rate of all LLMs in RMCBench is only 28.71%; ChatGPT-4 has a refusal rate of only 35.73%. We also analyze the factors that affect LLM's ability to resist malicious code generation and provide implications for developers to enhance model robustness.

6 October 2026 in ACM Transactions on Software Engineering and Methodology
Verified
Journal Article
Open access
KNKaiwen Ning
Kaiwen Ning
JCJiachi Chen
Jiachi Chen
QZQingyuan Zhong
Qingyuan Zhong
Tao ZhangTao Zhang
Tao Zhang
YWYanlin Wang
Yanlin Wang
ZZZibin Zheng
Zibin Zheng
With the advent of large language models (LLMs), numerous software service providers are developing LLMs tailored for code generation, such as CodeLlama. However, these models can be exploited by malicious developers to generate malicious code, posing severe threats to the software ecosystem. To address this issue, we first conducted an empirical study and built MCGTest , a dataset of \(406\) prompts designed to elicit malicious code from LLMs. Leveraging this dataset, we propose MCGMark , a watermarking method to trace and attribute LLM-generated malicious code. MCGMark subtly embeds user-specific information into generated code by controlling the token selection process, ensuring the watermark is imperceptible. Additionally, MCGMark dynamically adjusts the token selection range to induce the LLM to favor high-probability tokens, thus ensuring code quality. Furthermore, by leveraging code structure, MCGMark avoids embedding watermarks into regions easily modified by attackers, such as comments and variable names, enhancing robustness against tampering. Experiments on several advanced LLMs show that MCGMark successfully embeds watermarks in approximately \(85\%\) of cases, under the constraint of a \(400\) -token limit. Moreover, it maintains code quality and demonstrates strong resilience against common code modification. This approach offers a practical solution for tracing malicious code and mitigating the misuse of LLMs.