Zhaoyang Chu

Zhaoyang Chu

Auto-generated profile, based on publicly available information

Zhaoyang Chu

Huazhong University of Science and Technology, Wuhan, China

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
11 September 2024
Verified
Conference Paper
Open access
ZCZhaoyang Chu
Zhaoyang Chu
YWYao Wan
Yao Wan
QLQing Li
Qing Li
YWYang Wu
Yang Wu
Hongyu ZhangHongyu Zhang
Hongyu Zhang
HJHai Jin
Hai Jin
Vulnerability detection is crucial for ensuring the security and reliability of software systems. Recently, Graph Neural Networks (GNNs) have emerged as a prominent code embedding approach for vulnerability detection, owing to their ability to capture the underlying semantic structure of source code. However, GNNs face significant challenges in explainability due to their inherently black-box nature. To this end, several factual reasoning-based explainers have been proposed. These explainers provide explanations for the predictions made by GNNs by analyzing the key features that contribute to the outcomes. We argue that these factual reasoning-based explanations cannot answer critical what-if questions: "What would happen to the GNN's decision if we were to alter the code graph into alternative structures?" Inspired by advancements of counterfactual reasoning in artificial intelligence, we propose CFExplainer, a novel counterfactual explainer for GNN-based vulnerability detection. Unlike factual reasoning-based explainers, CFExplainer seeks the minimal perturbation to the input code graph that leads to a change in the prediction, thereby addressing the what-if questions for vulnerability detection. We term this perturbation a counterfactual explanation, which can pinpoint the root causes of the detected vulnerability and furnish valuable insights for developers to undertake appropriate actions for fixing the vulnerability. Extensive experiments on four GNN-based vulnerability detection models demonstrate the effectiveness of CFExplainer over existing state-of-the-art factual reasoning-based explainers.

19 August 2025 in IEEE Transactions on Software Engineering
Verified
Journal Article
Open access
YWYang Wu
Yang Wu
YWYao Wan
Yao Wan
ZCZhaoyang Chu
Zhaoyang Chu
WZWenting Zhao
Wenting Zhao
YLYe Liu
Ye Liu
PYPhilip S. Yu
Philip S. Yu
Code summarization facilitates program comprehension and software maintenance by converting code snippets into natural-language descriptions. Over the years, numerous methods have been developed for this task, but a key challenge remains: effectively evaluating the quality of generated summaries. While human evaluation is effective for assessing code summary quality, it is labor-intensive and difficult to scale. Commonly used automatic metrics, such as BLEU, ROUGE-L, METEOR, and BERTScore, often fail to align closely with human judgments. In this paper, we explore the potential of Large Language Models (LLMs) for evaluating code summarization. We propose CODERPE (Role-Player for Code Summarization Evaluation), a novel method that leverages role-player prompting to assess the quality of generated summaries. Specifically, we prompt LLM-based evaluators to take on diverse roles, such as code reviewer, code author, code editor, and system analyst. Each role evaluates the quality of code summaries across key dimensions, including coherence, consistency, fluency, and relevance. We further explore the robustness of LLMs as evaluators by employing various prompting strategies, including chain-of-thought reasoning, incontext learning, and tailored rating form designs. The results demonstrate that LLMs serve as effective evaluators for code summarization. Notably, our LLM-based evaluator, CODERPE , achieves an 80.18% Spearman correlation with human evaluations, outperforming the existing BERTScore metric by 10.39%.

12 April 2026
Verified
Conference Paper
Open access
ZCZhaoyang Chu
Zhaoyang Chu
YWYao Wan
Yao Wan
ZZZhikun Zhang
Zhikun Zhang
DWDi Wang
Di Wang
ZYZhou Yang
Zhou Yang
DLDavid Lo
David Lo

30 June 2026 in Proceedings of the ACM on Software Engineering
Verified
Journal Article
Open access
GWGuanghua Wan
Guanghua Wan
YFYuanning Feng
Yuanning Feng
YWYao Wan
Yao Wan
ZCZhaoyang Chu
Zhaoyang Chu
ZBZhangqian Bi
Zhangqian Bi
HJHai Jin
Hai Jin
Code summarization plays a vital role in program comprehension and software maintenance by generating natural language descriptions to summarize the semantics of code. While Large Language Models (LLMs) have shown remarkable performance in this area, recent empirical studies reveal a critical limitation: LLMs are prone to hallucinations, producing summaries that are factually inaccurate or unfaithful to the source code, potentially misleading developers. In this paper, we propose to unveil, detect, and mitigate hallucinations in LLM-based code summarization. First, we construct Hallu-Eval, a novel dataset for unveiling hallucination phenomena and rigorously evaluating the effectiveness of hallucination detection and mitigation in LLM-based code summarization. It comprises both original code snippets to capture naturally occurring hallucinations and their semantically perturbed counterparts, which are designed to systematically induce challenging logical hallucinations, all complemented with manual hallucination annotations on a curated testbed of 800 code-summary pairs. Next, we propose Hallu-Det, a synergistic approach that combines direct entity-level detection to identify explicit hallucinations with a synonymous mutation-based refinement to reliably confirm or refute more ambiguous cases. Finally, we introduce Hallu-Shield, an inference-time mitigation approach that leverages an external value model to guide LLMs toward producing more faithful summaries without costly retraining of the LLM itself. Extensive experiments show that Hallu-Eval effectively triggers hallucinations, increasing the hallucination rate of models such as Qwen2.5-Coder-7B from 17% to 97% on perturbed code. Our detection approach, Hallu-Det, achieves the best performance among baselines, reaching an F1-score of 0.95 for summaries generated by Qwen2.5-Coder-7B. Moreover, our mitigation method, Hallu-Shield, reduces hallucination rates. For example, it lowers the rate from 66% to 59%, a 10.6% relative reduction, on DeepSeek-Coder-6.7B, while simultaneously improving summary quality, achieving a 74.0% win rate evaluated by an LLM-as-a-judge majority vote ensemble.