Renyi Zhong

Renyi Zhong

Auto-generated profile, based on publicly available information

Renyi Zhong

The Chinese University of Hong Kong, Hong Kong

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
8 October 2024 in IEEE Transactions on Software Engineering
Verified
Journal Article
Closed access
YLYichen Li
Yichen Li
YHYintong Huo
Yintong Huo
ZJZhihan Jiang
Zhihan Jiang
RZRenyi Zhong
Renyi Zhong
PHPinjia He
Pinjia He
MLMichael R. Lyu
Michael R. Lyu
Automated logging statement generation supports developers in documenting critical software runtime behavior. While substantial recent research has focused on retrieval-based and learning-based methods, results suggest they fail to provide appropriate logging statements in real-world complex software. Given the great success in natural language generation and programming language comprehension, large language models (LLMs) might help developers generate logging statements, but this has not yet been investigated. To fill the gap, this paper performs the first study on exploring LLMs for logging statement generation. We first build a logging statement generation dataset, LogBench, with two parts: (1) LogBench-O: 3,870 methods with 6,849 logging statements collected from GitHub repositories, and (2) LogBench-T: the transformed unseen code from LogBench-O. Then, we leverage LogBench to evaluate the effectiveness and generalization capabilities (using LogBench-T) of 13 top-performing LLMs, from 60M to 405B parameters. In addition, we examine the performance of these LLMs against classical retrieval-based and machine learning-based logging methods from the era preceding LLMs. Specifically, we evaluate the logging effectiveness of LLMs by studying their ability to determine logging ingredients and the impact of prompts and external program information.We further evaluate LLM’s logging generalization capabilities using unseen data (LogBench-T) derived from code transformation techniques. While existing LLMs deliver decent predictions on logging levels and logging variables, our study indicates that they only achieve a maximum BLEU score of 0.249, thus calling for improvements. The paper also highlights the importance of prompt constructions and external factors (e.g., programming contexts and code comments) for LLMs’ logging performance. In addition, we observed that existing LLMs show a significant performance drop (8.2%-16.2% decrease) when dealing with logging unseen code, revealing their unsatisfactory generalization capabilities. Based on these findings, we identify five implications and provide practical advice for future logging research. Our empirical analysis discloses the limitations of current logging approaches while showcasing the potential of LLM-based logging tools, and provides actionable guidance for building more practical models.

12 July 2024 in Proceedings of the ACM on Software Engineering
Verified
Journal Article
Open access
YLYichen Li
Yichen Li
YHYintong Huo
Yintong Huo
RZRenyi Zhong
Renyi Zhong
ZJZhihan Jiang
Zhihan Jiang
JLJinyang Liu
Jinyang Liu
MLMichael R. Lyu
Michael R. Lyu
Logging practices have been extensively investigated to assist developers in writing appropriate logging statements for documenting software behaviors. Although numerous automatic logging approaches have been proposed, their performance remains unsatisfactory due to the constraint of the single-method input, without informative programming context outside the method. Specifically, we identify three inherent limitations with single-method context: limited static scope of logging statements, inconsistent logging styles, and missing type information of logging variables. To tackle these limitations, we propose SCLogger , the first contextualized logging statement generation approach with inter-method static contexts. First, SCLogger extracts inter-method contexts with static analysis to construct the contextualized prompt for language models to generate a tentative logging statement. The contextualized prompt consists of an extended static scope and sampled similar methods, ordered by the chain-of-thought (COT) strategy. Second, SCLogger refines the access of logging variables by formulating a new refinement prompt for language models, which incorporates detailed type information of variables in the tentative logging statement. The evaluation results show that SCLogger surpasses the state-of-the-art approach by 8.7% in logging position accuracy, 32.1% in level accuracy, 19.6% in variable precision, and 138.4% in text BLEU-4 score. Furthermore, SCLogger consistently boosts the performance of logging statement generation across a range of large language models, thereby showcasing the generalizability of this approach. CCS Concepts: • Software and its engineering → Maintaining software .

23 June 2025
Verified
Conference Paper
Open access
ZJZhihan Jiang
Zhihan Jiang
JHJunjie Huang
Junjie Huang
GYGuangba Yu
Guangba Yu
ZCZhuangbin Chen
Zhuangbin Chen
YLYichen Li
Yichen Li
MLMichael Lyu
Michael Lyu

14 April 2024
Verified
Conference Paper
Open access
JKJinxi Kuang
Jinxi Kuang
JLJinyang Liu
Jinyang Liu
JHJunjie Huang
Junjie Huang
RZRenyi Zhong
Renyi Zhong
JGJiazhen Gu
Jiazhen Gu
MLMichael R. Lyu
Michael R. Lyu
Due to the scale and complexity of cloud systems, a system failure would trigger an "alert storm", i.e., massive correlated alerts. Although these alerts can be traced back to a few root causes, the overwhelming number makes it infeasible for manual handling. Alert aggregation is thus critical to help engineers concentrate on the root cause and facilitate failure resolution. Existing methods typically utilize semantic similarity-based methods or statistical methods to aggregate alerts. However, semantic similarity-based methods overlook the causal rationale of alerts, while statistical methods can hardly handle infrequent alerts. To tackle these limitations, we introduce leveraging external knowledge, i.e., Standard Operation Procedure (SOP) of alerts as a supplement. We propose COLA, a novel hybrid approach based on correlation mining and LLM (Large Language Model) reasoning for online alert aggregation. The correlation mining module effectively captures the temporal and spatial relations between alerts, measuring their correlations in an efficient manner. Subsequently, only uncertain pairs with low confidence are forwarded to the LLM reasoning module for detailed analysis. This hybrid design harnesses both statistical evidence for frequent alerts and the reasoning capabilities of computationally intensive LLMs, ensuring the overall efficiency of COLA in handling large volumes of alerts in practical scenarios. We evaluate COLA on three datasets collected from the production environment of a large-scale cloud platform. The experimental results show COLA achieves F1-scores from 0.901 to 0.930, outperforming state-of-the-art methods and achieving comparable efficiency. We also share our experience in deploying COLA in our real-world cloud system, Cloud X1.

11 December 2025 in ACM Transactions on Software Engineering and Methodology
Verified
Journal Article
Open access
RZRenyi Zhong
Renyi Zhong
YLYichen Li
Yichen Li
JKJinxi Kuang
Jinxi Kuang
WGWenwei Gu
Wenwei Gu
YHYintong Huo
Yintong Huo
MLMichael R. Lyu
Michael R. Lyu
Developers write logging statements to monitor software runtime behaviors and system state. However, poorly constructed or misleading log messages can inadvertently obfuscate actual program execution patterns, thereby impeding effective software maintenance. Existing research on analyzing issues within logging statements is limited, primarily focusing on detecting a singular type of defect and relying on manual intervention for fixes rather than automated solutions. To address the limitation, we initiate a systematic study that pinpoints four specific types of defects in logging statements (i.e., statement code inconsistency, static dynamic inconsistency, temporal relation inconsistency, and readability issues) through the analysis of real-world log-centric changes. We then propose LogUpdater , a two-stage framework for automatically detecting and updating logging statements for these specific defects. In the offline stage, LogUpdater constructs a similarity-based classifier on a set of synthetic defective logging statements to identify specific defect types. During the online testing phase, this classifier first evaluates logging statements in a given code snippet to determine the necessity and type of improvements required. Then, LogUpdater constructs type-aware prompts from historical logging update changes for an LLM-based recommendation framework to suggest updates addressing these specific defects. We evaluate the effectiveness of LogUpdater on a dataset containing real-world logging changes, a synthetic dataset, and a new real-world project dataset. The results indicate that our approach is highly effective in detecting logging defects, achieving an F1-score of 0.625. Additionally, it exhibits significant improvements in suggesting precise static text and dynamic variables, with enhancements of 48.12% and 24.90%, respectively. Furthermore, LogUpdater achieves a 61.49% success rate in recommending correct updates on new real-world projects. We reported 40 problematic logging statements and their fixes to GitHub via pull requests, resulting in 25 changes confirmed and merged across 11 different projects.

11 July 2026 in ACM Transactions on Software Engineering and Methodology
Verified
Journal Article
Open access
RZRenyi Zhong
Renyi Zhong
YLYichen Li
Yichen Li
GYGuangba Yu
Guangba Yu
WGWenwei Gu
Wenwei Gu
JKJinxi Kuang
Jinxi Kuang
MLMichael R. Lyu
Michael R. Lyu
Developers use logging statements to create logs that document system behavior and aid in software maintenance. As such, high-quality logging is essential for effective maintenance; however, manual logging often leads to errors and inconsistency. Recent methods emphasize using Large Language Models (LLMs) for automated logging statement generation, but these present privacy and resource issues, hindering their suitability for enterprise use. This article presents the first large-scale empirical study evaluating Small Open-Source Language Models (SOLMs) for automated logging statement generation. We evaluate four prominent SOLMs using various prompt strategies and parameter-efficient fine-tuning techniques, such as Low-Rank Adaptation (LoRA) and Retrieval-Augmented Generation (RAG). Our results show that fine-tuned SOLMs with LoRA and RAG prompts, particularly Qwen2.5-coder-14B, outperform existing tools and LLM baselines (e.g., Claude3.7 sonnet and GPT4o) in predicting logging locations and generating high-quality statements, with robust generalization across diverse repositories. These findings highlight SOLMs as a privacy-preserving, efficient alternative for automated logging.