Wenting Zhao

Wenting Zhao

Auto-generated profile, based on publicly available information

Wenting Zhao

Salesforce Research, Palo Alto, CA, USA

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
19 August 2025 in IEEE Transactions on Software Engineering
Verified
Journal Article
Open access
YWYang Wu
Yang Wu
YWYao Wan
Yao Wan
ZCZhaoyang Chu
Zhaoyang Chu
WZWenting Zhao
Wenting Zhao
YLYe Liu
Ye Liu
PYPhilip S. Yu
Philip S. Yu
Code summarization facilitates program comprehension and software maintenance by converting code snippets into natural-language descriptions. Over the years, numerous methods have been developed for this task, but a key challenge remains: effectively evaluating the quality of generated summaries. While human evaluation is effective for assessing code summary quality, it is labor-intensive and difficult to scale. Commonly used automatic metrics, such as BLEU, ROUGE-L, METEOR, and BERTScore, often fail to align closely with human judgments. In this paper, we explore the potential of Large Language Models (LLMs) for evaluating code summarization. We propose CODERPE (Role-Player for Code Summarization Evaluation), a novel method that leverages role-player prompting to assess the quality of generated summaries. Specifically, we prompt LLM-based evaluators to take on diverse roles, such as code reviewer, code author, code editor, and system analyst. Each role evaluates the quality of code summaries across key dimensions, including coherence, consistency, fluency, and relevance. We further explore the robustness of LLMs as evaluators by employing various prompting strategies, including chain-of-thought reasoning, incontext learning, and tailored rating form designs. The results demonstrate that LLMs serve as effective evaluators for code summarization. Notably, our LLM-based evaluator, CODERPE , achieves an 80.18% Spearman correlation with human evaluations, outperforming the existing BERTScore metric by 10.39%.

13 October 2023 in IEEE Transactions on Knowledge and Data Engineering
Verified
Journal Article
Closed access
SLShuaiqi Liu
Shuaiqi Liu
JCJiannong Cao
Jiannong Cao
ZDZhongfen Deng
Zhongfen Deng
WZWenting Zhao
Wenting Zhao
RYRuosong Yang
Ruosong Yang
PYPhilip S. Yu
Philip S. Yu
Abstractive summarization aims to generate a concise summary covering the input document's salient information. Within a report document, the salient information can be scattered in the textual and non-textual content. However, existing document summarization datasets and methods usually focus on the text and filter out the non-textual content. Missing tabular data can limit produced summaries’ informativeness, especially when summaries require covering quantitative descriptions of critical metrics in tables. Existing datasets and methods cannot meet the requirements of summarizing long text and dozens of tables in each report document. To deal with the scarcity of available datasets, we propose FINDSum, the first large-scale dataset for long text and multi-table summarization. Built on 21,125 annual reports from 3,794 companies, FINDSum has two subsets for summarizing each company's results of operations and liquidity. Besides, we present four types of summarization methods to jointly consider text and table content when summarizing reports. Additionally, we propose a set of evaluation metrics to assess the usage of numerical information in produced summaries. Our summarization methods significantly outperform advanced baselines, which verifies the necessity of incorporating textual and tabular data when summarizing report documents. We also conduct extensive comparative experiments to identify vital model components and configurations that can improve summarization results.