Yuan Zhao

Yuan Zhao

Auto-generated profile, based on publicly available information

Yuan Zhao

State Key Laboratory for Novel Software Technology, Nanjing University, China

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
28 February 2025 in ACM Transactions on Software Engineering and Methodology
Verified
Journal Article
Open access
QZQuanjun Zhang
Quanjun Zhang
Chunrong FangChunrong Fang
Chunrong Fang
YZYi Zheng
Yi Zheng
YZYaxin Zhang
Yaxin Zhang
YZYuan Zhao
Yuan Zhao
Zhenyu ChenZhenyu Chen
Zhenyu Chen
Unit testing validates the correctness of the units of the software system under test and serves as the cornerstone in improving software quality and reliability. To reduce manual efforts in writing unit tests, some techniques have been proposed to generate test assertions automatically, including deep learning (DL)-based, retrieval-based, and integration-based ones. Among them, recent integration-based approaches inherit from both DL-based and retrieval-based approaches and are considered state-of-the-art. Despite being promising, such integration-based approaches suffer from inherent limitations, such as retrieving assertions with lexical matching while ignoring meaningful code semantics, and generating assertions with a limited training corpus. In this paper, we propose a novel Retri eval-Augmented Deep Assertion Gen eration approach, namely RetriGen, based on a hybrid assertion retriever and a pre-trained language model (PLM)-based assertion generator. Given a focal-test, RetriGen first builds a hybrid assertion retriever to search for the most relevant test-assert pair from external codebases. The retrieval process takes both lexical similarity and semantical similarity into account via a token-based and an embedding-based retriever, respectively. RetriGen then treats assertion generation as a sequence-to-sequence task and designs a PLM-based assertion generator to predict a correct assertion with historical test-assert pairs and the retrieved external assertion. Although our concept is general and can be adapted to various off-the-shelf encoder-decoder PLMs, we implement RetriGen to facilitate assertion generation based on the recent CodeT5 model. We conduct extensive experiments to evaluate RetriGen against six state-of-the-art approaches across two large-scale datasets and two metrics. The experimental results demonstrate that RetriGen achieves 57.66% and 73.24% in terms of accuracy and CodeBLEU, outperforming all baselines with an average improvement of 50.66% and 14.14%, respectively. Furthermore, RetriGen generates 1598 and 1818 unique correct assertions that all baselines fail to produce, 3.71X and 4.58X more than the most recent approach EditAS . We also demonstrate that adopting other PLMs can provide substantial advancement, e.g., four additionally-utilized PLMs outperform EditAS by 7.91% \(\sim\) 12.70% accuracy improvement, indicating the generalizability of RetriGen. Overall, our study highlights the promising future of fine-tuning off-the-shelf PLMs to generate accurate assertions by incorporating external knowledge sources.

26 February 2025 in IEEE Transactions on Software Engineering
Verified
Journal Article
Closed access
QZQuanjun Zhang
Quanjun Zhang
Chunrong FangChunrong Fang
Chunrong Fang
YZYi Zheng
Yi Zheng
RQRuixiang Qian
Ruixiang Qian
SYShengcheng Yu
Shengcheng Yu
Zhenyu ChenZhenyu Chen
Zhenyu Chen
Unit testing attempts to validate the correctness of basic units of the software system under test and has a crucial role in software development and testing. However, testing experts have to spend a huge amount of effort to write unit test cases manually. Very recent work proposes a retrieve-and-edit approach to automatically generate unit test oracles, i.e., assertions. Despite being promising, it is still far from perfect due to some limitations, such as splitting assertion retrieval and generation into two separate components without benefiting each other. In this paper, we propose AG-RAG, a retrieval-augmented automated assertion generation (AG) approach that leverages external codebases and joint training to address various technical limitations of prior work. Inspired by the plastic surgery hypothesis, AG-RAG attempts to combine relevant unit tests and advanced pre-trained language models (PLMs) with retrieval-augmented fine-tuning. The key insight of AG-RAG is to simultaneously optimize the retriever and the generator as a whole pipeline with a joint training strategy, enabling them to learn from each other. Particularly, AG-RAG builds a dense retriever to search for relevant test-assert pairs (TAPs) with semantic matching and a retrieval-augmented generator to synthesize accurate assertions with the focal-test and retrieved TAPs as input. Besides, AG-RAG leverages a code-aware language model CodeT5 as the cornerstone to facilitate both assertion retrieval and generation tasks. Furthermore, AG-RAG designs a joint training strategy that allows the retriever to learn from the feedback provided by the generator. This unified design fully adapts both components specifically for retrieving more useful TAPs, thereby generating accurate assertions. AG-RAG is a generic framework that can be adapted to various off-the-shelf PLMs. We extensively evaluate AG-RAG against six state-of-the-art AG approaches on two benchmarks and three metrics. Experimental results show that AG-RAG significantly outperforms previous AG approaches on all benchmarks and metrics, e.g., improving the most recent baseline EditAS by 20.82% and 26.98% in terms of accuracy. AG-RAG also correctly generates 1739 and 2866 unique assertions that all baselines fail to generate, 3.45X and 9.20X more than EditAS. We further demonstrate the positive contribution of our joint training strategy, e.g., AG-RAG improving a variant without the retriever by an average accuracy of 14.11%. Besides, adopting other PLMs can provide substantial advancement, e.g., AG-RAG with four different PLMs improving EditAS by an average accuracy of 9.02%, highlighting the generalizability of our framework. Overall, our work demonstrates the promising potential of jointly fine-tuning the PLMbased retriever and generator to predict accurate assertions by incorporating external knowledge sources, thereby reducing the manual efforts of unit testing experts in practical scenarios.

24 September 2019 in Computer
Verified
Journal Article
Closed access
XWXingya Wang
Xingya Wang
Weisong SunWeisong Sun
Weisong Sun
LHLinghuan Hu
Linghuan Hu
YZYuan Zhao
Yuan Zhao
WWW. Eric Wong
W. Eric Wong
Zhenyu ChenZhenyu Chen
Zhenyu Chen
While a significant amount of resources can be spent on software testing, the software produced may still suffer from low quality. The authors describe their experience of hosting industrysponsored software-testing contests to help undergraduate and graduate students, as well as practitioners, improve their testing skills.

13 April 2023 in Information and Software Technology
Verified
Journal Article
Closed access
YZYuan Zhao
Yuan Zhao
SLSining Liu
Sining Liu
QZQuanjun Zhang
Quanjun Zhang
XGXiuting Ge
Xiuting Ge
JLJia Liu
Jia Liu