Chaoxuan Shi

Chaoxuan Shi

Auto-generated profile, based on publicly available information

Chaoxuan Shi

State Key Lab for Novel Software Technology and the Department of Computer Science and Technology, Nanjing University, Nanjing, China

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
15 March 2024 in ACM Transactions on Software Engineering and Methodology
Verified
Journal Article
Open access
TZTingwei Zhu
Tingwei Zhu
ZLZhong Li
Zhong Li
MPMinxue Pan
Minxue Pan
CSChaoxuan Shi
Chaoxuan Shi
TZTian Zhang
Tian Zhang
XLXuandong Li
Xuandong Li
Code summarization aims to generate short functional descriptions for source code to facilitate code comprehension. While Information Retrieval (IR) approaches that leverage similar code snippets and corresponding summaries have led the early research, Deep Learning (DL) approaches that use neural models to capture statistical properties between code and summaries are now mainstream. Although some preliminary studies suggest that IR approaches are more effective in some cases, it is currently unclear how effective the existing approaches can be in general, where and why IR/DL approaches perform better, and whether the integration of IR and DL can achieve better performance. Consequently, there is an urgent need for a comprehensive study of the IR and DL code summarization approaches to provide guidance for future development in this area. This article presents the first large-scale empirical study of 18 IR, DL, and hybrid code summarization approaches on five benchmark datasets. We extensively compare different types of approaches using automatic metrics, we conduct quantitative and qualitative analyses of where and why IR and DL approaches perform better, respectively, and we also study hybrid approaches for assessing the effectiveness of integrating IR and DL. The study shows that the performance of IR approaches should not be underestimated, that while DL models perform better in predicting tokens from method signatures and capturing structural similarities in code, simple IR approaches tend to perform better in the presence of code with high similarity or long reference summaries, and that existing hybrid approaches do not perform as well as individual approaches in their respective areas of strength. Based on our findings, we discuss future research directions for better code summarization.

4 August 2023
Verified
Conference Paper
Open access
CSChaoxuan Shi
Chaoxuan Shi
TZTingwei Zhu
Tingwei Zhu
TZTian Zhang
Tian Zhang
Jun PangJun Pang
Jun Pang
MPMinxue Pan
Minxue Pan
Neural code intelligence models are cutting-edge automated code understanding technologies that have achieved remarkable performance in various software engineering tasks. However, the lack of deep learning models’ interpretability hinders the application of deep learning based code intelligence models in real-world scenarios, particularly in security-critical domains. Previous studies use program simplification to understand neural code intelligence models, but they have overlooked the fact that the most significant difference between source code and natural language is the code’s structural semantics. In this paper, we first conduct an empirical study to identify the critical code structural semantic features valued by neural code intelligence models, and then we propose a novel program simplification method called SSGPS (Structural-Semantics Guided Program Simplification). Results on three code summarization models show that SSGPS can reduce training and testing time by 20-40% while controlling the decrease in model performance by less than 4%, demonstrating that our method can retain the critical code structural semantics for understanding neural code intelligence models.