Junjie Chen

Junjie Chen

Auto-generated profile, based on publicly available information

Junjie Chen

Tianjin University, Tianjin, China

Research Focus & Metrics

Academic Career Length

9Years (2018-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
6 February 2020 in ACM Computing Surveys
Verified
Journal Article
Open access
JCJunjie Chen
Junjie Chen
JPJibesh Patra
Jibesh Patra
MPMichael Pradel
Michael Pradel
YXYingfei Xiong
Yingfei Xiong
HZHongyu Zhang
Hongyu Zhang
LZLu Zhang
Lu Zhang
Virtually any software running on a computer has been processed by a compiler or a compiler-like tool. Because compilers are such a crucial piece of infrastructure for building software, their correctness is of paramount importance. To validate and increase the correctness of compilers, significant research efforts have been devoted to testing compilers. This survey article provides a comprehensive summary of the current state-of-the-art of research on compiler testing. The survey covers different aspects of the compiler testing problem, including how to construct test programs, what test oracles to use for determining whether a compiler behaves correctly, how to execute compiler tests efficiently, and how to help compiler developers take action on bugs discovered by compiler testing. Moreover, we survey work that empirically studies the strengths and weaknesses of current compiler testing research and practice. Based on the discussion of existing work, we outline several open challenges that remain to be addressed in future work.

18 August 2021
Verified
Conference Paper
Open access
QSQingchao Shen
Qingchao Shen
HMHaoyang Ma
Haoyang Ma
JCJunjie Chen
Junjie Chen
YTYongqiang Tian
Yongqiang Tian
SCShing-Chi Cheung
Shing-Chi Cheung
Xiang ChenXiang Chen
Xiang Chen
There are increasing uses of deep learning (DL) compilers to generate optimized code, boosting the runtime performance of DL models on specific hardware. Like their traditional counterparts, DL compilers can generate incorrect code, resulting in unexpected model behaviors that may cause catastrophic consequences in mission-critical systems. On the other hand, the DL models processed by DL compilers differ fundamentally from imperative programs in that the program logic in DL models is implicit. As such, various characteristics of the bugs arising from traditional compilers need to be revisited in the context of DL compilers. In this paper, we present the first systematic study of DL compiler bugs by analyzing 603 bugs arising in three popular DL compilers (i.e., TVM from Apache, Glow from Facebook, and nGraph from Intel). We analyzed these bugs according to their root causes, symptoms, and the stages where they occur during compilation. We obtain 12 findings, and provide a series of valuable guidelines for future work on DL compiler bug detection and debugging. For example, a large portion (nearly 20%) of DL compiler bugs are related to types, especially tensor types. The analysis of these bugs helps design new mutation operators (e.g., adding type cast for a tensor to promote implicit type conversion in subsequent tensor computations) to facilitate type-related bug detection. Further, we developed TVMfuzz as a proof-of-concept application of our findings to test the TVM DL compiler. It generates new tests based on TVM's original test suite. They expose 8 TVM bugs that are missed by the original test suite. The result demonstrates the usefulness of our findings.

11 December 2025 in ACM Transactions on Software Engineering and Methodology
Verified
Journal Article
Open access
DHDong Huang
Dong Huang
JZJie M. Zhang
Jie M. Zhang
QBQingwen Bu
Qingwen Bu
XXXiaofei Xie
Xiaofei Xie
JCJunjie Chen
Junjie Chen
HCHeming Cui
Heming Cui
As the adoption of LLMs becomes more widespread in software coding ecosystems, a pressing issue has emerged: does the generated code contain social bias and unfairness, such as those related to age, gender, and race? This issue concerns the integrity, fairness, and ethical foundation of software applications that depend on the code generated by these models but are underexplored in the literature. This article presents a novel bias testing framework that is specifically designed for code generation tasks. Based on this framework, we conduct an extensive empirical study on the biases in code generated by five widely studied LLMs (i.e., PALM-2-CodeChat-bison, Claude-instant-1, GPT-3.5-turbo, GPT-4-turbo, and GPT-4). Our findings reveal that biases are prevalent. For example, 13.47% to 49.10% of the codes generated by these LLMs have biased behaviors towards gender. Moreover, we study five bias mitigation prompt strategies that are commonly used in current code generation scenarios, i.e., zero-shot, one-shot, few-shot, and two Chain-of-Thought (CoT) prompts, with and without provided feedback-driven refinement. Our evaluation results illustrate that using direct prompt engineering strategies has limited effectiveness in mitigating bias, but our test execution feedback can help to reduce the ratio of code biases to a large extent (e.g., from 59.88% to 4.79% for GPT-4).

29 September 2023 in ACM Transactions on Software Engineering and Methodology
Verified
Journal Article
Open access
JCJunjie Chen
Junjie Chen
YLYihua Liang
Yihua Liang
QSQingchao Shen
Qingchao Shen
JJJiajun Jiang
Jiajun Jiang
SLShuochuan Li
Shuochuan Li
DL frameworks are the basis of constructing all DL programs and models, and thus their bugs could lead to the unexpected behaviors of any DL program or model relying on them. Such a wide effect demonstrates the necessity and importance of guaranteeing DL frameworks’ quality. Understanding the characteristics of DL framework bugs is a fundamental step for this quality assurance task, facilitating designing effective bug detection and debugging approaches. Hence, in this work, we conduct the most large-scale study on 1,000 bugs from four popular and diverse DL frameworks (i.e., TensorFlow, PyTorch, MXNet, and DL4J). By analyzing the root causes and symptoms of DL framework bugs associated with five components decomposed from DL frameworks, as well as measuring test coverage achieved by three state-of-the-art testing techniques, we obtain 12 major findings for the comprehensive understanding of DL framework bugs and the current status of existing DL framework testing practice, and then provide a series of actionable guidelines for better DL framework bug detection and debugging. Finally, based on the guidelines, we design and implement a prototype DL-framework testing tool, called TenFuzz , which is evaluated to be effective and finds three unknown bugs on the latest TensorFlow framework in a preliminary study, indicating the significance of our guidelines.

17 October 2019 in IEEE Transactions on Software Engineering
Verified
Journal Article
Closed access
MWMing Wen
Ming Wen
JCJunjie Chen
Junjie Chen
YTYongqiang Tian
Yongqiang Tian
RWRongxin Wu
Rongxin Wu
DHDan Hao
Dan Hao
SCShing-Chi Cheung
Shing-Chi Cheung

27 October 2024
Verified
Conference Paper
Open access
LYLin Yang
Lin Yang
CYChen Yang
Chen Yang
SGShutao Gao
Shutao Gao
WWWeijing Wang
Weijing Wang
BWBo Wang
Bo Wang
JCJunjie Chen
Junjie Chen
Unit testing is an essential activity in software development for verifying the correctness of software components. However, manually writing unit tests is challenging and time-consuming. The emergence of Large Language Models (LLMs) offers a new direction for automating unit test generation. Existing research primarily focuses on closed-source LLMs (e.g., ChatGPT and CodeX) with fixed prompting strategies, leaving the capabilities of advanced open-source LLMs with various prompting settings unexplored. Particularly, open-source LLMs offer advantages in data privacy protection and have demonstrated superior performance in some tasks. Moreover, effective prompting is crucial for maximizing LLMs' capabilities. In this paper, we conduct the first empirical study to fill this gap, based on 17 Java projects, five widely-used open-source LLMs with different structures and parameter sizes, and comprehensive evaluation metrics. Our findings highlight the significant influence of various prompt factors, show the performance of open-source LLMs compared to the commercial GPT-4 and the traditional Evosuite, and identify limitations in LLM-based unit test generation. We then derive a series of implications from our study to guide future research and practical use of LLM-based unit test generation.