14 April 2019 in IEEE Transactions on Software Engineering
In large-scale software systems, error-prone or change-prone files rarely stand alone. They are typically architecturally connected and their connections usually exhibit architecture problems causing the propagation of error-proneness or change-proneness. In this paper, we propose and empirically validate a suite of architecture anti-patterns that occur in all large-scale software systems and are involved in high maintenance costs. We define these architecture anti-patterns based on fundamental design principles and Baldwin and Clark's design rule theory. We can automatically detect these anti-patterns by analyzing a project's structural relationships and revision history. Through our analyses of 19 large-scale software projects, we demonstrate that these architecture anti-patterns have significant impact on files' bug-proneness and change-proneness. In particular, we show that 1) files involved in these architecture anti-patterns are more error-prone and change-prone; 2) the more anti-patterns a file is involved in, the more error-prone and change-prone it is; and 3) while all of our defined architecture anti-patterns contribute to file's error-proneness and change-proneness, Unstable Interface and Crossing contribute the most by far.
1 April 2022 in Information and Software Technology
5 August 2021 in IEEE Transactions on Software Engineering
Architectural Technical Debt ( ATD ) refers to sub-optimal architectural design in a software system that incurs high maintenance “interest” over time. Previous research revealed that ATD has significant negative impact on daily development. This paper contributes an approach to enable an architect to precisely locate ATDs , as well as capture the trajectory of maintenance cost on each debt, based on which, predict the cost of the debt in a future release. The ATDs are expressed in four typical patterns, which entail the core of each debt. Furthermore, we aggregate compound ATDs to capture the complicated relationship among multiple ATD instances, which should be examined together for effective refactoring solutions. We evaluate our approach on 18 real-world projects. We identified ATDs that persistently incur significant (up to 95 percent of) maintenance costs in most projects. The maintenance costs on the majority of debts fit into a linear regression model—indicating stable “interest” rate. In five projects, 12.1 to 27.6 percent of debts fit into an exponential model, indicating increasing “interest” rate, which deserve higher priority from architects. The regression models can accurately predict the costs of the majority of (82 to 100 percent) debts in the next release of a system. By aggregating related ATDs , architects can focus on a small number of cost-effective compound debts, which contain a relatively small number of source files, but account for a large portion of maintenance costs in their projects. With these capabilities, our approach can help architects make informed decisions regarding whether, where, and how to refactor for eliminating ATDs in their systems.
26 June 2022 in Information and Software Technology
27 January 2025 in ACM Transactions on Software Engineering and Methodology
AI programming has become a popular topic in recent years. Code suggestion, with code suggestion being a key capability of AI programming. Copilot, an “AI programmer” that provides code suggestions from natural language descriptions, has been launched by GitHub and OpenAI. By far, Copilot has been widely used by millions of developers. However, little work has systematically evaluated the correctness of Copilot's suggestions. We conducted an empirical study on all 2,033 LeetCode problems to assess Copilot's code generation across four mainstream languages: C, Java, JavaScript, and Python. We have found that: 1) 70.0% of problems received at least one correct suggestion, with language-specific rates of 29.7% (C), 57.7% (Java), 54.1% (JavaScript), and 41.0% (Python); 2) Correctness decreases as problem difficulty increases, with acceptance rates of 89.3% (Easy), 72.1% (Medium), and 43.4% (Hard); 3) Acceptance rates vary across problem domains from 49.5% to 90.1%, while Graph problems challenge C and Python most, and Prefix Sum and Heap challenge Java and JavaScript most; 4) For the incorrect suggestions, we further summarize 17 types of error reasons accounting for their incorrectness and analyzed possible causes for why these errors occur. We believe our study can provide valuable insights into Copilot's capabilities and limitations.
28 September 2023 in ACM Transactions on Software Engineering and Methodology
Deep learning (DL) is a really active topic in recent years. Code cloning is a common code implementation that could negatively impact software maintenance. For DL software, developers rely heavily on frameworks to implement DL features. Meanwhile, to guarantee efficiency, developers often reuse the steps and configuration settings for building DL models. These may bring code copy-pastes or reuses inducing code clones. However, there is little work exploring code clones’ impact on DL software. In this article, we conduct an empirical study and show that: (1) code clones are prevalent in DL projects, about 16.3% of code fragments encounter clones, which is almost twice larger than the traditional projects; (2) 75.6% of DL projects contain co-changed clones, meaning changes are propagated among cloned fragments, which can bring maintenance difficulties; (3) Percentage of the clones and Number of clone lines are associated with the emergence of co-changes; (4) the prevalence of Code clones varies in DL projects with different frameworks, but the difference is not significant; (5) Type 1 co-changed clones often spread over different folders, but Types 2 and 3 co-changed clones mainly occur within the same files or folders; (6) 57.1% of all co-changed clones are involved in bugs.