22 June 2025 in Proceedings of the ACM on Software Engineering
Unit testing plays a crucial role in bug detection and ensuring software correctness. It helps developers identify errors early in development, thereby reducing software defects. In recent years, large language models (LLMs) have demonstrated significant potential in automating unit test generation. However, using LLMs to generate unit tests faces many challenges. 1) The execution pass rate of the test cases generated by LLMs is low. 2) The test case coverage is inadequate, making it challenging to detect potential risks in the code. 3) Current research methods primarily focus on languages such as Java and Python, while studies on C programming are scarce, despite its importance in the real world. To address these challenges, we propose STRUT, a novel unit test generation method. STRUT utilizes structured test cases as a bridge between complex programming languages and LLMs. Instead of directly generating test code, STRUT guides LLMs to produce structured test cases, thereby alleviating the limitations of LLMs when generating code for programming languages with complex features. First, STRUT analyzes the context of focal methods and constructs structured seed test cases for them. These seed test cases then guide LLMs to generate a set of structured test cases. Subsequently, a rule-based approach is employed to convert the structured set of test cases into executable test code. We conducted a comprehensive evaluation of STRUT, which achieved an impressive execution pass rate of 96.01%, along with 77.67% line coverage and 63.60% branch coverage. This performance significantly surpasses that of the LLMs-based baseline methods and the symbolic execution tool SunwiseAUnit. These results highlight STRUT's superior capability in generating high-quality unit test cases by leveraging the strengths of LLMs while addressing their inherent limitations.
28 February 2025 in ACM Transactions on Software Engineering and Methodology
Detecting atomicity violations effectively in interrupt-driven programs is difficult due to the asymmetric concurrency interleaving of interrupts. Current approaches face two main challenges: (1) A large number of false positives are generated by efficient static analysis techniques. (2) Loops with large or unknown bounds in these programs limit the scalability of the bounded verification techniques. To address these challenges, we present NIChecker, a new bounded verification tool designed to detect atomicity violations in interrupt-driven programs. The key ideas are: (1) Transforming an interrupt-driven program into a bounded sequential C program through lazy sequentialization technique. This sequential program accurately models interrupt masking and nested interrupt execution. (2) Combining a refined loop abstraction technique with our sequentialization to enhance the efficiency of detecting programs with intractable loops. (3) Integrating slicing and an interleaving path reduction technique known as preemption point reduction in NIChecker to shrink the explored state space. We prove the bounded correctness of our translation and discuss the impact of our optimizations. We evaluate NIChecker on 31 academic benchmark programs and 18 real-world interrupt-driven programs. Our results show that NIChecker achieves better precision, a lower false positive rate, and a significant verification speed-up than related state-of-the-art tools.