19 January 2021 in IEEE Transactions on Software Engineering
Mutation testing, a fundamental software testing technique, which is a typical way to evaluate the adequacy of a test suite. In mutation testing, a set of mutants are generated by seeding the different classes of faults into a program under test. Test data shall be generated in the way that as many mutants can be killed as possible. Thanks to numerous tools to implement mutation testing for different languages, a huge amount of mutants are normally generated even for small-sized programs. However, a large number of mutants not only leads to a high cost of mutation testing, but also make the corresponding test data generation a non-trivial task. In this paper, we make use of intelligent technologies to improve the effectiveness and efficiency of mutation testing from two perspectives. A machine learning technique, namely fuzzy clustering, is applied to categorize mutants into different clusters. Then, a multi-population genetic algorithm via individual sharing is employed to generate test data for killing the mutants in different clusters in parallel when the problem of test data generation as an optimization one. A comprehensive framework, termed as $\mathbf {FUZGENMUT}$ , is thus developed to implement the proposed techniques. The experiments based on nine programs of various sizes show that fuzzy clustering can help to reduce the cost of mutation testing effectively, and that the multi-population genetic algorithm improves the efficiency of test data generation while delivering the high mutant-killing capability. The results clearly indicate that the huge potential of using intelligent technologies to enhance the efficacy and thus the practicality of mutation testing.
25 August 2020 in IEEE Transactions on Software Engineering
For the path coverage testing of a Message-Passing Interface (MPI) program, test data generation based on an evolutionary optimization algorithm (EOA) has been widely known. However, during the use of the above technique, it is necessary to evaluate the fitness of each evolutionary individual by executing the program, which is generally computationally expensive. In order to reduce the computational cost, this paper proposes a method of integrating an ensemble surrogate model's estimation into the process of generating test data. The proposed method first produces a number of test inputs using an EOA, and forms a training set together with their real fitness. Then, this paper trains an ensemble surrogate model (ESM) based on the training set, which is employed to estimate the fitness of each individual. Finally, a small number of individuals with good estimations are selected to further execute the program, so as to have their real fitness for the subsequent evolution. This paper applies the proposed method to seven benchmark MPI programs, which is compared with several state-of-the-art approaches. The experimental results show that the proposed method can generate test data with significantly low computational cost.
1 September 2020 in Parallel Computing
31 March 2023 in IEEE Transactions on Software Engineering
When a large number of target paths in a sequential program need to be covered, we can divide similar target paths into the same group, and generate a test suite covering the same group of target paths at the same time, so as to reduce the testing cost. However, different communication edges may be run under a same test input when executing a Message-Passing I nterface (MPI) program with non-determinism, which cause different code fragments may be traversed, indicating the difficulty of generating a test suite to cover each group of target paths. This paper proposes an approach to evolutionary generation of test suites for multi-path coverage of MPI programs with non-determinism, which can significantly reduce the testing cost and difficulty. We first design an indicator for evaluating each traversal set of communication edges, which is used to form a relation matrix between each target path and each traversal set of communication edges, so as to divide all the target paths into a certain amount of groups. Then, we construct an optimization model for test suite generation associated with each group. Finally, an evolutionary optimization algorithm is extended to solve each model, and used to generate a test suite covering each group of target paths. The proposed approach is utilized and compared with several state-of-the-art approaches to seven benchmark MPI programs, as well as the experimental results illustrate that the proposed approach can efficiently generate a test suite, thus supporting the superiority of the proposed approach.
20 November 2013 in Chinese Journal of Computers
1 April 2014 in Chinese Journal of Electronics
Test data generation, the premise of software testing, has attracted scholars in the software engineering community in recent years. Influenced by task partitioning, process scheduling, and network delays, parallel programs are executed in a non-deterministic way, which makes test data generation of parallel programs different from that of serial programs in essence. This paper investigated the problem of generating test data for multiple paths coverage of message-passing parallel programs. A mathematical model of the above problem was built based on each given path and its equivalent ones. It was solved by using a genetic algorithm to generate all desired data in one run. The proposed method was applied to five benchmark programs, and compared with the existing methods. The experimental results show that the proposed method greatly shortens the number of iterations and time consumption without reducing the coverage rate.