This work presents a quantitative evaluation of the quality of a set of mutation operators for WS-BPEL 2.0 (Web Services Business Process Execution Language), an OASIS standardized language for the composition of Web Services. A series of experiments is conducted, aiming to determine how selective operators are for the qualification of test cases and, more generally, the quality of the operators themselves. This includes a formalization of mutation testing defining the key concepts and measures employed. Three different compositions are considered and special attention is paid to the properties that suitable test-suites should enjoy.
10 May 2014 in Software Testing, Verification and Reliability
Business processes based in WS-BPEL compositions are rapidly becoming commonplace in recent years, so it is important to pay special attention to testing in this context. Mutation testing is a white-box testing technique that has been applied successfully to programs written in several languages. In order to apply it we need a system for mutant generation. This paper presents the first results of applying a new system, GAmera, for automatic mutant generation for WS-BPEL compositions. It is composed by three different elements: an analyzer, a mutant generator and a system that executes and evaluates the mutants. GAmera is based in genetic algorithms and attempts to minimize the number of generated mutants, independently of the number and type of mutation operators, without losing relevant information. It can also detect potentially equivalent mutants allowing to improve the quality of the test suite.
Search-based testing generates test cases by encoding an adequacy criterion as the fitness function that drives a search-based optimization algorithm. Genetic algorithms have been successfully applied in search-based testing: while most of them use adequacy criteria based on the structure of the program, some try to maximize the mutation score of the test suite. This work presents a genetic algorithm for generating a test suite for mutation testing. The algorithm adopts several features from existing bacteriological algorithms, using single test cases as individuals and keeping generated individuals in a memory. The algorithm can optionally use automated seeding when producing the first population, by taking into account interesting constants in the source code. We have implemented this algorithm in a framework and we have applied it to a WS-BPEL composition, measuring to which extent the genetic algorithm improves the initial random test suite. We compare our genetic algorithm, with and without automated seeding, to random testing.