Higher Order Mutation Testing

Mutation testing is a fault-based software testing technique that has been studied widely for over three decades. To date, work in this field has focused largely on first order mutants because it is believed that higher order mutation testing is too computationally expensive to be practical. This thesis argues that some higher order mutants are potentially better able to simulate real world faults and to reveal insights into programming bugs than the restricted class of first order mutants. This thesis proposes a higher order mutation testing paradigm which combines valuable higher order mutants and non-trivial first order mutants together for mutation testing. To overcome the exponential increase in the number of higher order mutants a search process that seeks fit mutants (both first and higher order) from the space of all possible mutants is proposed. A fault-based higher order mutant classification scheme is introduced. Based on different types of fault interactions, this approach classifies higher order mutants into four categories: expected, worsening, fault masking and fault shifting. A search-based approach is then proposed for locating subsuming and strongly subsuming higher order mutants. These mutants are a subset of fault mask and fault shift classes of higher order mutants that are more difficult to kill than their constituent first order mutants. Finally, a hybrid test data generation approach is introduced, which combines the dynamic symbolic execution and search based software testing approaches to generate strongly adequate test data to kill first and higher order mutants

Impact of Hardware Emulation on the verification quality improvement

International audienceSoftware simulation remains the most used method for VHDL RTL functional verification. The functional verification process essentially consists of two parts. The first one is the functional qualification; the second one is the qualification- driven stimuli generation. Currently, the qualification and the generation tasks are iterative processes based on VHDL simulation which is dramatically time consuming. The simulation time increases with the circuits' size and the required level of quality. In our previous works, we have proposed some approaches based on the mutation testing technique to evaluate and to improve functional validation quality. Now, to reduce this simulation time, we propose in this paper a new approach based on FPGA emulation. So, an hardware-software platform called “Meta-Mutant Testbench” is used to emulate mutants. Experimental results for some ITC'99 benchmark circuits show that our mutation emulator is about 20 times faster than classical software simulators; this speedup increases with the circuits' size