4 research outputs found
Faster Mutation Analysis via Equivalence Modulo States
Mutation analysis has many applications, such as asserting the quality of
test suites and localizing faults. One important bottleneck of mutation
analysis is scalability. The latest work explores the possibility of reducing
the redundant execution via split-stream execution. However, split-stream
execution is only able to remove redundant execution before the first mutated
statement.
In this paper we try to also reduce some of the redundant execution after the
execution of the first mutated statement. We observe that, although many
mutated statements are not equivalent, the execution result of those mutated
statements may still be equivalent to the result of the original statement. In
other words, the statements are equivalent modulo the current state.
In this paper we propose a fast mutation analysis approach, AccMut. AccMut
automatically detects the equivalence modulo states among a statement and its
mutations, then groups the statements into equivalence classes modulo states,
and uses only one process to represent each class. In this way, we can
significantly reduce the number of split processes. Our experiments show that
our approach can further accelerate mutation analysis on top of split-stream
execution with a speedup of 2.56x on average.Comment: Submitted to conferenc
Directed Test Program Generation for JIT Compiler Bug Localization
Bug localization techniques for Just-in-Time (JIT) compilers are based on
analyzing the execution behaviors of the target JIT compiler on a set of test
programs generated for this purpose; characteristics of these test inputs can
significantly impact the accuracy of bug localization. However, current
approaches for automatic test program generation do not work well for bug
localization in JIT compilers. This paper proposes a novel technique for
automatic test program generation for JIT compiler bug localization that is
based on two key insights: (1) the generated test programs should contain both
passing inputs (which do not trigger the bug) and failing inputs (which trigger
the bug); and (2) the passing inputs should be as similar as possible to the
initial seed input, while the failing programs should be as different as
possible from it. We use a structural analysis of the seed program to determine
which parts of the code should be mutated for each of the passing and failing
cases. Experiments using a prototype implementation indicate that test inputs
generated using our approach result in significantly improved bug localization
results than existing approaches
Threats to the validity of mutation-based test assessment
Much research on software testing and test techniques relies on experimental studies based on mutation testing. In this paper we reveal that such studies are vulnerable to a potential threat to validity, leading to possible Type I errors; incorrectly rejecting the Null Hypothesis. Our findings indicate that Type I errors occur, for arbitrary experiments that fail to take countermeasures, approximately 62% of the time. Clearly, a Type I error would potentially compromise any scientific conclusion. We show that the problem derives from such studies’ combined use of both subsuming and subsumed mutants. We collected articles published in the last two years at three leading software engineering conferences. Of those that use mutation-based test assessment, we found that 68% are vulnerable to this threat to validity