8,773 research outputs found
Faster Mutation Analysis via Equivalence Modulo States
Mutation analysis has many applications, such as asserting the quality of
test suites and localizing faults. One important bottleneck of mutation
analysis is scalability. The latest work explores the possibility of reducing
the redundant execution via split-stream execution. However, split-stream
execution is only able to remove redundant execution before the first mutated
statement.
In this paper we try to also reduce some of the redundant execution after the
execution of the first mutated statement. We observe that, although many
mutated statements are not equivalent, the execution result of those mutated
statements may still be equivalent to the result of the original statement. In
other words, the statements are equivalent modulo the current state.
In this paper we propose a fast mutation analysis approach, AccMut. AccMut
automatically detects the equivalence modulo states among a statement and its
mutations, then groups the statements into equivalence classes modulo states,
and uses only one process to represent each class. In this way, we can
significantly reduce the number of split processes. Our experiments show that
our approach can further accelerate mutation analysis on top of split-stream
execution with a speedup of 2.56x on average.Comment: Submitted to conferenc
Automatic Repair of Real Bugs: An Experience Report on the Defects4J Dataset
Defects4J is a large, peer-reviewed, structured dataset of real-world Java
bugs. Each bug in Defects4J is provided with a test suite and at least one
failing test case that triggers the bug. In this paper, we report on an
experiment to explore the effectiveness of automatic repair on Defects4J. The
result of our experiment shows that 47 bugs of the Defects4J dataset can be
automatically repaired by state-of- the-art repair. This sets a baseline for
future research on automatic repair for Java. We have manually analyzed 84
different patches to assess their real correctness. In total, 9 real Java bugs
can be correctly fixed with test-suite based repair. This analysis shows that
test-suite based repair suffers from under-specified bugs, for which trivial
and incorrect patches still pass the test suite. With respect to practical
applicability, it takes in average 14.8 minutes to find a patch. The experiment
was done on a scientific grid, totaling 17.6 days of computation time. All
their systems and experimental results are publicly available on Github in
order to facilitate future research on automatic repair
Dynamic Mutant Subsumption Analysis using LittleDarwin
Many academic studies in the field of software testing rely on mutation
testing to use as their comparison criteria. However, recent studies have shown
that redundant mutants have a significant effect on the accuracy of their
results. One solution to this problem is to use mutant subsumption to detect
redundant mutants. Therefore, in order to facilitate research in this field, a
mutation testing tool that is capable of detecting redundant mutants is needed.
In this paper, we describe how we improved our tool, LittleDarwin, to fulfill
this requirement
Is the Stack Distance Between Test Case and Method Correlated With Test Effectiveness?
Mutation testing is a means to assess the effectiveness of a test suite and
its outcome is considered more meaningful than code coverage metrics. However,
despite several optimizations, mutation testing requires a significant
computational effort and has not been widely adopted in industry. Therefore, we
study in this paper whether test effectiveness can be approximated using a more
light-weight approach. We hypothesize that a test case is more likely to detect
faults in methods that are close to the test case on the call stack than in
methods that the test case accesses indirectly through many other methods.
Based on this hypothesis, we propose the minimal stack distance between test
case and method as a new test measure, which expresses how close any test case
comes to a given method, and study its correlation with test effectiveness. We
conducted an empirical study with 21 open-source projects, which comprise in
total 1.8 million LOC, and show that a correlation exists between stack
distance and test effectiveness. The correlation reaches a strength up to 0.58.
We further show that a classifier using the minimal stack distance along with
additional easily computable measures can predict the mutation testing result
of a method with 92.9% precision and 93.4% recall. Hence, such a classifier can
be taken into consideration as a light-weight alternative to mutation testing
or as a preceding, less costly step to that.Comment: EASE 201
Model-Based Security Testing
Security testing aims at validating software system requirements related to
security properties like confidentiality, integrity, authentication,
authorization, availability, and non-repudiation. Although security testing
techniques are available for many years, there has been little approaches that
allow for specification of test cases at a higher level of abstraction, for
enabling guidance on test identification and specification as well as for
automated test generation.
Model-based security testing (MBST) is a relatively new field and especially
dedicated to the systematic and efficient specification and documentation of
security test objectives, security test cases and test suites, as well as to
their automated or semi-automated generation. In particular, the combination of
security modelling and test generation approaches is still a challenge in
research and of high interest for industrial applications. MBST includes e.g.
security functional testing, model-based fuzzing, risk- and threat-oriented
testing, and the usage of security test patterns. This paper provides a survey
on MBST techniques and the related models as well as samples of new methods and
tools that are under development in the European ITEA2-project DIAMONDS.Comment: In Proceedings MBT 2012, arXiv:1202.582
- …