14,201 research outputs found
Quantitative metrics for mutation testing
Program mutation is the process of generating versions of a base program by applying elementary syntactic modifications; this technique has been used in program testing in a variety of applications, most notably to assess the quality of a test data set. A good test set will discover the difference between the original program and mutant except if the mutant is semantically equivalent to the original program, despite being syntactically distinct.
Equivalent mutants are a major nuisance in the practice of mutation testing, because they introduce a significant amount of bias and uncertainty in the analysis of test results; indeed, mutants are useful only to the extent that they define distinct functions from the base program. Yet, despite several decades of research, the identification of equivalent mutants remains a tedious, inefficient, ineffective and error prone process.
The approach that is adopted in this dissertation is to turn away from the goal of identifying individual mutants which are semantically equivalent to the base program, in favor of an approach that merely focuses on estimating their number. To this effect, the following question is considered: what makes a base program P prone to produce equivalent mutants? The position taken in this work is that what makes a program prone to generate equivalent mutants is the same property that makes a program fault tolerant, since fault tolerance is by definition the ability to maintain correct behavior despite the presence and sensitization of faults; whether these faults stem from poor design or from mutation operators does not matter. Hence if we could only quantify the redundancy of a program, we should be able to use the redundancy metrics to estimate the ratio of equivalent mutants (REM for short) of a program.
Using redundancy metrics that were previously defined to reflect the state redundancy of a program, its functional redundancy, its non injectivity and its non-determinacy, this dissertation makes the following contributions: The design and implementation of a Java compiler, using compiler generation technology, to analyze Java code and compute its redundancy metrics. An empirical study on standard mutation testing benchmarks to analyze the statistical relationships between the REM of a program and its redundancy metrics. The derivation of regression models to estimate the REM of a program from its compiler generated redundancy metrics, for a variety of mutation policies. The use of the REM to address a number of mutation related issues, including: estimating the level of redundancy between non-equivalent mutants; redefining the mutation score of a test data set to take into account the possibility that mutants may be semantically equivalent to each other; using the REM to derive a minimal set of mutants without having to analyze all the pairs of mutants for equivalence.
The main conclusions of this work are the following: The REM plays a very important role in the mutation analysis of a program, as it gives many useful insights into the properties of its mutants. All the attributes that can be computed from the REM of a program are very sensitive to the exact value of the REM; Hence the REM must be estimated with great precision.
Consequently, the focus of future research is to revisit the Java compiler and enhance the precision of its estimation of redundancy metrics, and to revisit the regression models accordingly
A Model to Estimate First-Order Mutation Coverage from Higher-Order Mutation Coverage
The test suite is essential for fault detection during software development.
First-order mutation coverage is an accurate metric to quantify the quality of
the test suite. However, it is computationally expensive. Hence, the adoption
of this metric is limited. In this study, we address this issue by proposing a
realistic model able to estimate first-order mutation coverage using only
higher-order mutation coverage. Our study shows how the estimation evolves
along with the order of mutation. We validate the model with an empirical study
based on 17 open-source projects.Comment: 2016 IEEE International Conference on Software Quality, Reliability,
and Security. 9 page
Fluctuation analysis with cell deaths
The classical Luria-Delbr\"uck model for fluctuation analysis is extended to
the case where cells can either divide or die at the end of their generation
time. This leads to a family of probability distributions generalizing the
Luria-Delbr\"uck family, and depending on three parameters: the expected number
of mutations, the relative fitness of normal cells compared to mutants, and the
death probability of mutants. The probabilistic treatment is similar to that of
the classical case; simulation and computing algorithms are provided. The
estimation problem is discussed: if the death probability is known, the two
other parameters can be reliably estimated. If the death probability is
unknown, the model can be identified only for large samples
Statistics for the Luria-Delbr\"uck distribution
The Luria-Delbr\"uck distribution is a classical model of mutations in cell
kinetics. It is obtained as a limit when the probability of mutation tends to
zero and the number of divisions to infinity. It can be interpreted as a
compound Poisson distribution (for the number of mutations) of exponential
mixtures (for the developing time of mutant clones) of geometric distributions
(for the number of cells produced by a mutant clone in a given time). The
probabilistic interpretation, and a rigourous proof of convergence in the
general case, are deduced from classical results on Bellman-Harris branching
processes. The two parameters of the Luria-Delbr\"uck distribution are the
expected number of mutations, which is the parameter of interest, and the
relative fitness of normal cells compared to mutants, which is the heavy tail
exponent. Both can be simultaneously estimated by the maximum likehood method.
However, the computation becomes numerically unstable as soon as the maximal
value of the sample is large, which occurs frequently due to the heavy tail
property. Based on the empirical generating function, robust estimators are
proposed and their asymptotic variance is given. They are comparable in
precision to maximum likelihood estimators, with a much broader range of
calculability, a better numerical stability, and a negligible computing time
Evaluating Random Mutant Selection at Class-Level in Projects with Non-Adequate Test Suites
Mutation testing is a standard technique to evaluate the quality of a test
suite. Due to its computationally intensive nature, many approaches have been
proposed to make this technique feasible in real case scenarios. Among these
approaches, uniform random mutant selection has been demonstrated to be simple
and promising. However, works on this area analyze mutant samples at project
level mainly on projects with adequate test suites. In this paper, we fill this
lack of empirical validation by analyzing random mutant selection at class
level on projects with non-adequate test suites. First, we show that uniform
random mutant selection underachieves the expected results. Then, we propose a
new approach named weighted random mutant selection which generates more
representative mutant samples. Finally, we show that representative mutant
samples are larger for projects with high test adequacy.Comment: EASE 2016, Article 11 , 10 page
Mutation testing from probabilistic finite state machines
Mutation testing traditionally involves mutating a program in order to produce a set of mutants and using these mutants in order to either estimate the effectiveness of a test suite or to drive test generation. Recently, however, this approach has been applied to specifications such as those written as finite state machines. This paper extends mutation testing to finite state machine models in which transitions have associated probabilities. The paper describes several ways of mutating a probabilistic finite state machine (PFSM) and shows how test sequences that distinguish between a PFSM and its mutants can be generated. Testing then involves applying each test sequence multiple times, observing the resultant output sequences and using results from statistical sampling theory in order to compare the observed frequency of each output sequence with that expected
Evolution of Biological Complexity
In order to make a case for or against a trend in the evolution of complexity
in biological evolution, complexity needs to be both rigorously defined and
measurable. A recent information-theoretic (but intuitively evident) definition
identifies genomic complexity with the amount of information a sequence stores
about its environment. We investigate the evolution of genomic complexity in
populations of digital organisms and monitor in detail the evolutionary
transitions that increase complexity. We show that because natural selection
forces genomes to behave as a natural ``Maxwell Demon'', within a fixed
environment genomic complexity is forced to increase.Comment: LaTeX 19 pages, incl. 4 fig
Non-equilibrium theory of the allele frequency spectrum
A forward diffusion equation describing the evolution of the allele frequency
spectrum is presented. The influx of mutations is accounted for by imposing a
suitable boundary condition. For a Wright-Fisher diffusion with or without
selection and varying population size, the boundary condition is , where is the frequency
spectrum of derived alleles at independent loci at time and is
the relative population size at time . When population size and selection
intensity are independent of time, the forward equation is equivalent to the
backwards diffusion usually used to derive the frequency spectrum, but the
forward equation allows computation of the time dependence of the spectrum both
before an equilibrium is attained and when population size and selection
intensity vary with time. From the diffusion equation, we derive a set of
ordinary differential equations for the moments of and express the
expected spectrum of a finite sample in terms of those moments. We illustrate
the use of the forward equation by considering neutral and selected alleles in
a highly simplified model of human history. For example, we show that
approximately 30% of the expected heterozygosity of neutral loci is
attributable to mutations that arose since the onset of population growth in
roughly the last years.Comment: 24 pages, 7 figures, updated to accomodate referees' suggestions, to
appear in Theoretical Population Biolog
Is the Stack Distance Between Test Case and Method Correlated With Test Effectiveness?
Mutation testing is a means to assess the effectiveness of a test suite and
its outcome is considered more meaningful than code coverage metrics. However,
despite several optimizations, mutation testing requires a significant
computational effort and has not been widely adopted in industry. Therefore, we
study in this paper whether test effectiveness can be approximated using a more
light-weight approach. We hypothesize that a test case is more likely to detect
faults in methods that are close to the test case on the call stack than in
methods that the test case accesses indirectly through many other methods.
Based on this hypothesis, we propose the minimal stack distance between test
case and method as a new test measure, which expresses how close any test case
comes to a given method, and study its correlation with test effectiveness. We
conducted an empirical study with 21 open-source projects, which comprise in
total 1.8 million LOC, and show that a correlation exists between stack
distance and test effectiveness. The correlation reaches a strength up to 0.58.
We further show that a classifier using the minimal stack distance along with
additional easily computable measures can predict the mutation testing result
of a method with 92.9% precision and 93.4% recall. Hence, such a classifier can
be taken into consideration as a light-weight alternative to mutation testing
or as a preceding, less costly step to that.Comment: EASE 201
Folding of Cu, Zn superoxide dismutase and Familial Amyotrophic Lateral Sclerosis
Cu,Zn superoxide dismutase (SOD1) has been implicated in the familial form of
the neurodegenerative disease Amyotrophic Lateral Sclerosis (ALS). It has been
suggested that mutant mediated SOD1 misfolding/aggregation is an integral part
of the pathology of ALS. We study the folding thermodynamics and kinetics of
SOD1 using a hybrid molecular dynamics approach. We reproduce the
experimentally observed SOD1 folding thermodynamics and find that the residues
which contribute the most to SOD1 thermal stability are also crucial for
apparent two-state folding kinetics. Surprisingly, we find that these residues
are located on the surface of the protein and not in the hydrophobic core.
Mutations in some of the identified residues are found in patients with the
disease. We argue that the identified residues may play an important role in
aggregation. To further characterize the folding of SOD1, we study the role of
cysteine residues in folding and find that non-native disulfide bond formation
may significantly alter SOD1 folding dynamics and aggregation propensity.Comment: 16 pages, 5 figure
- …