Exploring design decisions for mutation testing

Software testing is by far the most popular technique used in industry for quality assurance. One key challenge of software testing is how to evaluate the quality of test suites in terms of their bug-finding capability. A test suite with a large number of tests, or that achieves a high statement or branch coverage, does not necessarily have a high bug-finding capability. Mutation testing is widely used in research to evaluate the quality of test suites, and it is often considered the most powerful approach for this purpose. Mutation testing proceeds in two steps. The first step is mutant generation. A mutant is a modified version of the original program obtained by applying a mutation operator. A mutation operator is a program transformation that introduces a small syntactic change to the original program. The second step of mutation testing is to run the test suite and determine which mutants are killed, i.e., which mutants lead to tests having a different output when run on them compared against running on the original program. Mutation testing produces a measure of quality of the test suite called mutation score. The mutation score of a given test suite is the percentage of mutants killed by that test suite out of the total number of generated mutants. In this dissertation, we explore three design decisions related to mutation testing and provide recommendations to researchers in those regards. First, we look into mutation operators. To provide insights about how to improve the test suites, mutation testing requires both high quality and diverse mutation operators that lead to different program behaviors. We propose the use of approximate transformations as mutation operators. Approximate transformations were introduced in the emerging area of approximate computing for changing program semantics to trade the accuracy of results for improved energy efficiency or performance. We compared three approximate transformations with a set of conventional mutation operators from the literature, on nine open-source Java subjects. The results showed that approximate transformations change program behavior differently from conventional mutation operators. Our analysis uncovered code patterns in which approximate mutants survived (i.e., were not killed) and showed the practical value of approximate transformations both for understanding code amenable to approximations and for discovering bad tests. We submitted 11 pull requests to fix bad tests. Seven have already been integrated by the developers. Second, we explore the effect of compiler optimizations on mutation testing. Multiple mutation testing tools were developed that perform mutation at different levels. More recently mutation testing has been performed at the level of compiler intermediate representation (IR), e.g., for the LLVM IR and Java bytecode/IR. Compiler optimizations are automatic program transformations applied at the IR level with the goal of improving a measure of program performance, while preserving program semantics. Applying mutations at the IR level means that mutation testing becomes more susceptible to the effects of compiler optimizations. We investigate a new perspective on mutation testing: evaluating how standard compiler optimizations affect the cost and results of mutation testing performed at the IR level. Our study targets LLVM, a popular compiler infrastructure that supports multiple source and target languages. Our evaluation on 16 Coreutils programs discovers several interesting relations between the numbers of mutants (including the numbers on equivalent and duplicated mutants) and mutation scores on unoptimized and optimized programs. Third, we perform an empirical study to compare mutation testing at the source (SRC) and IR levels. Applying mutation at different levels offers different advantages and disadvantages, and the relation between mutants at the different levels is not clear. In our study, we compare mutation testing at the SRC and IR levels, specifically in the C programming language and the LLVM compiler IR. To make the comparison fair, we develop two mutation tools that implement conceptually the same operators at both levels. We also employ automated techniques to account for equivalent and duplicated mutants, and to determine hard-tokill mutants. We carry out our study on 16 programs from the Coreutils library, using a total of 948 tests. Our results show interesting characteristics that can help researchers better understand the relationship between mutation testing at both levels. Overall, we find mutation testing to be better at the SRC level than at the IR level: the SRC level produces much fewer (non-equivalent) mutants and is thus less expensive, but the SRC level still generates a similar number of hard-to-kill mutants

Measuring Coverage of Prolog Programs Using Mutation Testing

Testing is an important aspect in professional software development, both to avoid and identify bugs as well as to increase maintainability. However, increasing the number of tests beyond a reasonable amount hinders development progress. To decide on the completeness of a test suite, many approaches to assert test coverage have been suggested. Yet, frameworks for logic programs remain scarce. In this paper, we introduce a framework for Prolog programs measuring test coverage using mutations. We elaborate the main ideas of mutation testing and transfer them to logic programs. To do so, we discuss the usefulness of different mutations in the context of Prolog and empirically evaluate them in a new mutation testing framework on different examples.Comment: 16 pages, Accepted for presentation in WFLP 201

Squeezed Neutrino Oscillations in Quantum Field Theory

By resorting to recent results on fermion mixing which show that the Fock space of definite flavor states is unitarily inequivalent to the Fock space of definite mass states, we discuss the phenomenological implications on the neutrino oscillation formula. For finite momentum the oscillation amplitude is depressed, or "squeezed", by a momentum dependent factor. In the relativistic limit the conventional oscillation formula is recovered.Comment: 12 pages, LaTex, 1 figure ( on request ), in press on Phys. Lett. B. (minor changes: reformatted

Transmutations and spectral parameter power series in eigenvalue problems

We give an overview of recent developments in Sturm-Liouville theory concerning operators of transmutation (transformation) and spectral parameter power series (SPPS). The possibility to write down the dispersion (characteristic) equations corresponding to a variety of spectral problems related to Sturm-Liouville equations in an analytic form is an attractive feature of the SPPS method. It is based on a computation of certain systems of recursive integrals. Considered as families of functions these systems are complete in the $L_{2}$-space and result to be the images of the nonnegative integer powers of the independent variable under the action of a corresponding transmutation operator. This recently revealed property of the Delsarte transmutations opens the way to apply the transmutation operator even when its integral kernel is unknown and gives the possibility to obtain further interesting properties concerning the Darboux transformed Schr\"{o}dinger operators. We introduce the systems of recursive integrals and the SPPS approach, explain some of its applications to spectral problems with numerical illustrations, give the definition and basic properties of transmutation operators, introduce a parametrized family of transmutation operators, study their mapping properties and construct the transmutation operators for Darboux transformed Schr\"{o}dinger operators.Comment: 30 pages, 4 figures. arXiv admin note: text overlap with arXiv:1111.444

Properties of equations of the continuous Toda type

We study a modified version of an equation of the continuous Toda type in 1+1 dimensions. This equation contains a friction-like term which can be switched off by annihilating a free parameter \ep. We apply the prolongation method, the symmetry and the approximate symmetry approach. This strategy allows us to get insight into both the equations for \ep =0 and \ep \ne 0, whose properties arising in the above frameworks are mutually compared. For \ep =0, the related prolongation equations are solved by means of certain series expansions which lead to an infinite- dimensional Lie algebra. Furthermore, using a realization of the Lie algebra of the Euclidean group $E_{2}$, a connection is shown between the continuous Toda equation and a linear wave equation which resembles a special case of a three-dimensional wave equation that occurs in a generalized Gibbons-Hawking ansatz \cite{lebrun}. Nontrivial solutions to the wave equation expressed in terms of Bessel functions are determined. For \ep\,\ne\,0, we obtain a finite-dimensional Lie algebra with four elements. A matrix representation of this algebra yields solutions of the modified continuous Toda equation associated with a reduced form of a perturbative Liouville equation. This result coincides with that achieved in the context of the approximate symmetry approach. Example of exact solutions are also provided. In particular, the inverse of the exponential-integral function turns out to be defined by the reduced differential equation coming from a linear combination of the time and space translations. Finally, a Lie algebra characterizing the approximate symmetries is discussed.Comment: LaTex file, 27 page