Higher Order Mutation Testing

Mutation testing is a fault-based software testing technique that has been studied widely for over three decades. To date, work in this field has focused largely on first order mutants because it is believed that higher order mutation testing is too computationally expensive to be practical. This thesis argues that some higher order mutants are potentially better able to simulate real world faults and to reveal insights into programming bugs than the restricted class of first order mutants. This thesis proposes a higher order mutation testing paradigm which combines valuable higher order mutants and non-trivial first order mutants together for mutation testing. To overcome the exponential increase in the number of higher order mutants a search process that seeks fit mutants (both first and higher order) from the space of all possible mutants is proposed. A fault-based higher order mutant classification scheme is introduced. Based on different types of fault interactions, this approach classifies higher order mutants into four categories: expected, worsening, fault masking and fault shifting. A search-based approach is then proposed for locating subsuming and strongly subsuming higher order mutants. These mutants are a subset of fault mask and fault shift classes of higher order mutants that are more difficult to kill than their constituent first order mutants. Finally, a hybrid test data generation approach is introduced, which combines the dynamic symbolic execution and search based software testing approaches to generate strongly adequate test data to kill first and higher order mutants

LittleDarwin: a Feature-Rich and Extensible Mutation Testing Framework for Large and Complex Java Systems

Mutation testing is a well-studied method for increasing the quality of a test suite. We designed LittleDarwin as a mutation testing framework able to cope with large and complex Java software systems, while still being easily extensible with new experimental components. LittleDarwin addresses two existing problems in the domain of mutation testing: having a tool able to work within an industrial setting, and yet, be open to extension for cutting edge techniques provided by academia. LittleDarwin already offers higher-order mutation, null type mutants, mutant sampling, manual mutation, and mutant subsumption analysis. There is no tool today available with all these features that is able to work with typical industrial software systems.Comment: Pre-proceedings of the 7th IPM International Conference on Fundamentals of Software Engineerin

A next-generation sequencing approach to identify gene mutations in early-and late-onset hypertrophic cardiomyopathy patients of an Italian cohort

Sequencing of sarcomere protein genes in patients fulfilling the clinical diagnostic criteria for hypertrophic cardiomyopathy (HCM) identifies a disease-causing mutation in 35% to 60% of cases. Age at diagnosis and family history may increase the yield of mutations screening. In order to assess whether Next-Generation Sequencing (NGS) may fulfil the molecular diagnostic needs in HCM, we included 17 HCM-related genes in a sequencing panel run on PGM IonTorrent. We selected 70 HCM patients, 35 with early (≤25 years) and 35 with late (≥65 years) diagnosis of disease onset. All samples had a 98.6% average of target regions, with coverage higher than 20× (mean coverage 620×). We identified 41 different mutations (seven of them novel) in nine genes: MYBPC3 (17/41 = 41%); MYH7 (10/41 = 24%); TNNT2, CAV3 and MYH6 (3/41 = 7.5% each); TNNI3 (2/41 = 5%); GLA, MYL2, and MYL3 (1/41=2.5% each). Mutation detection rate was 30/35 (85.7%) in early-onset and 8/35 (22.9%) in late-onset HCM patients, respectively (p < 0.0001). The overall detection rate for patients with positive family history was 84%, and 90.5% in patients with early disease onset. In our study NGS revealed higher mutations yield in patients with early onset and with a family history of HCM. Appropriate patient selection can increase the yield of genetic testing and make diagnostic testing cost-effective

An Empirical Study of Off-by-one Loop Mutation

Context: Developing test cases that are measurably effective in finding faults in programs is a very challenging research problem. Mutation testing, a prominent technique developed to address this challenge, often becomes com- putationally too expensive for practical use due to the very large number of mutants that need to be analyzed. Objective: This paper evaluates the impact of One-by-one (OBO) loop mutation in reducing the cost of mutation analysis and investigates this technique\u27s effectiveness in measuring the strength or weakness of test suites. Method: A set of Java and C programs have been used to generate both OBO and traditional mutants. Mutation scores are computed and analyzed for both sets of mutants. An analysis of first order vs. higher order loop mutations have also been performed. Results: On average, 89.15% fewer mutants are generated by OBO op- erator in comparison to traditional operators while the two sets of muta- tion scores still remain highly positively correlated (correlation coefficient of .9228) indicating the usefulness of OBO operator in measuring test suite\u27s ef- fectiveness of finding faults in programs. We also investigate the relationship between first order OBO mutation (FOM) and their corresponding higher order mutations (HOM). We have found that OBO HOMs do not subsume their corresponding FOMs. Conclusion: We conclude that One-by-one (OBO) loop mutant operator, which targets specific program elements for mutation, can greatly reduce the number of mutants generated, and thus make the mutation analysis relatively inexpensive and practical while still being capable of providing useful measurement of the strength or weakness of a test suite. Our investigation into the relationship between higher order OBO mutants (HOM) and first order OBO mutants (FOM) has revealed that OBO HOMs usually do not add any value to the mutation analysis over the corresponding FOMs

Evaluating purifying selection in the mitochondrial DNA of various mammalian species

Mitochondrial DNA (mtDNA), the circular DNA molecule inside the mitochondria of all eukaryotic cells, has been shown to be under the effect of purifying selection in several species. Traditional testing of purifying selection has been based simply on ratios of nonsynonymous to synonymous mutations, without considering the relative age of each mutation, which can be determined by phylogenetic analysis of this non-recombining molecule. The incorporation of a mutation time-ordering from phylogeny and of predicted pathogenicity scores for nonsynonymous mutations allow a quantitative evaluation of the effects of purifying selection in human mtDNA. Here, by using this additional information, we show that purifying selection undoubtedly acts upon the mtDNA of other mammalian species/genera, namely Bos sp., Canis lupus, Mus musculus, Orcinus orca, Pan sp. and Sus scrofa. The effects of purifying selection were comparable in all species, leading to a significant major proportion of nonsynonymous variants with higher pathogenicity scores in the younger branches of the tree. We also derive recalibrated mutation rates for age estimates of ancestors of these various species and proposed a correction curve in order to take into account the effects of selection. Understanding this selection is fundamental to evolutionary studies and to the identification of deleterious mutations

Towards Systematic Mutations for and with ATL Model Transformations

Model transformation is a key technique to automate software engineering tasks, such as generating implementations of software systems from higher-level models. To enable this automation, transformation engines are used to synthesize various types of software artifacts from models, where the rules according to which these artifacts are generated are implemented by means of dedicated model transformation languages. Hence, the quality of the generated software artifacts depends on the quality of the transformation rules applied to generate them. Thus, there is the need for approaches to certify their behavior for a selected set of test models. As mutation analysis has proven useful as a practical testing approach, we propose a set of mutation operators for the ATLAS Transformation Language (ATL) derived by a comprehensive language-centric synthesis approach. We describe the rationale behind each of the mutation operators and propose an automated process to generate mutants for ATL transformations based on a combination of generic mutation operators and higher-order transformations. Finally, we describe a cost-effective solution for executing the obtained mutants.European Commission ICT Policy Support Programme 317859Ministerio de Ciencia e Innovación TIN2011-2379

A Model to Estimate First-Order Mutation Coverage from Higher-Order Mutation Coverage

The test suite is essential for fault detection during software development. First-order mutation coverage is an accurate metric to quantify the quality of the test suite. However, it is computationally expensive. Hence, the adoption of this metric is limited. In this study, we address this issue by proposing a realistic model able to estimate first-order mutation coverage using only higher-order mutation coverage. Our study shows how the estimation evolves along with the order of mutation. We validate the model with an empirical study based on 17 open-source projects.Comment: 2016 IEEE International Conference on Software Quality, Reliability, and Security. 9 page

Evaluating Random Mutant Selection at Class-Level in Projects with Non-Adequate Test Suites

Mutation testing is a standard technique to evaluate the quality of a test suite. Due to its computationally intensive nature, many approaches have been proposed to make this technique feasible in real case scenarios. Among these approaches, uniform random mutant selection has been demonstrated to be simple and promising. However, works on this area analyze mutant samples at project level mainly on projects with adequate test suites. In this paper, we fill this lack of empirical validation by analyzing random mutant selection at class level on projects with non-adequate test suites. First, we show that uniform random mutant selection underachieves the expected results. Then, we propose a new approach named weighted random mutant selection which generates more representative mutant samples. Finally, we show that representative mutant samples are larger for projects with high test adequacy.Comment: EASE 2016, Article 11 , 10 page