55,858 research outputs found
Mutation testing on an object-oriented framework: An experience report
This is the preprint version of the article - Copyright @ 2011 ElsevierContext
The increasing presence of Object-Oriented (OO) programs in industrial systems is progressively drawing the attention of mutation researchers toward this paradigm. However, while the number of research contributions in this topic is plentiful, the number of empirical results is still marginal and mostly provided by researchers rather than practitioners.
Objective
This article reports our experience using mutation testing to measure the effectiveness of an automated test data generator from a user perspective.
Method
In our study, we applied both traditional and class-level mutation operators to FaMa, an open source Java framework currently being used for research and commercial purposes. We also compared and contrasted our results with the data obtained from some motivating faults found in the literature and two real tools for the analysis of feature models, FaMa and SPLOT.
Results
Our results are summarized in a number of lessons learned supporting previous isolated results as well as new findings that hopefully will motivate further research in the field.
Conclusion
We conclude that mutation testing is an effective and affordable technique to measure the effectiveness of test mechanisms in OO systems. We found, however, several practical limitations in current tool support that should be addressed to facilitate the work of testers. We also missed specific techniques and tools to apply mutation testing at the system level.This work has been partially supported by the European Commission (FEDER) and Spanish Government under CICYT Project SETI (TIN2009-07366) and the Andalusian Government Projects ISABEL (TIC-2533) and THEOS (TIC-5906)
Evaluating Random Mutant Selection at Class-Level in Projects with Non-Adequate Test Suites
Mutation testing is a standard technique to evaluate the quality of a test
suite. Due to its computationally intensive nature, many approaches have been
proposed to make this technique feasible in real case scenarios. Among these
approaches, uniform random mutant selection has been demonstrated to be simple
and promising. However, works on this area analyze mutant samples at project
level mainly on projects with adequate test suites. In this paper, we fill this
lack of empirical validation by analyzing random mutant selection at class
level on projects with non-adequate test suites. First, we show that uniform
random mutant selection underachieves the expected results. Then, we propose a
new approach named weighted random mutant selection which generates more
representative mutant samples. Finally, we show that representative mutant
samples are larger for projects with high test adequacy.Comment: EASE 2016, Article 11 , 10 page
Semantic mutation testing
This is the Pre-print version of the Article. The official published version can be obtained from the link below - Copyright @ 2011 ElsevierMutation testing is a powerful and flexible test technique. Traditional mutation testing makes a small change to the syntax of a description (usually a program) in order to create a mutant. A test suite is considered to be good if it distinguishes between the original description and all of the (functionally non-equivalent) mutants. These mutants can be seen as representing potential small slips and thus mutation testing aims to produce a test suite that is good at finding such slips. It has also been argued that a test suite that finds such small changes is likely to find larger changes. This paper describes a new approach to mutation testing, called semantic mutation testing. Rather than mutate the description, semantic mutation testing mutates the semantics of the language in which the description is written. The mutations of the semantics of the language represent possible misunderstandings of the description language and thus capture a different class of faults. Since the likely misunderstandings are highly context dependent, this context should be used to determine which semantic mutants should be produced. The approach is illustrated through examples with statecharts and C code. The paper also describes a semantic mutation testing tool for C and the results of experiments that investigated the nature of some semantic mutation operators for C
Measuring Coverage of Prolog Programs Using Mutation Testing
Testing is an important aspect in professional software development, both to
avoid and identify bugs as well as to increase maintainability. However,
increasing the number of tests beyond a reasonable amount hinders development
progress. To decide on the completeness of a test suite, many approaches to
assert test coverage have been suggested. Yet, frameworks for logic programs
remain scarce.
In this paper, we introduce a framework for Prolog programs measuring test
coverage using mutations. We elaborate the main ideas of mutation testing and
transfer them to logic programs. To do so, we discuss the usefulness of
different mutations in the context of Prolog and empirically evaluate them in a
new mutation testing framework on different examples.Comment: 16 pages, Accepted for presentation in WFLP 201
Dynamic Mutant Subsumption Analysis using LittleDarwin
Many academic studies in the field of software testing rely on mutation
testing to use as their comparison criteria. However, recent studies have shown
that redundant mutants have a significant effect on the accuracy of their
results. One solution to this problem is to use mutant subsumption to detect
redundant mutants. Therefore, in order to facilitate research in this field, a
mutation testing tool that is capable of detecting redundant mutants is needed.
In this paper, we describe how we improved our tool, LittleDarwin, to fulfill
this requirement
A Model to Estimate First-Order Mutation Coverage from Higher-Order Mutation Coverage
The test suite is essential for fault detection during software development.
First-order mutation coverage is an accurate metric to quantify the quality of
the test suite. However, it is computationally expensive. Hence, the adoption
of this metric is limited. In this study, we address this issue by proposing a
realistic model able to estimate first-order mutation coverage using only
higher-order mutation coverage. Our study shows how the estimation evolves
along with the order of mutation. We validate the model with an empirical study
based on 17 open-source projects.Comment: 2016 IEEE International Conference on Software Quality, Reliability,
and Security. 9 page
Extreme genetic fragility of the HIV-1 capsid
Genetic robustness, or fragility, is defined as the ability, or lack thereof, of a biological entity to maintain function in the face of mutations. Viruses that replicate via RNA intermediates exhibit high mutation rates, and robustness should be particularly advantageous to them. The capsid (CA) domain of the HIV-1 Gag protein is under strong pressure to conserve functional roles in viral assembly, maturation, uncoating, and nuclear import. However, CA is also under strong immunological pressure to diversify. Therefore, it would be particularly advantageous for CA to evolve genetic robustness. To measure the genetic robustness of HIV-1 CA, we generated a library of single amino acid substitution mutants, encompassing almost half the residues in CA. Strikingly, we found HIV-1 CA to be the most genetically fragile protein that has been analyzed using such an approach, with 70% of mutations yielding replication-defective viruses. Although CA participates in several steps in HIV-1 replication, analysis of conditionally (temperature sensitive) and constitutively non-viable mutants revealed that the biological basis for its genetic fragility was primarily the need to coordinate the accurate and efficient assembly of mature virions. All mutations that exist in naturally occurring HIV-1 subtype B populations at a frequency >3%, and were also present in the mutant library, had fitness levels that were >40% of WT. However, a substantial fraction of mutations with high fitness did not occur in natural populations, suggesting another form of selection pressure limiting variation in vivo. Additionally, known protective CTL epitopes occurred preferentially in domains of the HIV-1 CA that were even more genetically fragile than HIV-1 CA as a whole. The extreme genetic fragility of HIV-1 CA may be one reason why cell-mediated immune responses to Gag correlate with better prognosis in HIV-1 infection, and suggests that CA is a good target for therapy and vaccination strategies
Evolutionary mutation testing for IoT with recorded and generated events
Mutation testing is a testing technique that has been applied successfully to several programming languages. Despite its benefits for software testing, the high computational cost of mutation testing has kept it from being widely used. Several refinements have been proposed to reduce its cost by reducing the number of generated mutants; one of those is evolutionary mutation testing (EMT). Evolutionary mutation testing aims at generating a reduced set of mutants with an evolutionary algorithm, which searches for potentially equivalent and difficult to kill mutants that help improve the test suite. Evolutionary mutation testing has been evaluated in two contexts so far, ie, web service compositions and object‐oriented C++ programmes. This study explores its performance when applied to event processing language queries of various domains. This study also considers the impact of the test data, since a lack of events or the need to have specific values in them can hinder testing. The effectiveness of evolutionary mutation testing with the original test data generators and the new internet of things test event generator tool is compared in multiple case studies
- …