Using control flow analysis to improve the effectiveness of incremental mutation testing

Incremental Mutation Testing attempts to make mutation testing less expensive by applying it incrementally to a system as it evolves. This approach fits current trends of iterative software development with the main idea being that by carrying out mutation analysis in frequent bite-sized chunks focused on areas of the code which have changed, one can build confidence in the adequacy of a test suite incrementally. Yet this depends on how precisely one can characterise the effects of a change to a program. The original technique uses a naïve approach whereby changes are characterised only by syntactic changes. In this paper we propose bolstering incremental mutation testing by using control flow analysis to identify semantic repercussions which a syntactic change will have on a system. Our initial results based on two case studies demonstrate that numerous relevant mutants which would have otherwise not been considered using the naïve approach, are now being generated. However, the cost of identifying these mutants is significant when compared to the naïve approach, although it remains advantageous when compared to traditional mutation testing so long as the increment is sufficiently small.peer-reviewe

Improving the cost effectiveness equation of cascade testing for Familial Hypercholesterolaemia (FH)

Purpose of Review : Many International recommendations for the management of Familial Hypercholesterolaemia (FH) propose the use of Cascade Testing (CT) using the family mutation to unambiguously identify affected relatives. In the current economic climate DNA information is often regarded as too expensive. Here we review the literature and suggest strategies to improve cost effectiveness of CT. Recent findings : Advances in next generation sequencing have both speeded up the time taken for a genetic diagnosis and reduced costs. Also, it is now clear that, in the majority of patients with a clinical diagnosis of FH where no mutation can be found, the most likely cause of their elevated LDL-cholesterol (LDL-C) is because they have inherited a greater number than average of common LDL-C raising variants in many different genes. The major cost driver for CT is not DNA testing but of treatment over the remaining lifetime of the identified relative. With potent statins now off-patent, the overall cost has reduced considerably, and combining these three factors, a FH service based around DNA-CT is now less than 25% of that estimated by NICE in 2009. Summary : While all patients with a clinical diagnosis of FH need to have their LDL-C lowered, CT should be focused on those with the monogenic form and not the polygenic form

Jumble Java Byte Code to Measure the Effectiveness of Unit Tests

Jumble is a byte code level mutation testing tool for Java which inter-operates with JUnit. It has been designed to operate in an industrial setting with large projects. Heuristics have been included to speed the checking of mutations, for example, noting which test fails for each mutation and running this first in subsequent mutation checks. Significant effort has been put into ensuring that it can test code which uses custom class loading and reflection. This requires careful attention to class path handling and coexistence with foreign class-loaders. Jumble is currently used on a continuous basis within an agile programming environment with approximately 370,000 lines of Java code under source control. This checks out project code every fifteen minutes and runs an incremental set of unit tests and mutation tests for modified classes. Jumble is being made available as open source

The Progress, Challenges, and Perspectives of Directed Greybox Fuzzing

Most greybox fuzzing tools are coverage-guided as code coverage is strongly correlated with bug coverage. However, since most covered codes may not contain bugs, blindly extending code coverage is less efficient, especially for corner cases. Unlike coverage-guided greybox fuzzers who extend code coverage in an undirected manner, a directed greybox fuzzer spends most of its time allocation on reaching specific targets (e.g., the bug-prone zone) without wasting resources stressing unrelated parts. Thus, directed greybox fuzzing (DGF) is particularly suitable for scenarios such as patch testing, bug reproduction, and specialist bug hunting. This paper studies DGF from a broader view, which takes into account not only the location-directed type that targets specific code parts, but also the behaviour-directed type that aims to expose abnormal program behaviours. Herein, the first in-depth study of DGF is made based on the investigation of 32 state-of-the-art fuzzers (78% were published after 2019) that are closely related to DGF. A thorough assessment of the collected tools is conducted so as to systemise recent progress in this field. Finally, it summarises the challenges and provides perspectives for future research.Comment: 16 pages, 4 figure

Evaluating Random Mutant Selection at Class-Level in Projects with Non-Adequate Test Suites

Mutation testing is a standard technique to evaluate the quality of a test suite. Due to its computationally intensive nature, many approaches have been proposed to make this technique feasible in real case scenarios. Among these approaches, uniform random mutant selection has been demonstrated to be simple and promising. However, works on this area analyze mutant samples at project level mainly on projects with adequate test suites. In this paper, we fill this lack of empirical validation by analyzing random mutant selection at class level on projects with non-adequate test suites. First, we show that uniform random mutant selection underachieves the expected results. Then, we propose a new approach named weighted random mutant selection which generates more representative mutant samples. Finally, we show that representative mutant samples are larger for projects with high test adequacy.Comment: EASE 2016, Article 11 , 10 page