mrstudyr: Retrospectively Studying the Effectiveness of Mutant Reduction Techniques

Mutation testing is a well-known method for measuring a test suite’s quality. However, due to its computational expense and intrinsic difficulties (e.g., detecting equivalent mutants and potentially checking a mutant’s status for each test), mutation testing is often challenging to practically use. To control the computational cost of mutation testing, many reduction strategies have been proposed (e.g., uniform random sampling over mutants). Yet, a stand-alone tool to compare the efficiency and effectiveness of these methods is heretofore unavailable. Since existing mutation testing tools are often complex and languagedependent, this paper presents a tool, called mrstudyr, that enables the “retrospective” study of mutant reduction methods using the data collected from a prior analysis of all mutants. Focusing on the mutation operators and the mutants that they produce, the presented tool allows developers to prototype and evaluate mutant reducers without being burdened by the implementation details of mutation testing tools. Along with describing mrstudyr’s design and overviewing the experimental results from using it, this paper inaugurates the public release of this open-source tool

Model-Based Security Testing

Security testing aims at validating software system requirements related to security properties like confidentiality, integrity, authentication, authorization, availability, and non-repudiation. Although security testing techniques are available for many years, there has been little approaches that allow for specification of test cases at a higher level of abstraction, for enabling guidance on test identification and specification as well as for automated test generation. Model-based security testing (MBST) is a relatively new field and especially dedicated to the systematic and efficient specification and documentation of security test objectives, security test cases and test suites, as well as to their automated or semi-automated generation. In particular, the combination of security modelling and test generation approaches is still a challenge in research and of high interest for industrial applications. MBST includes e.g. security functional testing, model-based fuzzing, risk- and threat-oriented testing, and the usage of security test patterns. This paper provides a survey on MBST techniques and the related models as well as samples of new methods and tools that are under development in the European ITEA2-project DIAMONDS.Comment: In Proceedings MBT 2012, arXiv:1202.582

Experiments in fault tolerant software reliability

Twenty functionally equivalent programs were built and tested in a multiversion software experiment. Following unit testing, all programs were subjected to an extensive system test. In the process sixty-one distinct faults were identified among the versions. Less than 12 percent of the faults exhibited varying degrees of positive correlation. The common-cause (or similar) faults spanned as many as 14 components. However, a majority of these faults were trivial, and easily detected by proper unit and/or system testing. Only two of the seven similar faults were difficult faults, and both were caused by specification ambiguities. One of these faults exhibited variable identical-and-wrong response span, i.e. response span which varied with the testing conditions and input data. Techniques that could have been used to avoid the faults are discussed. For example, it was determined that back-to-back testing of 2-tuples could have been used to eliminate about 90 percent of the faults. In addition, four of the seven similar faults could have been detected by using back-to-back testing of 5-tuples. It is believed that most, if not all, similar faults could have been avoided had the specifications been written using more formal notation, the unit testing phase was subject to more stringent standards and controls, and better tools for measuring the quality and adequacy of the test data (e.g. coverage) were used

Modeling collective rationality : an nonparametric test on experimental data.

We provide a .first nonparametric (revealed preference) test of the collective consumption model on the basis of experimental data. By using nonparametric testing tools and experimental data, we avoid the usual problems associated with parametric tests (e.g. non-verifiable parametric structure) and the use of ‘real life’ data sets (e.g. preference heterogeneity). In addition, our collective rationality test complements the existing nonparametric-experimental evidence on individual rationality. Focusing on dyads, we find that all observed consumption choices are consistent with the nonparametric collective rationality conditions. In fact, the consistency results for the parsimonious ‘egoistic’ collective consumption model (as a tool for describing dyads’ choice behavior) are closely similar to those for the individual rationality model (as a tool for describing individuals’.choice behavior). This suggest that for simple consumption decision settings, such as that considered in our experiment, the egoistic model may be useful for practical analysis. Still, our results also suggest that the more general collective consumption model, which accounts for consumption externalities and public consumption, can be useful even for modeling such simple decision settings. In fact, we can interpret that the appropriate model specification also depends on the specific dyad type (e.g. friends or partners; gender composition) and choice setting (e.g. public consumption or not) at hand.Collective consumption decisions; Generalized Axiom of Revealed Preference; Nonparametric analysis; Experimental data;

External validity in healthy public policy: application of the RE-AIM tool to the field of housing improvement

<b>Background</b><p></p> Researchers and publishers have called for improved reporting of external validity items and for testing of existing tools designed to assess reporting of items relevant to external validity. Few tools are available and most of this work has been done within the field of health promotion.<p></p> <b>Methods</b><p></p> We tested a tool assessing reporting of external validity items which was developed by Green & Glasgow on 39 studies assessing the health impacts of housing improvement. The tool was adapted to the topic area and criteria were developed to define the level of reporting, e.g. “some extent”. Each study was assessed by two reviewers.<p></p> <b>Results</b><p></p> The tool was applicable to the studies but some items required considerable editing to facilitate agreement between the two reviewers. Levels of reporting of the 17 external validity items were low (mean 6). The most commonly reported items related to outcomes. Details of the intervention were poorly reported. Study characteristics were not associated with variation in reporting.<p></p> <b>Conclusions</b><p></p> The Green & Glasgow tool was useful to assess reporting of external validity items but required tailoring to the topic area. In some public health evaluations the hypothesised impact is dependent on the intervention effecting change, e.g. improving socio-economic conditions. In such studies data confirming the function of the intervention may be as important as details of the components and implementation of the intervention

The Progress, Challenges, and Perspectives of Directed Greybox Fuzzing

Most greybox fuzzing tools are coverage-guided as code coverage is strongly correlated with bug coverage. However, since most covered codes may not contain bugs, blindly extending code coverage is less efficient, especially for corner cases. Unlike coverage-guided greybox fuzzers who extend code coverage in an undirected manner, a directed greybox fuzzer spends most of its time allocation on reaching specific targets (e.g., the bug-prone zone) without wasting resources stressing unrelated parts. Thus, directed greybox fuzzing (DGF) is particularly suitable for scenarios such as patch testing, bug reproduction, and specialist bug hunting. This paper studies DGF from a broader view, which takes into account not only the location-directed type that targets specific code parts, but also the behaviour-directed type that aims to expose abnormal program behaviours. Herein, the first in-depth study of DGF is made based on the investigation of 32 state-of-the-art fuzzers (78% were published after 2019) that are closely related to DGF. A thorough assessment of the collected tools is conducted so as to systemise recent progress in this field. Finally, it summarises the challenges and provides perspectives for future research.Comment: 16 pages, 4 figure