Automated unique input output sequence generation for conformance testing of FSMs

This paper describes a method for automatically generating unique input output (UIO) sequences for FSM conformance testing. UIOs are used in conformance testing to verify the end state of a transition sequence. UIO sequence generation is represented as a search problem and genetic algorithms are used to search this space. Empirical evidence indicates that the proposed method yields considerably better (up to 62% better) results compared with random UIO sequence generation

Sensitivity of Ag/Al Interface Specific Resistances to Interfacial Intermixing

We have measured an Ag/Al interface specific resistance, 2AR(Ag/Al)(111) = 1.4 fOhm-m^2, that is twice that predicted for a perfect interface, 50% larger than for a 2 ML 50%-50% alloy, and even larger than our newly predicted 1.3 fOhmm^2 for a 4 ML 50%-50% alloy. Such a large value of 2ARAg/Al(111) confirms a predicted sensitivity to interfacial disorder and suggests an interface greater than or equal to 4 ML thick. From our calculations, a predicted anisotropy ratio, 2AR(Ag/Al)(001)/2AR(Ag/Al)(111), of more then 4 for a perfect interface, should be reduced to less than 2 for a 4 ML interface, making it harder to detect any such anisotropy.Comment: 3 pages, 2 figures, 1 table. In Press: Journal of Applied Physic

Automated Search for Good Coverage Criteria: Moving from Code Coverage to Fault Coverage Through Search-Based Software Engineering

We propose to use Search-Based Software Engineering to automatically evolve coverage criteria that are well correlated with fault revelation, through the use of existing fault databases. We explain how problems of bloat and overfitting can be ameliorated in our approach, and show how this new method will yield insight into faults — as well as better guidance for Search-Based Software Testing

Test oracle assessment and improvement

We introduce a technique for assessing and improving test oracles by reducing the incidence of both false positives and false negatives. We prove that our approach can always result in an increase in the mutual information between the actual and perfect oracles. Our technique combines test case generation to reveal false positives and mutation testing to reveal false negatives. We applied the decision support tool that implements our oracle improvement technique to five real-world subjects. The experimental results show that the fault detection rate of the oracles after improvement increases, on average, by 48.6% (86% over the implicit oracle). Three actual, exposed faults in the studied systems were subsequently confirmed and fixed by the developers

Enhancing Genetic Improvement of Software with Regression Test Selection

Genetic improvement uses artificial intelligence to automatically improve software with respect to non-functional properties (AI for SE). In this paper, we propose the use of existing software engineering best practice to enhance Genetic Improvement (SE for AI). We conjecture that existing Regression Test Selection (RTS) techniques (which have been proven to be efficient and effective) can and should be used as a core component of the GI search process for maximising its effectiveness. To assess our idea, we have carried out a thorough empirical study assessing the use of both dynamic and static RTS techniques with GI to improve seven real-world software programs. The results of our empirical evaluation show that incorporation of RTS within GI significantly speeds up the whole GI process, making it up to 78% faster on our benchmark set, being still able to produce valid software improvements. Our findings are significant in that they can save hours to days of computational time, and can facilitate the uptake of GI in an industrial setting, by significantly reducing the time for the developer to receive feedback from such an automated technique. Therefore, we recommend the use of RTS in future test-based automated software improvement work. Finally, we hope this successful application of SE for AI will encourage other researchers to investigate further applications in this area

Artifact for Enhancing Genetic Improvement of Software with Regression Test Selection

We present in this document the basic information needed to download, unpack, and then interpret the instructions we provide as requested in the ICSE 2021 Artifact Submission Guidelines. The artifact contains all the subject programs, scripts, tools, results, and a series of guidelines on how to use them. We aim at obtaining the badges of Available and Reusable. In order to do so, we have added all the components needed for the full execution of the experiments and analyses as we originally did during the writing of our paper, making it readily available. Furthermore, we have included instructions to the INSTALL file on how to add new programs to the experiments and analyses, making it reusable for the next researchers that intend to replicate or extend our experiments

Using formal methods to support testing

Formal methods and testing are two important approaches that assist in the development of high quality software. While traditionally these approaches have been seen as rivals, in recent years a new consensus has developed in which they are seen as complementary. This article reviews the state of the art regarding ways in which the presence of a formal specification can be used to assist testing

AVMf: An Open-Source Framework and Implementation of the Alternating Variable Method

The Alternating Variable Method (AVM) has been shown to be a fast and effective local search technique for search-based software engineering. Recent improvements to the AVM have generalized the representations it can optimize and have provably reduced its running time. However, until now, there has been no general, publicly-available implementation of the AVM incorporating all of these developments. We introduce AVMf, an object-oriented Java framework that provides such an implementation. AVMf is available from http://avmframework.org for configuration and use in a wide variety of projects

Constructing Search Spaces for Search-Based Software Testing Using Neural Networks

A central requirement for any Search-Based Software Testing (SBST) technique is a convenient and meaningful fitness landscape. Whether one follows a targeted or a diversification driven strategy, a search landscape needs to be large, continuous, easy to construct and representative of the underlying property of interest. Constructing such a landscape is not a trivial task often requiring a significant manual effort by an expert. We present an approach for constructing meaningful and convenient fitness landscapes using neural networks (NN) – for targeted and diversification strategies alike. We suggest that output of an NN predictor can be interpreted as a fitness for a targeted strategy. The NN is trained on a corpus of execution traces and various properties of interest, prior to searching. During search, the trained NN is queried to predict an estimate of a property given an execution trace. The outputs of the NN form a convenient search space which is strongly representative of a number of properties. We believe that such a search space can be readily used for driving a search towards specific properties of interest. For a diversification strategy, we propose the use of an autoencoder; a mechanism for compacting data into an n-dimensional “latent” space. In it, datapoints are arranged according to the similarity of their salient features. We show that a latent space of execution traces possesses characteristics of a convenient search landscape: it is continuous, large and crucially, it defines a notion of similarity to arbitrary observations