A Survey on Software Testing Techniques using Genetic Algorithm

The overall aim of the software industry is to ensure delivery of high quality software to the end user. To ensure high quality software, it is required to test software. Testing ensures that software meets user specifications and requirements. However, the field of software testing has a number of underlying issues like effective generation of test cases, prioritisation of test cases etc which need to be tackled. These issues demand on effort, time and cost of the testing. Different techniques and methodologies have been proposed for taking care of these issues. Use of evolutionary algorithms for automatic test generation has been an area of interest for many researchers. Genetic Algorithm (GA) is one such form of evolutionary algorithms. In this research paper, we present a survey of GA approach for addressing the various issues encountered during software testing.Comment: 13 Page

Automated metamorphic testing on the analyses of feature models

Copyright © 2010 Elsevier B.V. All rights reserved.Context: A feature model (FM) represents the valid combinations of features in a domain. The automated extraction of information from FMs is a complex task that involves numerous analysis operations, techniques and tools. Current testing methods in this context are manual and rely on the ability of the tester to decide whether the output of an analysis is correct. However, this is acknowledged to be time-consuming, error-prone and in most cases infeasible due to the combinatorial complexity of the analyses, this is known as the oracle problem.Objective: In this paper, we propose using metamorphic testing to automate the generation of test data for feature model analysis tools overcoming the oracle problem. An automated test data generator is presented and evaluated to show the feasibility of our approach.Method: We present a set of relations (so-called metamorphic relations) between input FMs and the set of products they represent. Based on these relations and given a FM and its known set of products, a set of neighbouring FMs together with their corresponding set of products are automatically generated and used for testing multiple analyses. Complex FMs representing millions of products can be efficiently created by applying this process iteratively.Results: Our evaluation results using mutation testing and real faults reveal that most faults can be automatically detected within a few seconds. Two defects were found in FaMa and another two in SPLOT, two real tools for the automated analysis of feature models. Also, we show how our generator outperforms a related manual suite for the automated analysis of feature models and how this suite can be used to guide the automated generation of test cases obtaining important gains in efficiency.Conclusion: Our results show that the application of metamorphic testing in the domain of automated analysis of feature models is efficient and effective in detecting most faults in a few seconds without the need for a human oracle.This work has been partially supported by the European Commission(FEDER)and Spanish Government under CICYT project SETI(TIN2009-07366)and the Andalusian Government project ISABEL(TIC-2533)

Metaheuristic Optimization Frameworks: a Survey and Benchmarking

This paper performs an unprecedented comparative study of Metaheuristic optimization frameworks. As criteria for comparison a set of 271 features grouped in 30 characteristics and 6 areas has been selected. These features include the different metaheuristic techniques covered, mechanisms for solution encoding, constraint handling, neighborhood specification, hybridization, parallel and distributed computation, software engineering best practices, documentation and user interface, etc. A metric has been defined for each feature so that the scores obtained by a framework are averaged within each group of features, leading to a final average score for each framework. Out of 33 frameworks ten have been selected from the literature using well-defined filtering criteria, and the results of the comparison are analyzed with the aim of identifying improvement areas and gaps in specific frameworks and the whole set. Generally speaking, a significant lack of support has been found for hyper-heuristics, and parallel and distributed computing capabilities. It is also desirable to have a wider implementation of some Software Engineering best practices. Finally, a wider support for some metaheuristics and hybridization capabilities is needed

JavaEvA : a Java based framework for Evolutionary Algorithms

Das Softwarepaket JavaEvA (eine Java Implementierung Evolutionärer Algorithmen) ist ein allgemeines modulares Framework für Optimierungsalgorithmen basierend auf einer Client-Server Architektur, das geeignet ist eine Vielzahl von Optimierungsproblemen zu lösen. Das Paket wurde mit dem Schwerpunkt entwickelt neue Verfahren im Bereich der Evolutionären Algorithmen einfach entwickeln und testen zu können und diese Verfahren letztlich in praktischen Anwendungen anzuwenden. JavaEvA beinhaltet Implementierungen der üblichen Evolutionären Verfahren wie zum Beispiel Genetische Algorithmen, die CHC Adaptive Search, Population Based Incremental Learning, Evolutionsstrategien, Modellunterstützte Evolutionsstrategien, Genetisches Programmieren und Grammatical Evolution. Zusätzlich erlaubt es das modulare Framework von JavaEvA eigene eventuell problemspezifische Optimierungsmodule zu ergänzen und mit den implementieren Verfahren zu vergleichen. Das JavaEvA Paket benutzt ein generisches Verfahren zur GUI Generierung und erlaubt so einen einfachen Objektorientierten Zugang zu allen relevanten Parametern eines Evolutionären Algorithmus. Das gleiche Verfahren generiert auch entsprechende GUI Elemente für neu entwickelte Methoden und vereinfacht so den Aufwand bei der Entwicklung neuer Methoden erheblich. Zusätzlich besteht die Möglichkeit spezialisiere GUI Elemente für einzelne Objekte zu entwickeln und in das bestehende System zu integrieren, um die Benutzerfreundlichkeit weiter zu erhöhen. Da es uns unmöglich ist jedwede potenzielle Anwendung oder Optimierungsproblem zu antizipieren, ist es aus praktischen Gründen fast immer nötig eigene Implementierungen des jeweiligen Anwendungsproblems zu erstellen. Um diesen Vorgang zu erleichtern bietet diese Anleitung zusätzliche Beispiele mit detaillierten Beschreibungen, wie man ein eigenes Problem implementieren kann und JavaEvA lediglich als Optimierungstoolbox integriert. Auf diese Weise behält ihre jeweilige Anwendung die vollständige Kontrolle über die verwendeten Verfahren und die anwendungsspezifische Darstellung der Optimierungsergebnisse.The package JavaEvA (a Java implementation of Evolutionary Algorithms) is a general modular framework with an inherent client server structure to solve practical optimization problems. This package was especially designed to test and develop new approaches for Evolutionary Algorithms and to utilize them in real-world applications. JavaEvA already provides implementations of the most common Evolutionary Algorithms, like Genetic Algorithms, CHC Adaptive Search, Population Based Incremental Learning, Evolution Strategies, Model-Assisted Evolution Strategies, Genetic Programming and Grammatical Evolution. In addition the modular framework of JavaEvA allows everyone to add their own optimization modules to meet their specific requirements. The JavaEvA package uses a generic GUI framework that allows GUI access to any member of a class if get and set methods are provided and an editor is defined for the given data type. This approach allows very fast development cycles, since hardly any additional effort is necessary for implementing GUI elements, while still at the same time user specific GUI elements can be developed and integrated to increase usability. Since we cannot anticipate specific optimization problem and requirements, it is necessary for users to define their optimization problem. Therefore, we provide an additional framework and explain how one can include JavaEvA in an existing Java project or how one can implement ones own optimization problem and optimize it by using JavaEvA. This gives users total control of the optimization algorithms used

Deviant: A Mutation Testing Tool for Solidity Smart Contracts

Blockchain in recent years has exploded in popularity with Ethereum being one of the leading blockchain platforms. Solidity is a widely used scripting language for creating smart contracts in Ethereum applications. Quality assurance in Solidity contracts is of critical importance because bugs or vulnerabilities can lead to a considerable loss of financial assets. However, it is unclear what level of quality assurance is provided in many of these applications. Mutation testing is the process of intentionally injecting faults into a target program and then running the provided test suite against the various injected faults. Mutation testing is used to evaluate the effectiveness of a test suite, measuring the test suite’s capability of covering certain types of faults. This thesis presents Deviant, the first implementation of a mutation testing tool for Solidity smart contracts. Deviant implements mutation operators that cover the unique features of Solidity according to our constructed fault model, in addition to traditional mutation operators that exist for other programming languages. Deviant has been applied to five open-source Solidity projects: MetaCoin [30], MultiSigWallet [31], Alice [29], aragonOS [32], and OpenZeppelin [33]. Experimental results show that the provided test suites result in low mutation scores. These results indicate that the provided tests cannot ensure high-level assurance of code quality. Such evaluation results offer important guidelines for Solidity developers to implement more effective tests in order to deliver trustworthy code and reduce the risk of financial loss

Sampling-Strategien zur Erzeugung von Szenarien für die simulationsbasierte Validierung von Fahrerassistenzsystemen

Scenario-based testing is a common approach to verify and validate Advanced Driving Assistance System / Autonomous Driving (ADAS/AD) of motor vehicles. The main challenge in scenario-based testing is the selection of a finite number of scenarios to represent an infinite amount of possible scenarios. Beyond that, there is no metric to evaluate scenarios thus the quality of the testing process. We introduce a generic process chain to ensure traceability and reproducibility of scenario selection, by generating scenarios automatically. A Feature Model (FM) builds the input data for our process chain. We identify three concepts to represent a scenario using a FM. We create a tool to transfer a configuration of the FM into a concrete scenario. A sample represents a set of scenarios, we define them as scenario suite. We evaluate the quality of a scenario suite by applying its scenarios to various mutants of driving functions in a simulation tool. The quality of the scenario suites is then determined by the number of discovered mutants. We evaluate the influence of various FMs in combination with common sampling algorithms such as ICPL, Chvatal, and IncLing, using an Autonomous Emergency Braking (AEB) as subject system. We discover a correlation between FM and mutation score as well as between mutation score and sampling algorithm. Within a scenario suite, we identify a strict separation between scenarios that are good to kill a mutant and those which are not. We discover, that sampling algorithms that aim for feature interaction coverage produce stronger scenario suites than feature-wise sampling algorithms. An evaluation of the relevance of single features on the mutation score provides features that are frequently involved in scenarios that are good to kill mutants. Beyond that, we discover a correlation between scenario suite and mutants that affects the mutant detection