The SOS Platform: Designing, Tuning and Statistically Benchmarking Optimisation Algorithms

open access articleWe present Stochastic Optimisation Software (SOS), a Java platform facilitating the algorithmic design process and the evaluation of metaheuristic optimisation algorithms. SOS reduces the burden of coding miscellaneous methods for dealing with several bothersome and time-demanding tasks such as parameter tuning, implementation of comparison algorithms and testbed problems, collecting and processing data to display results, measuring algorithmic overhead, etc. SOS provides numerous off-the-shelf methods including: (1) customised implementations of statistical tests, such as the Wilcoxon rank-sum test and the Holm–Bonferroni procedure, for comparing the performances of optimisation algorithms and automatically generating result tables in PDF and formats; (2) the implementation of an original advanced statistical routine for accurately comparing couples of stochastic optimisation algorithms; (3) the implementation of a novel testbed suite for continuous optimisation, derived from the IEEE CEC 2014 benchmark, allowing for controlled activation of the rotation on each testbed function. Moreover, we briefly comment on the current state of the literature in stochastic optimisation and highlight similarities shared by modern metaheuristics inspired by nature. We argue that the vast majority of these algorithms are simply a reformulation of the same methods and that metaheuristics for optimisation should be simply treated as stochastic processes with less emphasis on the inspiring metaphor behind them

A Memetic Algorithm for whole test suite generation

The generation of unit-level test cases for structural code coverage is a task well-suited to Genetic Algorithms. Method call sequences must be created that construct objects, put them into the right state and then execute uncovered code. However, the generation of primitive values, such as integers and doubles, characters that appear in strings, and arrays of primitive values, are not so straightforward. Often, small local changes are required to drive the value toward the one needed to execute some target structure. However, global searches like Genetic Algorithms tend to make larger changes that are not concentrated on any particular aspect of a test case. In this paper, we extend the Genetic Algorithm behind the EvoSuiTE test generation tool into a Memetic Algorithm, by equipping it with several local search operators. These operators are designed to efficiently optimize primitive values and other aspects of a test suite that allow the search for test cases to function more effectively. We evaluate our operators using a rigorous experimental methodology on over 12,000 Java classes, comprising open source classes of various different kinds, including numerical applications and text processors. Our study shows that increases in branch coverage of up to 53% are possible for an individual class in practice

A Systematic Review of the Application and Empirical Investigation of Search-Based Test Case Generation

Otsingupõhine tarkvara testimine kasutab metaheuristilisi algoritme, et automatiseerida testide genereerimist. Selle töö eesmärgiks on osaliselt taasluua 2010. aastal kirjutatud Ali et al. artikkel, et uurida, kuidas on aastatel 2008-2015 kasutatud metaheuristilisi algoritme testide loomiseks. See töö analüüsib, kuidas on antud artiklid koostatud ning kuidas neis on algoritmide maksumust ja efektiivsust hinnatud. Kogutud tulemusi võrreldakse Ali et al. tulemustega.Search based software testing uses metaheuristic algorithms to automate the generation of test cases. This thesis partially replicates a literature study published in 2010 by Ali et al. to determine how studies published in 2008-2015 use metaheuristic algorithms to automate the generation of test cases. The thesis analyses how these studies were conducted and how the cost-effectiveness is assessed in these papers. The trends detected in the new publications are compared to those presented in Ali et al

Using memetic algorithm for robustness testing of contract-based software models

Graph Transformation System (GTS) can formally specify the behavioral aspects of complex systems through graph-based contracts. Test suite generation under normal conditions from GTS specifications is a task well-suited to evolutionary algorithms such as Genetic and Particle Swarm Optimization (PSO) metaheuristics. However, testing the vulnerabilities of a system under unexpected events such as invalid inputs is essential. Furthermore, the mentioned global search algorithms tend to make big jumps in the system’s state-space that are not concentrated on particular test goals. In this paper, we extend the HGAPSO approach into a cost-aware Memetic Algorithm (MA) by making small local changes through a proposed local search operator to optimize coverage score and testing costs. Moreover, we test GTS specifications not only under normal events but also under unexpected situations. So, three coverage-based testing strategies are investigated, including normal testing, robustness testing, and a hybrid strategy. The effectiveness of the proposed test generation algorithm and the testing strategies are evaluated through a type of mutation analysis at the model-level. Our experimental results show that (1) the hybrid testing strategy outperforms normal and robustness testing strategies in terms of fault-detection capability, (2) the robustness testing is the most cost-efficient strategy, and (3) the proposed MA with the hybrid testing strategy outperforms the state-of-the-art global search algorithms

Pair-Wise Time-Aware Test Case Prioritization for Regression Testing

After maintenance, software requires regression testing for its validation. Prioritization of test cases for regression testing is required as software is tested under strict time and other constraints. A Pair-wise time-aware Test Case Prioritization (PTCP) technique has been proposed in this paper that determines the effectiveness of a test case on the basis of total number of faults present in software, number of faults detected till time, and the time of execution of different test cases. It selects that test case which determines maximum new faults, not yet detected, within minimum time. Thus prioritized test suite contains those test cases which are effective and tend to minimize repetitive faults detection. Through two comparative studies, it has been observed that with least wastage of time, the proposed technique performed equally well as other two parallel prioritizing techniques, Average Percentage of Fault Detection (APFD) based prioritization, and Optimal Test Case Prioritization (OTCP)

Chemical Reaction Optimization: A tutorial

Chemical Reaction Optimization (CRO) is a recently established metaheuristics for optimization, inspired by the nature of chemical reactions. A chemical reaction is a natural process of transforming the unstable substances to the stable ones. In microscopic view, a chemical reaction starts with some unstable molecules with excessive energy. The molecules interact with each other through a sequence of elementary reactions. At the end, they are converted to those with minimum energy to support their existence. This property is embedded in CRO to solve optimization problems. CRO can be applied to tackle problems in both the discrete and continuous domains. We have successfully exploited CRO to solve a broad range of engineering problems, including the quadratic assignment problem, neural network training, multimodal continuous problems, etc. The simulation results demonstrate that CRO has superior performance when compared with other existing optimization algorithms. This tutorial aims to assist the readers in implementing CRO to solve their problems. It also serves as a technical overview of the current development of CRO and provides potential future research directions. © 2012 The Author(s).published_or_final_versionSpringer Open Choice, 25 May 201

AVMf: An Open-Source Framework and Implementation of the Alternating Variable Method

The Alternating Variable Method (AVM) has been shown to be a fast and effective local search technique for search-based software engineering. Recent improvements to the AVM have generalized the representations it can optimize and have provably reduced its running time. However, until now, there has been no general, publicly-available implementation of the AVM incorporating all of these developments. We introduce AVMf, an object-oriented Java framework that provides such an implementation. AVMf is available from http://avmframework.org for configuration and use in a wide variety of projects