Dynamic generation of test cases with metaheuristics

The resolution of optimization problems is of great interest nowadays and has encouraged the development of various information technology methods to attempt solving them. There are several problems related to Software Engineering that can be solved by using this approach. In this paper, a new alternative based on the combination of population metaheuristics with a Tabu List to solve the problem of test cases generation when testing software is presented. This problem is of great importance for the development of software with a high computational cost and which is generally hard to solve. The performance of the solution proposed has been tested on a set of varying complexity programs. The results obtained show that the method proposed allows obtaining a reduced test data set in a suitable timeframe and with a greater coverage than conventional methods such as Random Method or Tabu Search.I Workshop Innovación en Sistemas de Software (WISS)Red de Universidades con Carreras en Informática (RedUNCI

Parameterizing Random Test Data According to Equivalence Classes

We are concerned with the problem of detecting bugs in machine learning applications. In the absence of sufficient real-world data, creating suitably large data sets for testing can be a difficult task. Random testing is one solution, but may have limited effectiveness in cases in which a reliable test oracle does not exist, as is the case of the machine learning applications of interest. To address this problem, we have developed an approach to creating data sets called "parameterized random data generation"Â. Our data generation framework allows us to isolate or combine different equivalence classes as desired, and then randomly generate large data sets using the properties of those equivalence classes as parameters. This allows us to take advantage of randomness but still have control over test case selection at the system testing level. We present our findings from using the approach to test two different machine learning ranking applications

Evolving Test Environments to Identify Faults in Swarm Robotics Algorithms

Swarm robotic systems are often considered to be dependable. However, there is little empirical evidence or theoretical analysis showing that dependability is an inherent property of all swarm robotic system. Recent literature has identified potential issues with respect to dependability within certain types of swarm robotic algorithms. There appears to be a dearth of literature relating to the testing of swarm robotic systems; this provides motivation for the development of the novel testing methods for swarm robotic systems presented in this paper. We present a search based approach, using genetic algorithms, for the automated identification of unintended behaviors during the execution of a flocking type algorithm, implemented on a simulated robotic swarm. Results show that this proposed approach is able to reveal faults in such flocking algorithms and has the potential to be used in further swarm robotic applications

A Comparative Study of Randomized Constraint Solvers for Random-Symbolic Testing

The complexity of constraints is a major obstacle for constraint-based software verification. Automatic constraint solvers are fundamentally incomplete: input constraints often build on some undecidable theory or some theory the solver does not support. This paper proposes and evaluates several randomized solvers to address this issue. We compare the effectiveness of a symbolic solver (CVC3), a random solver, three hybrid solvers (i.e., mix of random and symbolic), and two heuristic search solvers. We evaluate the solvers on two benchmarks: one consisting of manually generated constraints and another generated with a concolic execution of 8 subjects. In addition to fully decidable constraints, the benchmarks include constraints with non-linear integer arithmetic, integer modulo and division, bitwise arithmetic, and floating-point arithmetic. As expected symbolic solving (in particular, CVC3) subsumes the other solvers for the concolic execution of subjects that only generate decidable constraints. For the remaining subjects the solvers are complementary

Code coverage of adaptive random testing

Random testing is a basic software testing technique that can be used to assess the software reliability as well as to detect software failures. Adaptive random testing has been proposed to enhance the failure-detection capability of random testing. Previous studies have shown that adaptive random testing can use fewer test cases than random testing to detect the first software failure. In this paper, we evaluate and compare the performance of adaptive random testing and random testing from another perspective, that of code coverage. As shown in various investigations, a higher code coverage not only brings a higher failure-detection capability, but also improves the effectiveness of software reliability estimation. We conduct a series of experiments based on two categories of code coverage criteria: structure-based coverage, and fault-based coverage. Adaptive random testing can achieve higher code coverage than random testing with the same number of test cases. Our experimental results imply that, in addition to having a better failure-detection capability than random testing, adaptive random testing also delivers a higher effectiveness in assessing software reliability, and a higher confidence in the reliability of the software under test even when no failure is detected

Dynamic generation of test cases with metaheuristics

The resolution of optimization problems is of great interest nowadays and has encouraged the development of various information technology methods to attempt solving them. There are several problems related to Software Engineering that can be solved by using this approach. In this paper, a new alternative based on the combination of population metaheuristics with a Tabu List to solve the problem of test cases generation when testing software is presented. This problem is of great importance for the development of software with a high computational cost and which is generally hard to solve. The performance of the solution proposed has been tested on a set of varying complexity programs. The results obtained show that the method proposed allows obtaining a reduced test data set in a suitable timeframe and with a greater coverage than conventional methods such as Random Method or Tabu Search.Facultad de Informátic