Evolutionary algorithms for the multi-objective test data generation problem

Software: Practice & Experience, 42(11):1331-1362Automatic test data generation is a very popular domain in the field of search-based software engineering. Traditionally, the main goal has been to maximize coverage. However, other objectives can be defined, such as the oracle cost, which is the cost of executing the entire test suite and the cost of checking the system behavior. Indeed, in very large software systems, the cost spent to test the system can be an issue, and then it makes sense by considering two conflicting objectives: maximizing the coverage and minimizing the oracle cost. This is what we did in this paper. We mainly compared two approaches to deal with the multi-objective test data generation problem: a direct multi-objective approach and a combination of a mono-objective algorithm together with multi-objective test case selection optimization. Concretely, in this work, we used four state-of-the-art multi-objective algorithms and two mono-objective evolutionary algorithms followed by a multi-objective test case selection based on Pareto efficiency. The experimental analysis compares these techniques on two different benchmarks. The first one is composed of 800 Java programs created through a program generator. The second benchmark is composed of 13 real programs extracted from the literature. In the direct multi-objective approach, the results indicate that the oracle cost can be properly optimized; however, the full branch coverage of the system poses a great challenge. Regarding the mono-objective algorithms, although they need a second phase of test case selection for reducing the oracle cost, they are very effective in maximizing the branch coverage.Spanish Ministry of Science and Innovation and FEDER under contract TIN2008-06491-C04-01 (the M project). Andalusian Government under contract P07-TIC-03044 (DIRICOM project)

A multi-objective evolutionary approach for automatic generation of test cases from state machines

Orientador: Eliane MartinsTese (doutorado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: A geração automática de casos de teste contribui tanto para melhorar a produtividade quanto para reduzir esforço e custo no processo de desenvolvimento de software. Neste trabalho é proposta uma abordagem, denominada MOST (Multi-Objective Search-based Testing approach from EFSM), para gerar casos de teste a partir de Máquina de Estados Finitos Estendida (MEFE) com a aplicação de uma técnica de otimização. No teste baseado em MEFE, é necessário encontrar uma sequência de entrada para exercitar um caminho no modelo, a fim de cobrir um critério de teste (e.g. todas as transições). Como as sequências podem ter diferentes tamanhos, motivou-se o desenvolvimento do algoritmo M-GEOvsl (Multi-Objective Generalized Extremal Optimization with variable string length) que permite gerar soluções de diferentes tamanhos. Além disso, por ser um algoritmo multiobjetivo, M-GEOvsl também possibilita que mais de um critério seja usado para avaliar as soluções. Com a aplicação desse algoritmo em MOST, tanto a cobertura da transição alvo quanto o tamanho da sequência são levados em consideração na geração de casos de teste. Para guiar a busca, são utilizadas informações das dependências do modelo. O algoritmo gera as sequências de entrada, incluindo os valores de seus parâmetros. Em MOST, um modelo executável da MEFE recebe como entrada os dados gerados pelo M-GEOvsl e produz dinamicamente os caminhos percorridos. Uma vez que os aspectos de controle e dados do modelo são considerados durante a execução do modelo, evita-se o problema de geração de caminhos infactíveis. Um caminho pode ser sintaticamente possível, mas semanticamente infactível, devido aos conitos de dados envolvidos no modelo. Para avaliar a abordagem proposta foram realizados vários experimentos com modelos da literatura e de aplicações reais. Os resultados da abordagem também foram comparados com os casos de teste obtidos em um trabalho relacionado.Abstract: Automated test case generation can improve the productivity as well as reduce effort and cost in the software development process. In this work an approach, named MOST (Multi- Objective Search-based Testing approach from EFSM), is proposed to generate test cases from Extended Finite State Machine (EFSM) using an optimization technique. In EFSM based testing, an input sequence should be found to sensitize a path in the model, in order to cover a test criterion (e.g. all transitions). As the sequences can have different lengths, it motivates the development of the M-GEOvsl (Multi-Objective Generalized Extremal Optimization with variable string length) algorithm that makes possible the generation of solutions with different lengths. Moreover, as a multiobjective algorithm, M-GEOvsl also allows to use more than one criterion to evaluate the solutions. Using this algorithm in MOST, the coverage of the target transition as well as the sequence length are taken into account in the test case generation. To guide the search, the information about the model dependences is used. The algorithm generates the input sequences, including the values of their parameters. In MOST, an executable model of the EFSM receives as input the data generated by M-GEOvsl and produces the traversed paths dynamically. Since the control and data aspects are considered during model execution, the problem of infeasible path generation is avoided. A path can be syntatically possible, but semantically infeasible, due to the data conicts in the model. In order to evaluate the proposed approach, experiments were performed with models of the literature and real-world applications. The results were also compared to the test cases obtained in a related workDoutoradoCiência da ComputaçãoDoutor em Ciência da Computaçã

Dynamic resource constrained multi-project scheduling problem with weighted earliness/tardiness costs

In this study, a conceptual framework is given for the dynamic multi-project scheduling problem with weighted earliness/tardiness costs (DRCMPSPWET) and a mathematical programming formulation of the problem is provided. In DRCMPSPWET, a project arrives on top of an existing project portfolio and a due date has to be quoted for the new project while minimizing the costs of schedule changes. The objective function consists of the weighted earliness tardiness costs of the activities of the existing projects in the current baseline schedule plus a term that increases linearly with the anticipated completion time of the new project. An iterated local search based approach is developed for large instances of this problem. In order to analyze the performance and behavior of the proposed method, a new multi-project data set is created by controlling the total number of activities, the due date tightness, the due date range, the number of resource types, and the completion time factor in an instance. A series of computational experiments are carried out to test the performance of the local search approach. Exact solutions are provided for the small instances. The results indicate that the local search heuristic performs well in terms of both solution quality and solution time

Evolutionary improvement of programs

Most applications of genetic programming (GP) involve the creation of an entirely new function, program or expression to solve a specific problem. In this paper, we propose a new approach that applies GP to improve existing software by optimizing its non-functional properties such as execution time, memory usage, or power consumption. In general, satisfying non-functional requirements is a difficult task and often achieved in part by optimizing compilers. However, modern compilers are in general not always able to produce semantically equivalent alternatives that optimize non-functional properties, even if such alternatives are known to exist: this is usually due to the limited local nature of such optimizations. In this paper, we discuss how best to combine and extend the existing evolutionary methods of GP, multiobjective optimization, and coevolution in order to improve existing software. Given as input the implementation of a function, we attempt to evolve a semantically equivalent version, in this case optimized to reduce execution time subject to a given probability distribution of inputs. We demonstrate that our framework is able to produce non-obvious optimizations that compilers are not yet able to generate on eight example functions. We employ a coevolved population of test cases to encourage the preservation of the function's semantics. We exploit the original program both through seeding of the population in order to focus the search, and as an oracle for testing purposes. As well as discussing the issues that arise when attempting to improve software, we employ rigorous experimental method to provide interesting and practical insights to suggest how to address these issues

DeepSQLi: Deep Semantic Learning for Testing SQL Injection

Security is unarguably the most serious concern for Web applications, to which SQL injection (SQLi) attack is one of the most devastating attacks. Automatically testing SQLi vulnerabilities is of ultimate importance, yet is unfortunately far from trivial to implement. This is because the existence of a huge, or potentially infinite, number of variants and semantic possibilities of SQL leading to SQLi attacks on various Web applications. In this paper, we propose a deep natural language processing based tool, dubbed DeepSQLi, to generate test cases for detecting SQLi vulnerabilities. Through adopting deep learning based neural language model and sequence of words prediction, DeepSQLi is equipped with the ability to learn the semantic knowledge embedded in SQLi attacks, allowing it to translate user inputs (or a test case) into a new test case, which is semantically related and potentially more sophisticated. Experiments are conducted to compare DeepSQLi with SQLmap, a state-of-the-art SQLi testing automation tool, on six real-world Web applications that are of different scales, characteristics and domains. Empirical results demonstrate the effectiveness and the remarkable superiority of DeepSQLi over SQLmap, such that more SQLi vulnerabilities can be identified by using a less number of test cases, whilst running much faster