Branch-coverage testability transformation for unstructured programs

Test data generation by hand is a tedious, expensive and error-prone activity, yet testing is a vital part of the development process. Several techniques have been proposed to automate the generation of test data, but all of these are hindered by the presence of unstructured control flow. This paper addresses the problem using testability transformation. Testability transformation does not preserve the traditional meaning of the program, rather it deals with preserving test-adequate sets of input data. This requires new equivalence relations which, in turn, entail novel proof obligations. The paper illustrates this using the branch coverage adequacy criterion and develops a branch adequacy equivalence relation and a testability transformation for restructuring. It then presents a proof that the transformation preserves branch adequacy

Generating feasible transition paths for testing from an extended finite state machine (EFSM)

The problem of testing from an extended finite state machine (EFSM) can be expressed in terms of finding suitable paths through the EFSM and then deriving test data to follow the paths. A chosen path may be infeasible and so it is desirable to have methods that can direct the search for appropriate paths through the EFSM towards those that are likely to be feasible. However, generating feasible transition paths (FTPs) for model based testing is a challenging task and is an open research problem. This paper introduces a novel fitness metric that analyzes data flow dependence among the actions and conditions of the transitions in order to estimate the feasibility of a transition path. The proposed fitness metric is evaluated by being used in a genetic algorithm to guide the search for FTPs

Evolutionary testing supported by slicing and transformation

Evolutionary testing is a search based approach to the automated generation of systematic test data, in which the search is guided by the test data adequacy criterion. Two problems for evolutionary testing are the large size of the search space and structural impediments in the implementation of the program which inhibit the formulation of a suitable fitness function to guide the search. In this paper we claim that slicing can be used to narrow the search space and transformation can be applied to the problem of structural impediments. The paper presents examples of how these two techniques have been successfully employed to make evolutionary testing both more efficient and more effective

A survey on software testability

Context: Software testability is the degree to which a software system or a unit under test supports its own testing. To predict and improve software testability, a large number of techniques and metrics have been proposed by both practitioners and researchers in the last several decades. Reviewing and getting an overview of the entire state-of-the-art and state-of-the-practice in this area is often challenging for a practitioner or a new researcher. Objective: Our objective is to summarize the body of knowledge in this area and to benefit the readers (both practitioners and researchers) in preparing, measuring and improving software testability. Method: To address the above need, the authors conducted a survey in the form of a systematic literature mapping (classification) to find out what we as a community know about this topic. After compiling an initial pool of 303 papers, and applying a set of inclusion/exclusion criteria, our final pool included 208 papers. Results: The area of software testability has been comprehensively studied by researchers and practitioners. Approaches for measurement of testability and improvement of testability are the most-frequently addressed in the papers. The two most often mentioned factors affecting testability are observability and controllability. Common ways to improve testability are testability transformation, improving observability, adding assertions, and improving controllability. Conclusion: This paper serves for both researchers and practitioners as an "index" to the vast body of knowledge in the area of testability. The results could help practitioners measure and improve software testability in their projects

A search-based technique for testing from extended finite state machine model

Extended finite state machines (EFSMs), and languages such as state-charts that are similar to EFSMs, are widely used to model state-based systems. When testing from an EFSM M it is common to aim to produce a set of test sequences (input sequences) that satisfies a test criterion that relates to the transition paths (TPs) of M that are executed by the test sequences. For example, we might require that the set of TPs triggered includes all of the transitions of M. One approach to generating such a set of test sequences is to split the problem into two stages: choosing a set of TPs that achieves the test criterion and then producing test sequences to trigger these TPs. However, the EFSM may contain infeasible TPs and the problem of generating a test sequence to trigger a given feasible TP (FTP) is generally uncomputable. In this paper we present a search-based approach that uses two techniques: (1) A TP fitness metric based on our previous work that estimates the feasibility of a given transition path; and (2) A fitness function to guide the search for a test sequence to trigger a given FTP. We evaluated our approach on five EFSMs: A simple in-flight safety system; a class II transport protocol; a lift system; an ATM; and the Inres initiator. In the experiments the proposed approach successfully tested approximately 96.75 % of the transitions and the proposed test sequence generation technique triggered all of the generated FTPs

Learning to Accelerate Symbolic Execution via Code Transformation

Symbolic execution is an effective but expensive technique for automated test generation. Over the years, a large number of refined symbolic execution techniques have been proposed to improve its efficiency. However, the symbolic execution efficiency problem remains, and largely limits the application of symbolic execution in practice. Orthogonal to refined symbolic execution, in this paper we propose to accelerate symbolic execution through semantic-preserving code transformation on the target programs. During the initial stage of this direction, we adopt a particular code transformation, compiler optimization, which is initially proposed to accelerate program concrete execution by transforming the source program into another semantic-preserving target program with increased efficiency (e.g., faster or smaller). However, compiler optimizations are mostly designed to accelerate program concrete execution rather than symbolic execution. Recent work also reported that unified settings on compiler optimizations that can accelerate symbolic execution for any program do not exist at all. Therefore, in this work we propose a machine-learning based approach to tuning compiler optimizations to accelerate symbolic execution, whose results may also aid further design of specific code transformations for symbolic execution. In particular, the proposed approach LEO separates source-code functions and libraries through our program-splitter, and predicts individual compiler optimization (i.e., whether a type of code transformation is chosen) separately through analyzing the performance of existing symbolic execution. Finally, LEO applies symbolic execution on the code transformed by compiler optimization (through our local-optimizer). We conduct an empirical study on GNU Coreutils programs using the KLEE symbolic execution engine. The results show that LEO significantly accelerates symbolic execution, outperforming the default KLEE configurations (i.e., turning on/off all compiler optimizations) in various settings, e.g., with the default training/testing time, LEO achieves the highest line coverage in 50/68 programs, and its average improvement rate on all programs is 46.48%/88.92% in terms of line coverage compared with turning on/off all compiler optimizations

Enhancing the Process of Testing Object -Oriented Systems.

Testing is a crucial step in the overall system development process. Using testing techniques that support features of the underlying software paradigm more effectively tests program than do testing techniques that support features of other paradigms. Systems developed with the object-oriented paradigm require techniques that support object-oriented features such as inheritance, data abstraction, encapsulation, and dynamic binding. Many techniques that are used to test systems developed with the structured paradigm are not sufficient for the testing of object-oriented systems. The goal of this research is to develop methods that will improve the process of testing object-oriented systems. Specifically, emphasis is given to improving the level of testing of methods because the level of method testing is generally considered inadequate. Algorithms are included that identify the set of methods, both interobject and intraobject, that should be tested for a given system. These algorithms are implemented as a part of an automated testing system that derives a framework for the testing of methods. This system includes the automatic generation of test drivers to facilitate the testing. It captures the results of tests for the purposes of reuse for future system maintenance. This framework provides the software engineer who is testing a system a mechanism to determine the level of method coverage that has been achieved in the testing process

The miracle of the Septuagint and the promise of data mining in economics

This paper argues that the sometimes-conflicting results of a modern revisionist literature on data mining in econometrics reflect different approaches to solving the central problem of model uncertainty in a science of non-experimental data. The literature has entered an exciting phase with theoretical development, methodological reflection, considerable technological strides on the computing front and interesting empirical applications providing momentum for this branch of econometrics. The organising principle for this discussion of data mining is a philosophical spectrum that sorts the various econometric traditions according to their epistemological assumptions (about the underlying data-generating-process DGP) starting with nihilism at one end and reaching claims of encompassing the DGP at the other end; call it the DGP-spectrum. In the course of exploring this spectrum the reader will encounter various Bayesian, specific-to-general (S-G) as well general-to-specific (G-S) methods. To set the stage for this exploration the paper starts with a description of data mining, its potential risks and a short section on potential institutional safeguards to these problems.Data mining, model selection, automated model selection, general to specific modelling, extreme bounds analysis, Bayesian model selection

An integrated search-based approach for automatic testing from extended finite state machine (EFSM) models

This is the post-print version of the Article - Copyright @ 2011 ElsevierThe extended finite state machine (EFSM) is a modelling approach that has been used to represent a wide range of systems. When testing from an EFSM, it is normal to use a test criterion such as transition coverage. Such test criteria are often expressed in terms of transition paths (TPs) through an EFSM. Despite the popularity of EFSMs, testing from an EFSM is difficult for two main reasons: path feasibility and path input sequence generation. The path feasibility problem concerns generating paths that are feasible whereas the path input sequence generation problem is to find an input sequence that can traverse a feasible path. While search-based approaches have been used in test automation, there has been relatively little work that uses them when testing from an EFSM. In this paper, we propose an integrated search-based approach to automate testing from an EFSM. The approach has two phases, the aim of the first phase being to produce a feasible TP (FTP) while the second phase searches for an input sequence to trigger this TP. The first phase uses a Genetic Algorithm whose fitness function is a TP feasibility metric based on dataflow dependence. The second phase uses a Genetic Algorithm whose fitness function is based on a combination of a branch distance function and approach level. Experimental results using five EFSMs found the first phase to be effective in generating FTPs with a success rate of approximately 96.6%. Furthermore, the proposed input sequence generator could trigger all the generated feasible TPs (success rate = 100%). The results derived from the experiment demonstrate that the proposed approach is effective in automating testing from an EFSM