69,398 research outputs found
Badger: Complexity Analysis with Fuzzing and Symbolic Execution
Hybrid testing approaches that involve fuzz testing and symbolic execution
have shown promising results in achieving high code coverage, uncovering subtle
errors and vulnerabilities in a variety of software applications. In this paper
we describe Badger - a new hybrid approach for complexity analysis, with the
goal of discovering vulnerabilities which occur when the worst-case time or
space complexity of an application is significantly higher than the average
case. Badger uses fuzz testing to generate a diverse set of inputs that aim to
increase not only coverage but also a resource-related cost associated with
each path. Since fuzzing may fail to execute deep program paths due to its
limited knowledge about the conditions that influence these paths, we
complement the analysis with a symbolic execution, which is also customized to
search for paths that increase the resource-related cost. Symbolic execution is
particularly good at generating inputs that satisfy various program conditions
but by itself suffers from path explosion. Therefore, Badger uses fuzzing and
symbolic execution in tandem, to leverage their benefits and overcome their
weaknesses. We implemented our approach for the analysis of Java programs,
based on Kelinci and Symbolic PathFinder. We evaluated Badger on Java
applications, showing that our approach is significantly faster in generating
worst-case executions compared to fuzzing or symbolic execution on their own
A Symbolic Execution Algorithm for Constraint-Based Testing of Database Programs
In so-called constraint-based testing, symbolic execution is a common
technique used as a part of the process to generate test data for imperative
programs. Databases are ubiquitous in software and testing of programs
manipulating databases is thus essential to enhance the reliability of
software. This work proposes and evaluates experimentally a symbolic ex-
ecution algorithm for constraint-based testing of database programs. First, we
describe SimpleDB, a formal language which offers a minimal and well-defined
syntax and seman- tics, to model common interaction scenarios between pro-
grams and databases. Secondly, we detail the proposed al- gorithm for symbolic
execution of SimpleDB models. This algorithm considers a SimpleDB program as a
sequence of operations over a set of relational variables, modeling both the
database tables and the program variables. By inte- grating this relational
model of the program with classical static symbolic execution, the algorithm
can generate a set of path constraints for any finite path to test in the
control- flow graph of the program. Solutions of these constraints are test
inputs for the program, including an initial content for the database. When the
program is executed with respect to these inputs, it is guaranteed to follow
the path with re- spect to which the constraints were generated. Finally, the
algorithm is evaluated experimentally using representative SimpleDB models.Comment: 12 pages - preliminary wor
Shadow symbolic execution for better testing of evolving software
In this idea paper, we propose a novel way for improving the testing of program changes via symbolic execution. At a high-level, our technique runs two different program versions in the same symbolic execution instance, with the old version effectively shadowing the new one. In this way, the technique can exploit precise dynamic value information to effectively drive execution toward the behaviour that has changed from one version to the next. We discuss the main challenges and opportunities of this approach in terms of pruning and prioritising path exploration, mapping elements across versions, and sharing common symbolic state between versions. Copyright © 2014 ACM
Recommended from our members
Symbolic execution and the testing of COBOL programs
The thesis is in two parts. Part one is a review of existing work in the area of software testing and more specifically symbolic execution. Part two is a description of the symbolic execution testing system for COBOL (SYM-BOL). Much of the work presented has been published or accepted for publication.
Part one commences by introducing the aims of software testing and is followed by a review of the tools and techniques of software testing that have been developed over the past 25 years. A simple taxonomy of software testing techniques is given. One potentially powerful technique is symbolic execution. The principles of symbolic execution are described followed by the problems in applying symbolic execution. Part one is completed by a review of existing symbolic execution testing systems. No symbolic execution testing system has previously been built for a commercial data processing language such as COBOL. Part two commences by outlining the features of the SYM-BOL system and describes the user strategies that may be employed when using the system.
The system generates an intermediate form in stages by transforming the source program into one that contains only a limited number of language constructs. Path selection can be automatic or undertaken by the user. In both cases the results of the symbolic execution already undertaken are available to the path selector to help reduce the likelihood of selecting an infeasible path. A description of how the Nag-library linear optimizer E04MBF is used for feasibility checking is given. Feasible solutions are turned into files of test cases. Simple assertions may be included in the source program which do not affect the normal execution of the software but which can be verified by inclusion in the symbolic execution
A Survey of New Trends in Symbolic Execution for Software Testing and Analysis
Symbolic execution is a well-known program analysis technique which represents values of program inputs with symbolic values instead of concrete (initialized) data and executes the program by manipulating program expressions involving the symbolic values. Symbolic execution has been proposed over three decades ago but recently it has found renewed interest in the research community, due in part to the progress in decision procedures, availability of powerful computers and new algorithmic developments. We provide a survey of some of the new research trends in symbolic execution, with particular emphasis on applications to test generation and program analysis. We first describe an approach that handles complex programming constructs such as input data structures, arrays, as well as multi-threading. We follow with a discussion of abstraction techniques that can be used to limit the (possibly infinite) number of symbolic configurations that need to be analyzed for the symbolic execution of looping programs. Furthermore, we describe recent hybrid techniques that combine concrete and symbolic execution to overcome some of the inherent limitations of symbolic execution, such as handling native code or availability of decision procedures for the application domain. Finally, we give a short survey of interesting new applications, such as predictive testing, invariant inference, program repair, analysis of parallel numerical programs and differential symbolic execution
ConSUS: A light-weight program conditioner
Program conditioning consists of identifying and removing a set of statements which cannot be executed when a condition of interest holds at some point in a program. It has been applied to problems in maintenance, testing, re-use and re-engineering. All current approaches to program conditioning rely upon both symbolic execution and reasoning about symbolic predicates. The reasoning can be performed by a ‘heavy duty’ theorem prover but this may impose unrealistic performance constraints.
This paper reports on a lightweight approach to theorem proving using the FermaT Simplify decision procedure. This is used as a component to ConSUS, a program conditioning system for the Wide Spectrum Language WSL. The paper describes the symbolic execution algorithm used by ConSUS, which prunes as it conditions.
The paper also provides empirical evidence that conditioning produces a significant reduction in program size and, although exponential in the worst case, the conditioning system has low degree polynomial behaviour in many cases, thereby making it scalable to unit level applications of program conditioning
Learning to Accelerate Symbolic Execution via Code Transformation
Symbolic execution is an effective but expensive technique for automated test generation. Over the years, a large number of refined symbolic execution techniques have been proposed to improve its efficiency. However, the symbolic execution efficiency problem remains, and largely limits the application of symbolic execution in practice. Orthogonal to refined symbolic execution, in this paper we propose to accelerate symbolic execution through semantic-preserving code transformation on the target programs. During the initial stage of this direction, we adopt a particular code transformation, compiler optimization, which is initially proposed to accelerate program concrete execution by transforming the source program into another semantic-preserving target program with increased efficiency (e.g., faster or smaller). However, compiler optimizations are mostly designed to accelerate program concrete execution rather than symbolic execution. Recent work also reported that unified settings on compiler optimizations that can accelerate symbolic execution for any program do not exist at all. Therefore, in this work we propose a machine-learning based approach to tuning compiler optimizations to accelerate symbolic execution, whose results may also aid further design of specific code transformations for symbolic execution. In particular, the proposed approach LEO separates source-code functions and libraries through our program-splitter, and predicts individual compiler optimization (i.e., whether a type of code transformation is chosen) separately through analyzing the performance of existing symbolic execution. Finally, LEO applies symbolic execution on the code transformed by compiler optimization (through our local-optimizer). We conduct an empirical study on GNU Coreutils programs using the KLEE symbolic execution engine. The results show that LEO significantly accelerates symbolic execution, outperforming the default KLEE configurations (i.e., turning on/off all compiler optimizations) in various settings, e.g., with the default training/testing time, LEO achieves the highest line coverage in 50/68 programs, and its average improvement rate on all programs is 46.48%/88.92% in terms of line coverage compared with turning on/off all compiler optimizations
Benchmarking Symbolic Execution Using Constraint Problems -- Initial Results
Symbolic execution is a powerful technique for bug finding and program
testing. It is successful in finding bugs in real-world code. The core
reasoning techniques use constraint solving, path exploration, and search,
which are also the same techniques used in solving combinatorial problems,
e.g., finite-domain constraint satisfaction problems (CSPs). We propose CSP
instances as more challenging benchmarks to evaluate the effectiveness of the
core techniques in symbolic execution. We transform CSP benchmarks into C
programs suitable for testing the reasoning capabilities of symbolic execution
tools. From a single CSP P, we transform P depending on transformation choice
into different C programs. Preliminary testing with the KLEE, Tracer-X, and
LLBMC tools show substantial runtime differences from transformation and solver
choice. Our C benchmarks are effective in showing the limitations of existing
symbolic execution tools. The motivation for this work is we believe that
benchmarks of this form can spur the development and engineering of improved
core reasoning in symbolic execution engines
Test Case Generation for Object-Oriented Imperative Languages in CLP
Testing is a vital part of the software development process. Test Case
Generation (TCG) is the process of automatically generating a collection of
test cases which are applied to a system under test. White-box TCG is usually
performed by means of symbolic execution, i.e., instead of executing the
program on normal values (e.g., numbers), the program is executed on symbolic
values representing arbitrary values. When dealing with an object-oriented (OO)
imperative language, symbolic execution becomes challenging as, among other
things, it must be able to backtrack, complex heap-allocated data structures
should be created during the TCG process and features like inheritance, virtual
invocations and exceptions have to be taken into account. Due to its inherent
symbolic execution mechanism, we pursue in this paper that Constraint Logic
Programming (CLP) has a promising unexploited application field in TCG. We will
support our claim by developing a fully CLP-based framework to TCG of an OO
imperative language, and by assessing it on a corresponding implementation on a
set of challenging Java programs. A unique characteristic of our approach is
that it handles all language features using only CLP and without the need of
developing specific constraint operators (e.g., to model the heap)
Fuzzing for CPS Mutation Testing
Mutation testing can help reduce the risks of releasing faulty software. For
such reason, it is a desired practice for the development of embedded software
running in safety-critical cyber-physical systems (CPS). Unfortunately,
state-of-the-art test data generation techniques for mutation testing of C and
C++ software, two typical languages for CPS software, rely on symbolic
execution, whose limitations often prevent its application (e.g., it cannot
test black-box components).
We propose a mutation testing approach that leverages fuzz testing, which has
proved effective with C and C++ software. Fuzz testing automatically generates
diverse test inputs that exercise program branches in a varied number of ways
and, therefore, exercise statements in different program states, thus
maximizing the likelihood of killing mutants, our objective.
We performed an empirical assessment of our approach with software components
used in satellite systems currently in orbit. Our empirical evaluation shows
that mutation testing based on fuzz testing kills a significantly higher
proportion of live mutants than symbolic execution (i.e., up to an additional
47 percentage points). Further, when symbolic execution cannot be applied, fuzz
testing provides significant benefits (i.e., up to 41% mutants killed). Our
study is the first one comparing fuzz testing and symbolic execution for
mutation testing; our results provide guidance towards the development of fuzz
testing tools dedicated to mutation testing.Comment: This article is the camera-ready version for ASE 202
- …