261 research outputs found
Improving Function Coverage with Munch: A Hybrid Fuzzing and Directed Symbolic Execution Approach
Fuzzing and symbolic execution are popular techniques for finding
vulnerabilities and generating test-cases for programs. Fuzzing, a blackbox
method that mutates seed input values, is generally incapable of generating
diverse inputs that exercise all paths in the program. Due to the
path-explosion problem and dependence on SMT solvers, symbolic execution may
also not achieve high path coverage. A hybrid technique involving fuzzing and
symbolic execution may achieve better function coverage than fuzzing or
symbolic execution alone. In this paper, we present Munch, an open source
framework implementing two hybrid techniques based on fuzzing and symbolic
execution. We empirically show using nine large open-source programs that
overall, Munch achieves higher (in-depth) function coverage than symbolic
execution or fuzzing alone. Using metrics based on total analyses time and
number of queries issued to the SMT solver, we also show that Munch is more
efficient at achieving better function coverage.Comment: To appear at 33rd ACM/SIGAPP Symposium On Applied Computing (SAC). To
be held from 9th to 13th April, 201
Automatically Discovering, Reporting and Reproducing Android Application Crashes
Mobile developers face unique challenges when detecting and reporting crashes
in apps due to their prevailing GUI event-driven nature and additional sources
of inputs (e.g., sensor readings). To support developers in these tasks, we
introduce a novel, automated approach called CRASHSCOPE. This tool explores a
given Android app using systematic input generation, according to several
strategies informed by static and dynamic analyses, with the intrinsic goal of
triggering crashes. When a crash is detected, CRASHSCOPE generates an augmented
crash report containing screenshots, detailed crash reproduction steps, the
captured exception stack trace, and a fully replayable script that
automatically reproduces the crash on a target device(s). We evaluated
CRASHSCOPE's effectiveness in discovering crashes as compared to five
state-of-the-art Android input generation tools on 61 applications. The results
demonstrate that CRASHSCOPE performs about as well as current tools for
detecting crashes and provides more detailed fault information. Additionally,
in a study analyzing eight real-world Android app crashes, we found that
CRASHSCOPE's reports are easily readable and allow for reliable reproduction of
crashes by presenting more explicit information than human written reports.Comment: 12 pages, in Proceedings of 9th IEEE International Conference on
Software Testing, Verification and Validation (ICST'16), Chicago, IL, April
10-15, 2016, pp. 33-4
Coyote C++: An Industrial-Strength Fully Automated Unit Testing Tool
Coyote C++ is an automated testing tool that uses a sophisticated
concolic-execution-based approach to realize fully automated unit testing for C
and C++. While concolic testing has proven effective for languages such as C
and Java, tools have struggled to achieve a practical level of automation for
C++ due to its many syntactical intricacies and overall complexity. Coyote C++
is the first automated testing tool to breach the barrier and bring automated
unit testing for C++ to a practical level suitable for industrial adoption,
consistently reaching around 90% code coverage. Notably, this testing process
requires no user involvement and performs test harness generation, test case
generation and test execution with "one-click" automation. In this paper, we
introduce Coyote C++ by outlining its high-level structure and discussing the
core design decisions that shaped the implementation of its concolic execution
engine. Finally, we demonstrate that Coyote C++ is capable of achieving high
coverage results within a reasonable timespan by presenting the results from
experiments on both open-source and industrial software
Erlang Code Evolution Control
During the software lifecycle, a program can evolve several times for
different reasons such as the optimisation of a bottle-neck, the refactoring of
an obscure function, etc. These code changes often involve several functions or
modules, so it can be difficult to know whether the correct behaviour of the
previous releases has been preserved in the new release. Most developers rely
on a previously defined test suite to check this behaviour preservation. We
propose here an alternative approach to automatically obtain a test suite that
specifically focusses on comparing the old and new versions of the code. Our
test case generation is directed by a sophisticated combination of several
already existing tools such as TypEr, CutEr, and PropEr; and other ideas such
as allowing the programmer to chose an expression of interest that must
preserve the behaviour, or the recording of the sequences of values to which
this expression is evaluated. All the presented work has been implemented in an
open-source tool that is publicly available on GitHub.Comment: Pre-proceedings paper presented at the 27th International Symposium
on Logic-Based Program Synthesis and Transformation (LOPSTR 2017), Namur,
Belgium, 10-12 October 2017 (arXiv:1708.07854
Size-Change Termination as a Contract
Termination is an important but undecidable program property, which has led
to a large body of work on static methods for conservatively predicting or
enforcing termination. One such method is the size-change termination approach
of Lee, Jones, and Ben-Amram, which operates in two phases: (1) abstract
programs into "size-change graphs," and (2) check these graphs for the
size-change property: the existence of paths that lead to infinite decreasing
sequences.
We transpose these two phases with an operational semantics that accounts for
the run-time enforcement of the size-change property, postponing (or entirely
avoiding) program abstraction. This choice has two key consequences: (1)
size-change termination can be checked at run-time and (2) termination can be
rephrased as a safety property analyzed using existing methods for systematic
abstraction.
We formulate run-time size-change checks as contracts in the style of Findler
and Felleisen. The result compliments existing contracts that enforce partial
correctness specifications to obtain contracts for total correctness. Our
approach combines the robustness of the size-change principle for termination
with the precise information available at run-time. It has tunable overhead and
can check for nontermination without the conservativeness necessary in static
checking. To obtain a sound and computable termination analysis, we apply
existing abstract interpretation techniques directly to the operational
semantics, avoiding the need for custom abstractions for termination. The
resulting analyzer is competitive with with existing, purpose-built analyzers
An empirical investigation into branch coverage for C programs using CUTE and AUSTIN
Automated test data generation has remained a topic of considerable interest for several decades because it lies at the heart of attempts to automate the process of Software Testing. This paper reports the results of an empirical study using the dynamic symbolic-execution tool. CUTE, and a search based tool, AUSTIN on five non-trivial open source applications. The aim is to provide practitioners with an assessment of what can be achieved by existing techniques with little or no specialist knowledge and to provide researchers with baseline data against which to measure subsequent work. To achieve this, each tool is applied 'as is', with neither additional tuning nor supporting harnesses and with no adjustments applied to the subject programs under test. The mere fact that these tools can be applied 'out of the box' in this manner reflects the growing maturity of Automated test data generation. However, as might be expected, the study reveals opportunities for improvement and suggests ways to hybridize these two approaches that have hitherto been developed entirely independently. (C) 2010 Elsevier Inc. All rights reserved
Formal Methods for Constraint-Based Testing and Reversible Debugging in Erlang
Tesis por compendio[ES] Erlang es un lenguaje de programación funcional con concurrencia mediante paso de mensajes basado en el modelo de actores. Éstas y otras caracterÃsticas lo hacen especialmente adecuado para aplicaciones distribuidas en tiempo real acrÃtico. En los últimos años, la popularidad de Erlang ha aumentado debido a la demanda de servicios concurrentes.
No obstante, desarrollar sistemas Erlang libres de errores es un reto considerable. A pesar de que Erlang evita muchos problemas por diseño (por ejemplo, puntos muertos), algunos otros problemas pueden aparecer. En este contexto, las técnicas de testing y depuración basadas en métodos formales pueden ser útiles para detectar, localizar y arreglar errores de programación en Erlang.
En esta tesis proponemos varios métodos para testing y depuración en Erlang. En particular, estos métodos están basados en modelos semánticos para concolic testing, pruebas basadas en propiedades, depuración reversible con consistencia causal y repetición reversible con consistencia causal de programas Erlang. Además, probamos formalmente las principales propiedades de nuestras propuestas y diseñamos herramientas de código abierto que implementan estos métodos.[CA] Erlang és un llenguatge de programació funcional amb concurrència mitjançant pas de missatges basat en el model d'actors. Estes i altres caracterÃstiques el fan especialment adequat per a aplicacions distribuïdes en temps real acrÃtic. En els últims anys, la popularitat d'Erlang ha augmentat degut a la demanda de servicis concurrents.
No obstant, desenvolupar sistemes Erlang lliures d'errors és un repte considerable. Encara que Erlang evita molts problemes per disseny (per exemple, punts morts), alguns altres problemes poden aparéixer. En este context, les tècniques de testing y depuració basades en mètodes formals poden ser útils per a detectar, localitzar y arreglar errors de programació en Erlang.
En esta tesis proposem diversos mètodes per a testing i depuració en Erlang. En particular, estos mètodes estan basats en models semà ntics per a concolic testing, testing basat en propietats, depuració reversible amb consistència causal i repetició reversible amb consistència causal de programes Erlang. A més, provem formalment les principals propietats de les nostres propostes i dissenyem ferramentes de codi obert que implementen estos mètodes.[EN] Erlang is a message-passing concurrent, functional programming language based on the actor model. These and other features make it especially appropriate for distributed, soft real-time applications. In the recent years, Erlang's popularity has increased due to the demand for concurrent services.
However, developing error-free systems in Erlang is quite a challenge. Although Erlang avoids many problems by design (e.g., deadlocks), some other problems may appear. Here, testing and debugging techniques based on formal methods may be helpful to detect, locate and fix programming errors in Erlang.
In this thesis we propose several methods for testing and debugging in Erlang. In particular, these methods are based on semantics models for concolic testing, property-based testing, causal-consistent reversible debugging and causal-consistent replay debugging of Erlang programs. We formally prove the main properties of our proposals and design open-source tools that implement these methods.Palacios Corella, A. (2020). Formal Methods for Constraint-Based Testing and Reversible Debugging in Erlang [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/139076TESISCompendi
CTGEN - a Unit Test Generator for C
We present a new unit test generator for C code, CTGEN. It generates test
data for C1 structural coverage and functional coverage based on
pre-/post-condition specifications or internal assertions. The generator
supports automated stub generation, and data to be returned by the stub to the
unit under test (UUT) may be specified by means of constraints. The typical
application field for CTGEN is embedded systems testing; therefore the tool can
cope with the typical aliasing problems present in low-level C, including
pointer arithmetics, structures and unions. CTGEN creates complete test
procedures which are ready to be compiled and run against the UUT. In this
paper we describe the main features of CTGEN, their technical realisation, and
we elaborate on its performance in comparison to a list of competing test
generation tools. Since 2011, CTGEN is used in industrial scale test campaigns
for embedded systems code in the automotive domain.Comment: In Proceedings SSV 2012, arXiv:1211.587
- …