199,646 research outputs found
Automated test generation from formal specifications of real-time reactive systems
Real-time reactive systems are among the most difficult systems to design and implement because of their size and complex functional and timing requirements. They are often used in safety-critical contexts. Consequently, the correction of such systems must be assured before they are deployed. This thesis addresses the quality assurance of real-time reactive systems through rigorous testing methods. The thesis gives methods to generate test cases from the formal specifications of real-time reactive systems developed in TROMLAB framework. The scope of this thesis encompasses two major components. A Homomorphism Theorem is given that serves as a basis for automated test case generation from the grid automaton associated with the extended state machine formalism. A number of algorithms are also given for generating test cases for black-box testing of reactive class implementations, implementations of class refinements and system configurations. The testing methodologies are theoretically sound, yet being practical for automated test case generation
Recommended from our members
Automatic generation of test sequences form EFSM models using evolutionary algorithms
Automated test data generation through evolutionary testing (ET) is a topic of interest to the software engineering community. While there are many ET-based techniques for automatically generating test data from code, the problem of generating test data from an extended finite state machine (EFSMs) is more complex and has received little attention. In this paper, we introduce a novel approach that addresses the problem of generating input test sequences that trigger given feasible paths in an EFSM model by employing an ET-based technique. The proposed approach expresses the problem as a search for input parameters to be applied to a set of functions to be called sequentially. In order to apply ET-based technique, a new fitness function is introduced to cope with the case when a test target involves calls to a set of transitions sequentially. We evaluate our approach empirically using five sets of randomly generated paths through two EFSM case studies: INRES and class 2 transport protocols. In the experiments, we apply two search techniques: a random and an ET-based which utilizes our new fitness function. Experimental results show that the proposed approach produces input test sequences that trigger all the feasible paths used with a success rate of 100%, however, the random technique failed in most cases with a success rate of 20.8%
JWalk: a tool for lazy, systematic testing of java classes by design introspection and user interaction
Popular software testing tools, such as JUnit, allow frequent retesting of modified code; yet the manually created test scripts are often seriously incomplete. A unit-testing tool called JWalk has therefore been developed to address the need for systematic unit testing within the context of agile methods. The tool operates directly on the compiled code for Java classes and uses a new lazy method for inducing the changing design of a class on the fly. This is achieved partly through introspection, using Java’s reflection capability, and partly through interaction with the user, constructing and saving test oracles on the fly. Predictive rules reduce the number of oracle values that must be confirmed by the tester. Without human intervention, JWalk performs bounded exhaustive exploration of the class’s method protocols and may be directed to explore the space of algebraic constructions, or the intended design state-space of the tested class. With some human interaction, JWalk performs up to the equivalent of fully automated state-based testing, from a specification that was acquired incrementally
ChimpCheck: Property-Based Randomized Test Generation for Interactive Apps
We consider the problem of generating relevant execution traces to test rich
interactive applications. Rich interactive applications, such as apps on mobile
platforms, are complex stateful and often distributed systems where
sufficiently exercising the app with user-interaction (UI) event sequences to
expose defects is both hard and time-consuming. In particular, there is a
fundamental tension between brute-force random UI exercising tools, which are
fully-automated but offer low relevance, and UI test scripts, which are manual
but offer high relevance. In this paper, we consider a middle way---enabling a
seamless fusion of scripted and randomized UI testing. This fusion is
prototyped in a testing tool called ChimpCheck for programming, generating, and
executing property-based randomized test cases for Android apps. Our approach
realizes this fusion by offering a high-level, embedded domain-specific
language for defining custom generators of simulated user-interaction event
sequences. What follows is a combinator library built on industrial strength
frameworks for property-based testing (ScalaCheck) and Android testing (Android
JUnit and Espresso) to implement property-based randomized testing for Android
development. Driven by real, reported issues in open source Android apps, we
show, through case studies, how ChimpCheck enables expressing effective testing
patterns in a compact manner.Comment: 20 pages, 21 figures, Symposium on New ideas, New Paradigms, and
Reflections on Programming and Software (Onward!2017
Automatic Test Generation for Space
The European Space Agency (ESA) uses an engine to perform tests in the Ground
Segment infrastructure, specially the Operational Simulator. This engine uses
many different tools to ensure the development of regression testing
infrastructure and these tests perform black-box testing to the C++ simulator
implementation. VST (VisionSpace Technologies) is one of the companies that
provides these services to ESA and they need a tool to infer automatically
tests from the existing C++ code, instead of writing manually scripts to
perform tests. With this motivation in mind, this paper explores automatic
testing approaches and tools in order to propose a system that satisfies VST
needs
Stateful Testing: Finding More Errors in Code and Contracts
Automated random testing has shown to be an effective approach to finding
faults but still faces a major unsolved issue: how to generate test inputs
diverse enough to find many faults and find them quickly. Stateful testing, the
automated testing technique introduced in this article, generates new test
cases that improve an existing test suite. The generated test cases are
designed to violate the dynamically inferred contracts (invariants)
characterizing the existing test suite. As a consequence, they are in a good
position to detect new errors, and also to improve the accuracy of the inferred
contracts by discovering those that are unsound. Experiments on 13 data
structure classes totalling over 28,000 lines of code demonstrate the
effectiveness of stateful testing in improving over the results of long
sessions of random testing: stateful testing found 68.4% new errors and
improved the accuracy of automatically inferred contracts to over 99%, with
just a 7% time overhead.Comment: 11 pages, 3 figure
Recommended from our members
Using formal methods to support testing
Formal methods and testing are two important approaches that assist in the development of high quality software. While traditionally these approaches have been seen as rivals, in recent
years a new consensus has developed in which they are seen as complementary. This article reviews the state of the art regarding ways in which the presence of a formal specification can be used to assist testing
Automated Fixing of Programs with Contracts
This paper describes AutoFix, an automatic debugging technique that can fix
faults in general-purpose software. To provide high-quality fix suggestions and
to enable automation of the whole debugging process, AutoFix relies on the
presence of simple specification elements in the form of contracts (such as
pre- and postconditions). Using contracts enhances the precision of dynamic
analysis techniques for fault detection and localization, and for validating
fixes. The only required user input to the AutoFix supporting tool is then a
faulty program annotated with contracts; the tool produces a collection of
validated fixes for the fault ranked according to an estimate of their
suitability.
In an extensive experimental evaluation, we applied AutoFix to over 200
faults in four code bases of different maturity and quality (of implementation
and of contracts). AutoFix successfully fixed 42% of the faults, producing, in
the majority of cases, corrections of quality comparable to those competent
programmers would write; the used computational resources were modest, with an
average time per fix below 20 minutes on commodity hardware. These figures
compare favorably to the state of the art in automated program fixing, and
demonstrate that the AutoFix approach is successfully applicable to reduce the
debugging burden in real-world scenarios.Comment: Minor changes after proofreadin
- …