Test of preemptive real-time systems

Time Petri nets with stopwatches not only model system/environment interactions and time constraints. They further enable modeling of suspend/resume operations in real-time systems. Assuming the modelled systems are non deterministic and partially observable, the paper proposes a test generation approach which implements an online testing policy and outputs test results that are valid for the (part of the) selected environment. A relativized conformance relation named rswtioco is defined and a test generation algorithm is presented. The proposed approach is illustrated on an example

Testing real-time systems using TINA

The paper presents a technique for model-based black-box conformance testing of real-time systems using the Time Petri Net Analyzer TINA. Such test suites are derived from a prioritized time Petri net composed of two concurrent sub-nets specifying respectively the expected behaviour of the system under test and its environment.We describe how the toolbox TINA has been extended to support automatic generation of time-optimal test suites. The result is optimal in the sense that the set of test cases in the test suite have the shortest possible accumulated time to be executed. Input/output conformance serves as the notion of implementation correctness, essentially timed trace inclusion taking environment assumptions into account. Test cases selection is based either on using manually formulated test purposes or automatically from various coverage criteria specifying structural criteria of the model to be fulfilled by the test suite. We discuss how test purposes and coverage criterion are specified in the linear temporal logic SE-LTL, derive test sequences, and assign verdicts

System Testing of Timing Requirements based on Use Cases and Timed Automata

In the context of use-case centric development and requirements-driven testing, this paper addresses the problem of automatically deriving system test cases to verify timing requirements. Inspired by engineering practice in an automotive software development context, we rely on an analyzable form of use case specifications and augment such functional descriptions with timed automata, capturing timing requirements, following a methodology aiming at minimizing modeling overhead. We automate the generation of executable test cases using a test strategy based on maximizing test suite diversity and building over the UPPAAL model checker. Initial empirical results based on an industrial case study provide evidence of the effectiveness of the approach

Online On-the-Fly Testing of Real-time Systems

In this paper we present a framework, an algorithm and a new tool for online testing of real-time systems based on symbolic techniques used in UPPAAL model checker. We extend UPPAAL timed automata network model to a test specification which is used to generate test primitives and to check the correctness of system responses including the timing aspects. We use timed trace inclusion as a conformance relation between system and specification to draw a test verdict. The test generation and execution algorithm is implemented as an extension to UPPAAL and experiments carried out to examine the correctness and performance of the tool. The experiment results are promising

Online On-the-Fly Testing of Real-time Systems

In this paper we present a framework, an algorithm and a new tool for online testing of real-time systems based on symbolic techniques used in UPPAAL model checker. We extend UPPAAL timed automata network model to a test specification which is used to generate test primitives and to check the correctness of system responses including the timing aspects. We use timed trace inclusion as a conformance relation between system and specification to draw a test verdict. The test generation and execution algorithm is implemented as an extension to UPPAAL and experiments carried out to examine the correctness and performance of the tool. The experiment results are promising

Test Case Generation for Mutation-based Testing of Timeliness

AbstractTemporal correctness is crucial for real-time systems. Few methods exist to test temporal correctness and most methods used in practice are ad-hoc. A problem with testing real-time applications is the response-time dependency on the execution order of concurrent tasks. Execution order in turn depends on execution environment properties such as scheduling protocols, use of mutual exclusive resources as well as the point in time when stimuli is injected. Model based mutation testing has previously been proposed to determine the execution orders that need to be verified to increase confidence in timeliness. An effective way to automatically generate such test cases for dynamic real-time systems is still needed. This paper presents a method using heuristic-driven simulation to generate test cases

Test Case Minimization for Real-Time Systems Using Timed Bound Traces

Abstract. Real-Time systems (RTS for short) are those systems whose behavior is time dependent. Reliability and safety are of paramount importance in designing and building RTS because a failure of an RTS puts the public and/or the environment at risk. For the purpose of effective error reporting and testing, this paper considers the trace inclusion problem for RTS: given a path ρ (resp. ρ ′ ) of length n of a timed automaton A (resp. B), find whether the set of timed traces of ρ of length n are included in the set of timed traces of ρ ′ of length n such that A is known but not B. We assume that the traces of ρ ′ are only defined by a decision procedure. The proposed solution is based on the identification of a set of timed bound traces. The latter gives a finite representation of the trace space of a path. The number of these timed bounds varies between 1 and 2 × (n + 1). The trace inclusion problem is then reduced to the inclusion of timed bound traces. The paper shows also how these results can be used to reduce the number of test cases for an RTS

Active Learning of Deterministic Timed Automata with Myhill-Nerode Style Characterization

We present an algorithm to learn a deterministic timed automaton (DTA) via membership and equivalence queries. Our algorithm is an extension of the L* algorithm with a Myhill-Nerode style characterization of recognizable timed languages, which is the class of timed languages recognizable by DTAs. We first characterize the recognizable timed languages with a Nerode-style congruence. Using it, we give an algorithm with a smart teacher answering symbolic membership queries in addition to membership and equivalence queries. With a symbolic membership query, one can ask the membership of a certain set of timed words at one time. We prove that for any recognizable timed language, our learning algorithm returns a DTA recognizing it. We show how to answer a symbolic membership query with finitely many membership queries. We also show that our learning algorithm requires a polynomial number of queries with a smart teacher and an exponential number of queries with a normal teacher. We applied our algorithm to various benchmarks and confirmed its effectiveness with a normal teacher