Actual Test Coverage for Embedded Systems

Testing embedded systems is inherently incomplete; no test suite will ever be able to test all possible usage scenarios. Therefore, in the past decades many coverage measures have been developed. These measures denote the portion of a system that is tested, that way providing a quality criterion for test suites. Formulating coverage criteria is not an easy task. The measures provided in the literature are consequently almost all very trivial and syntax-dependent. Well-known examples are statement and path coverage in white-box testing, and state and transition coverage in black-box testing. The complexity of designing coverage measures for embedded systems is contained in the highly dynamic behaviour of such systems, which is state-dependent and subject to many interleavings. In this talk we introduce a framework on actual test coverage. This measure denotes the number of faults actually shown present or absent. Our framework contains a method to evaluate the actual coverage of a given set of test suite executions after testing has taken place, providing a means to express the quality of a testing process. It also contains a method to predict the actual coverage a certain number of executions will yield, providing a means to select the best test suite. Both the evaluation afterwards and the prediction in advance are quite efficient, making it feasible to implement the theory in a tool and use it in a practical context

Model-based Testing

This paper provides a comprehensive introduction to a framework for formal testing using labelled transition systems, based on an extension and reformulation of the ioco theory introduced by Tretmans. We introduce the underlying models needed to specify the requirements, and formalise the notion of test cases. We discuss conformance, and in particular the conformance relation ioco. For this relation we prove several interesting properties, and we provide algorithms to derive test cases (either in batches, or on the fly)

Confluence Reduction for Probabilistic Systems (extended version)

This paper presents a novel technique for state space reduction of probabilistic specifications, based on a newly developed notion of confluence for probabilistic automata. We prove that this reduction preserves branching probabilistic bisimulation and can be applied on-the-fly. To support the technique, we introduce a method for detecting confluent transitions in the context of a probabilistic process algebra with data, facilitated by an earlier defined linear format. A case study demonstrates that significant reductions can be obtained

Lesson study - deel 3. Ervaringen bij de introductie van periodieke bewegingen

Het eerste artikel in deze serie ging over 'lesson study' als methode om het vak van docent te leren of blijvend te verbeteren. Het tweede artikel beschreef de ervaringen met bewijzen in de meetkunde (zie de paragraaf Info). Dit derde artikel gaat in op lesontwerpen die gaan over de overgang van een meetkundige benadering van de (co)sinus naar de analytische benadering in de vorm van een periodieke beweging. Een onderwerp dat in 4-vwo aan de orde zou komen, en dat de docenten na aan het hart lag

A comparison of confluence and ample sets in probabilistic and non-probabilistic branching time

Confluence reduction and partial order reduction by means of ample sets are two different techniques for state space reduction in both traditional and probabilistic model checking. This paper provides an extensive comparison between these two methods, and answers the question how they relate in terms of reduction power when preserving branching time properties. We prove that, while both preserve the same properties, confluence reduction is strictly more powerful than partial order reduction: every reduction that can be obtained with partial order reduction can also be obtained with confluence reduction, but the converse is not true. The main challenge for the comparison is that confluence reduction was defined in an action-based setting, whereas ample set reduction is often defined in a state-based setting. We therefore redefine confluence reduction in the state-based setting of Markov decision processes, and provide a nontrivial proof of its correctness. Additionally, we pinpoint precisely in what way confluence reduction is more general, and provide conditions under which the two notions coincide. The results we present also hold for non-probabilistic models, as they can just as well be applied in a context where all transitions are non-probabilistic. To discuss the practical applicability of our results, we adapt a state space generation technique based on representative states, already known in combination with confluence reduction, so that it can also be applied to ample sets

Efficient Modelling and Generation of Markov Automata (extended version)

This paper introduces a framework for the efficient modelling and generation of Markov automata. It consists of (1) the data-rich process-algebraic language MAPA, allowing concise modelling of systems with nondeterminism, probability and Markovian timing; (2) a restricted form of the language, the MLPPE, enabling easy state space generation and parallel composition; and (3) several syntactic reduction techniques on the MLPPE format, for generating equivalent but smaller models. Technically, the framework relies on an encoding of MAPA into the existing prCRL language for probabilistic automata. First, we identify a class of transformations on prCRL that can be lifted to the Markovian realm using our encoding. Then, we employ this result to reuse prCRL's linearisation procedure to transform any MAPA specification to an equivalent MLPPE, and to lift three prCRL reduction techniques to MAPA. Additionally, we define two novel reduction techniques for MLPPEs. All our techniques treat data as well as Markovian and interactive behaviour in a fully symbolic manner, working on specifications instead of models and thus reducing state spaces prior to their construction. The framework has been implemented in our tool SCOOP, and a case study on polling systems and mutual exclusion protocols shows its practical applicability

Analysis of Timed and Long-Run Objectives for Markov Automata

Markov automata (MAs) extend labelled transition systems with random delays and probabilistic branching. Action-labelled transitions are instantaneous and yield a distribution over states, whereas timed transitions impose a random delay governed by an exponential distribution. MAs are thus a nondeterministic variation of continuous-time Markov chains. MAs are compositional and are used to provide a semantics for engineering frameworks such as (dynamic) fault trees, (generalised) stochastic Petri nets, and the Architecture Analysis & Design Language (AADL). This paper considers the quantitative analysis of MAs. We consider three objectives: expected time, long-run average, and timed (interval) reachability. Expected time objectives focus on determining the minimal (or maximal) expected time to reach a set of states. Long-run objectives determine the fraction of time to be in a set of states when considering an infinite time horizon. Timed reachability objectives are about computing the probability to reach a set of states within a given time interval. This paper presents the foundations and details of the algorithms and their correctness proofs. We report on several case studies conducted using a prototypical tool implementation of the algorithms, driven by the MAPA modelling language for efficiently generating MAs.Comment: arXiv admin note: substantial text overlap with arXiv:1305.705