Specification-Based Testing with Linear Temporal Logic

This paper considers the specification-based testing in which the requirement is given in the linear temporal logic (LTL). The required LTL property must hold on all the executions of the system, which are often infinite in size and/or in length. The central piece of our framework is a property-coverage metric. Based on requirement mutation, the metric measures how well a property has been tested by a test suite. We define a coverage criterion based on the metric that selects a finite set of tests from all the possible executions of the system. We also discuss the technique of generating a test suite for specification testing by using the counterexample mechanism of a model checker. By exploiting the special structure of a generated test, we are able to reduce a test with infinite length to an equivalent one of finite length. Our framework provides a model-checking-assisted approach that generates a test suite that is finite in size and in length for testing linear temporal properties on an implementation

Quickstrom: property-based acceptance testing with LTL specifications

We present Quickstrom, a property-based testing system for acceptance testing of interactive applications. Using Quickstrom, programmers can specify the behaviour of web applications as properties in our testing-oriented dialect of Linear Temporal Logic (LTL) called QuickLTL, and then automatically test their application against the given specification with hundreds of automatically generated interactions. QuickLTL extends existing finite variants of LTL for the testing use-case, determining likely outcomes from partial traces whose minimum length is itself determined by the LTL formula. This temporal logic is embedded in our specification language, Specstrom, which is designed to be approachable to web programmers, expressive for writing specifications, and easy to analyse. Because Quickstrom tests only user-facing behaviour, it is agnostic to the implementation language of the system under test. We therefore formally specify and test many implementations of the popular TodoMVC benchmark, used for evaluation and comparison across various web frontend frameworks and languages. Our tests uncovered bugs in almost half of the available implementations.Comment: 13 pages, this is a technical report of a paper to appear at Programming Languages Design and Implementation (PLDI 2022

Approximate probabilistic verification of hybrid systems

Hybrid systems whose mode dynamics are governed by non-linear ordinary differential equations (ODEs) are often a natural model for biological processes. However such models are difficult to analyze. To address this, we develop a probabilistic analysis method by approximating the mode transitions as stochastic events. We assume that the probability of making a mode transition is proportional to the measure of the set of pairs of time points and value states at which the mode transition is enabled. To ensure a sound mathematical basis, we impose a natural continuity property on the non-linear ODEs. We also assume that the states of the system are observed at discrete time points but that the mode transitions may take place at any time between two successive discrete time points. This leads to a discrete time Markov chain as a probabilistic approximation of the hybrid system. We then show that for BLTL (bounded linear time temporal logic) specifications the hybrid system meets a specification iff its Markov chain approximation meets the same specification with probability $1$. Based on this, we formulate a sequential hypothesis testing procedure for verifying -approximately- that the Markov chain meets a BLTL specification with high probability. Our case studies on cardiac cell dynamics and the circadian rhythm indicate that our scheme can be applied in a number of realistic settings

Verification of temporal properties of infinite state systems

No es ningún secreto que tanto los sistemas software como hardware generalmente presentan errores. Los métodos de testeo y simulación pueden identificar muchos problemas importantes, pero para sistemas que tienen requerimientos de seguridad o que son económicamente críticos, es indispensable llevar a cabo una verificación exhaustiva. Tal análisis se puede realizar utilizando métodos de verificación formal. Un enfoque de la verificación formal es la verificación de modelos, que es un proceso totalmente automático basado en la construcción de modelos abstractos para representar sistemas. Poste- riormente, sobre estos modelos se comprueban propiedades deseadas del sistema, normalmente expresadas en alguna lógica temporal, como por ejemplo lógica linear temporal. Las propiedades expresadas con fórmulas de lógica linear temporal pueden describir el orden de los eventos en el tiempo sin describir el tiempo explícitamente. Por eso mismo, son útiles a la hora de verificar las posibles ejecuciones de un sistema. Este proyecto pretende implementar algoritmos de verificación de modelos que determinen si una fórmula de lógica linear temporal que exprese una propiedad de un cierto sistema es satisfecha por éste.It is no secret that computer software programs, computer hardware designs, and computer sys- tems in general exhibit errors. Testing and simulation methods can identify many significant problems, but for systems that have safety or economically critical requirements, exhaustive ver- ification is indispensable. Such exhaustive analysis can be performed with the use of formal verification methods. One approach to formal verification is model checking, which is a fully automated process based on the construction of abstract models to represent systems. These models are then checked against desired properties defining a specification, usually expressed in some temporal logic, such as linear temporal logic (LTL). Temporal properties can describe the ordering of events in time without introducing time explicitly, thereby being useful when verifying the possible executions of a system. This project aims to implement model checking algorithms that determine whether an LTL formula expressing a desired property is satisfied in a computing system

On-Line Monitoring for Temporal Logic Robustness

In this paper, we provide a Dynamic Programming algorithm for on-line monitoring of the state robustness of Metric Temporal Logic specifications with past time operators. We compute the robustness of MTL with unbounded past and bounded future temporal operators MTL over sampled traces of Cyber-Physical Systems. We implemented our tool in Matlab as a Simulink block that can be used in any Simulink model. We experimentally demonstrate that the overhead of the MTL robustness monitoring is acceptable for certain classes of practical specifications

A System for Deduction-based Formal Verification of Workflow-oriented Software Models

The work concerns formal verification of workflow-oriented software models using deductive approach. The formal correctness of a model's behaviour is considered. Manually building logical specifications, which are considered as a set of temporal logic formulas, seems to be the significant obstacle for an inexperienced user when applying the deductive approach. A system, and its architecture, for the deduction-based verification of workflow-oriented models is proposed. The process of inference is based on the semantic tableaux method which has some advantages when compared to traditional deduction strategies. The algorithm for an automatic generation of logical specifications is proposed. The generation procedure is based on the predefined workflow patterns for BPMN, which is a standard and dominant notation for the modeling of business processes. The main idea for the approach is to consider patterns, defined in terms of temporal logic,as a kind of (logical) primitives which enable the transformation of models to temporal logic formulas constituting a logical specification. Automation of the generation process is crucial for bridging the gap between intuitiveness of the deductive reasoning and the difficulty of its practical application in the case when logical specifications are built manually. This approach has gone some way towards supporting, hopefully enhancing our understanding of, the deduction-based formal verification of workflow-oriented models.Comment: International Journal of Applied Mathematics and Computer Scienc