LNCS

We introduce the monitoring of trace properties under assumptions. An assumption limits the space of possible traces that the monitor may encounter. An assumption may result from knowledge about the system that is being monitored, about the environment, or about another, connected monitor. We define monitorability under assumptions and study its theoretical properties. In particular, we show that for every assumption A, the boolean combinations of properties that are safe or co-safe relative to A are monitorable under A. We give several examples and constructions on how an assumption can make a non-monitorable property monitorable, and how an assumption can make a monitorable property monitorable with fewer resources, such as integer registers

Safety and Liveness of Quantitative Automata

The safety-liveness dichotomy is a fundamental concept in formal languages which plays a key role in verification. Recently, this dichotomy has been lifted to quantitative properties, which are arbitrary functions from infinite words to partially-ordered domains. We look into harnessing the dichotomy for the specific classes of quantitative properties expressed by quantitative automata. These automata contain finitely many states and rational-valued transition weights, and their common value functions Inf, Sup, LimInf, LimSup, LimInfAvg, LimSupAvg, and DSum map infinite words into the totally-ordered domain of real numbers. In this automata-theoretic setting, we establish a connection between quantitative safety and topological continuity and provide an alternative characterization of quantitative safety and liveness in terms of their boolean counterparts. For all common value functions, we show how the safety closure of a quantitative automaton can be constructed in PTime, and we provide PSpace-complete checks of whether a given quantitative automaton is safe or live, with the exception of LimInfAvg and LimSupAvg automata, for which the safety check is in ExpSpace. Moreover, for deterministic Sup, LimInf, and LimSup automata, we give PTime decompositions into safe and live automata. These decompositions enable the separation of techniques for safety and liveness verification for quantitative specifications

Four-valued monitorability of ω-regular languages

The use of runtime verification has led to interest in deciding whether a property is monitorable: whether it is always possible for the satisfaction or violation of the property to be determined after a finite future continuation during system execution. However, classical two-valued monitorability suffers from two inherent limitations, which eventually increase runtime overhead. First, no information is available regarding whether only one verdict (satisfaction or violation) can be detected. Second, it does not tell us whether verdicts can be detected starting from the current monitor state during system execution. This paper proposes a new notion of four-valued monitorability for ω -languages and applies it at the state-level. Four-valued monitorability is more informative than two-valued monitorability as a property can be evaluated as a four-valued result, denoting that only satisfaction, only violation, or both are active for a monitorable property. We can also compute state-level weak monitorability, i.e., whether satisfaction or violation can be detected starting from a given state in a monitor, which enables state-level optimizations of monitoring algorithms. Based on a new six-valued semantics, we propose procedures for computing four-valued monitorability of ω -regular languages, both at the language-level and at the state-level. Experimental results show that our tool implementation Monic can correctly, and quickly, report both two-valued and four-valued monitorability

The Best a Monitor Can Do

Existing notions of monitorability for branching-time properties are fairly restrictive. This, in turn, impacts the ability to incorporate prior knowledge about the system under scrutiny - which corresponds to a branching-time property - into the runtime analysis. We propose a definition of optimal monitors that verify the best monitorable under- or over-approximation of a specification, regardless of its monitorability status. Optimal monitors can be obtained for arbitrary branching-time properties by synthesising a sound and complete monitor for their strongest monitorable consequence. We show that the strongest monitorable consequence of specifications expressed in Hennessy-Milner logic with recursion is itself expressible in this logic, and present a procedure to find it. Our procedure enables prior knowledge to be optimally incorporated into runtime monitors

Ain't No Stopping Us Monitoring Now

Not all properties are monitorable. This is a well-known fact, and it means there exist properties that cannot be fully verified at runtime. However, given a non-monitorable property, a monitor can still be synthesised, but it could end up in a state where no verdict will ever be concluded on the satisfaction (resp., violation) of the property. For this reason, non-monitorable properties are usually discarded. In this paper, we carry out an in-depth analysis on monitorability, and how non-monitorable properties can still be partially verified. We present our theoretical results at a semantic level, without focusing on a specific formalism. Then, we show how our theory can be applied to achieve partial runtime verification of Linear Temporal Logic (LTL)

Safety and Liveness of Quantitative Automata

The safety-liveness dichotomy is a fundamental concept in formal languages which plays a key role in verification. Recently, this dichotomy has been lifted to quantitative properties, which are arbitrary functions from infinite words to partially-ordered domains. We look into harnessing the dichotomy for the specific classes of quantitative properties expressed by quantitative automata. These automata contain finitely many states and rational-valued transition weights, and their common value functions Inf, Sup, LimInf, LimSup, LimInfAvg, LimSupAvg, and DSum map infinite words into the totally-ordered domain of real numbers. In this automata-theoretic setting, we establish a connection between quantitative safety and topological continuity and provide an alternative characterization of quantitative safety and liveness in terms of their boolean counterparts. For all common value functions, we show how the safety closure of a quantitative automaton can be constructed in PTime, and we provide PSpace-complete checks of whether a given quantitative automaton is safe or live, with the exception of LimInfAvg and LimSupAvg automata, for which the safety check is in ExpSpace. Moreover, for deterministic Sup, LimInf, and LimSup automata, we give PTime decompositions into safe and live automata. These decompositions enable the separation of techniques for safety and liveness verification for quantitative specifications.Comment: Full version of the paper to appear in CONCUR 202

Runtime Monitoring for Uncertain Times

In Runtime Verification (RV), monitors check programs for correct operation at execution time. Also called Runtime Monitoring, RV offers advantages over other approaches to program verification. Efficient monitoring is possible for programs where static checking is cost-prohibitive. Runtime monitors may test for execution faults like hardware failure, as well as logical faults. Unlike simple log checking, monitors are typically constructed using formal languages and methods that precisely define expectations and guarantees. Despite the advantages of RV, however, adoption remains low. Applying Runtime Monitoring techniques to real systems requires addressing practical concerns that have garnered little attention from researchers. System operators need monitors that provide immediate diagnostic information before and after failures, that are simple to operate over distributed systems, and that remain reliable when communication is not. These challenges are solvable, and solving them is a necessary step towards widespread RV deployment. This thesis provides solutions to these and other barriers to practical Runtime Monitoring. We address the need for reporting diagnostic information from monitored programs with nfer, a language and system for event stream abstraction. Nfer supports the automatic extraction of the structure of real-time software and includes integrations with popular programming languages. We also provide for the operation of nfer and other monitoring tools over distributed systems with Palisade, a framework built for low-latency detection of embedded system anomalies. Finally, we supply a method to ensure program properties may be monitored despite unreliable communication channels. We classify monitorable properties over general unreliable conditions and define an algorithm for when more specific conditions are known

A Retrospective Look at the Monitoring and Checking (MaC) Framework

The Monitoring and Checking (MaC) project gave rise to a framework for runtime monitoring with respect to formally specified properties, which later came to be known as runtime verification. The project also built a pioneering runtime verification tool, Java-MaC, that was an instantiation of the approach to check properties of Java programs. In this retrospective, we discuss decisions made in the design of the framework and summarize lessons learned in the course of the project

Runtime verification on data-carrying traces

Malfunctioning software systems can cause severe loss of money, sensitive data, or even human life. The ambition is therefore to verify these systems not only statically, but also monitor their behaviour at runtime. For the latter case, the temporal logic LTL---a de facto standard specification formalism in runtime verification---is widely used and well-understood. However, propositional variables are usually not a natural nor sufficient model to represent the behaviour of complex, interactive systems that can process arbitrary input values. Consequently, there is a demand for more expressive formalisms that are defined what we call traces with data, i.e., traces that contain propositions enriched with values from a (possibly) infinite domain. This thesis studies the runtime monitoring with data for a natural extension of LTL that includes first-order quantification, called LTLFO. The logic's quantifiers range over values that appear in a trace. Under assumptions laid out of what should arguably be considered a ``proper'' runtime monitor, this thesis first identifies and analyses the underlying decision problems of monitoring properties in LTL and LTLFO. Moreover, it proposes a monitoring procedure for the latter. A result is that LTLFO is undecidable, and the prefix problem too, which an online monitor has to preferably solve to coincide with monotonicity. Hence, the obtained monitor cannot be complete for LTLFO; however, this thesis proves the soundness of its construction and gives experimental results from an implementation, in order to justify its usefulness and efficiency in practice. The monitor is based on a new type of automaton, called spawning automaton; it helps to efficiently decide what parts of a possibly infinite state space need to be memorised at runtime. Furthermore, the problem occurs that not every property can be monitored trace-length independently, which is possible in LTL. For that reason, a hierarchy of effectively monitorable properties is proposed. It distinguishes properties for which a monitor requires only constant memory from ones for which a monitor inevitably has to grow ad infinitum, independently of how the future of a trace evolves. Last but not least, a proof of concept validates the monitoring means developed in this thesis on a widely established system with intensive data use: Malicious behaviour is checked on Android devices based on the most comprehensive malware set presently available. The overall detection and false positive rates are 93.9% and 28%, respectively. As a means of conducting the experiments and as a contribution in itself, an application-agnostic logging-layer for the Android system has been developed and its technical insights are explained. It aims at leveraging runtime verification techniques on Android, like other domain-specific instrumentation approaches did, such as AspectJ for Java