Logic and model checking for hidden Markov models

The branching-time temporal logic PCTL* has been introduced to specify quantitative properties over probability systems, such as discrete-time Markov chains. Until now, however, no logics have been defined to specify properties over hidden Markov models (HMMs). In HMMs the states are hidden, and the hidden processes produce a sequence of observations. In this paper we extend the logic PCTL* to POCTL*. With our logic one can state properties such as "there is at least a 90 percent probability that the model produces a given sequence of observations" over HMMs. Subsequently, we give model checking algorithms for POCTL* over HMMs

Formalization of Discrete-time Markov Chains in HOL

Markov chains are extensively used in the modeling and analysis of engineering and scientific problems which can be expressed as random processes with the memoryless property. Usually, paper-and-pencil proofs, simulation or computer algebra software are used to analyze Markovian models. However, these techniques either are not scalable or do not guarantee accurate results, which are vital in safety-critical systems. To improve the accuracy of the analysis, probabilistic model checking has been recently proposed to formally analyze Markovian systems. However, model checking suffers from the inherent state-explosion problem and thus has a very limited scope in terms of analyzing Markovian models.\newline \indent In order to overcome the above mentioned limitations, this thesis advocates the usage of higher-order-logic theorem proving for conducting the analysis of Markov chains. We present the higher-order-logic formalization of Discrete-time Markov Chains with finite number of discrete states. We also verify some of their most widely used properties using a theorem prover. These foundations allow us to formally express and reason about Markov chains within the sound core of a theorem prover and thus attain precise results. Moreover, by building upon these foundational results, this thesis also presents the formalization of classified discrete-time Markov chains and hidden Markov chains in higher-order logic. These are widely used concepts in the analysis of Markovian models and thus allow us to tackle the formal analysis of a wide range of engineering and scientific systems. For illustration purposes, the thesis also presents some applications including a binary communication channel, the automatic mail quality measurement (AMQM) protocol, a DNA sequence, a least recently used (LRU) stack model and the birth-death process

Technical Report: Distribution Temporal Logic: Combining Correctness with Quality of Estimation

We present a new temporal logic called Distribution Temporal Logic (DTL) defined over predicates of belief states and hidden states of partially observable systems. DTL can express properties involving uncertainty and likelihood that cannot be described by existing logics. A co-safe formulation of DTL is defined and algorithmic procedures are given for monitoring executions of a partially observable Markov decision process with respect to such formulae. A simulation case study of a rescue robotics application outlines our approach.Comment: More expanded version of "Distribution Temporal Logic: Combining Correctness with Quality of Estimation" to appear in IEEE CDC 201

Inference with Constrained Hidden Markov Models in PRISM

A Hidden Markov Model (HMM) is a common statistical model which is widely used for analysis of biological sequence data and other sequential phenomena. In the present paper we show how HMMs can be extended with side-constraints and present constraint solving techniques for efficient inference. Defining HMMs with side-constraints in Constraint Logic Programming have advantages in terms of more compact expression and pruning opportunities during inference. We present a PRISM-based framework for extending HMMs with side-constraints and show how well-known constraints such as cardinality and all different are integrated. We experimentally validate our approach on the biologically motivated problem of global pairwise alignment

Technical report: Distribution Temporal Logic: combining correctness with quality of estimation

We present a new temporal logic called Distribution Temporal Logic (DTL) defined over predicates of belief states and hidden states of partially observable systems. DTL can express properties involving uncertainty and likelihood that cannot be described by existing logics. A co-safe formulation of DTL is defined and algorithmic procedures are given for monitoring executions of a partially observable Markov decision process with respect to such formulae. A simulation case study of a rescue robotics application outlines our approach

Model-checking tool support for quantitative risk analysis and design for safety

This paper is concerned with quantitative analysis of tolerance of sensor hardware failures by control system software. The aim is to help the system designer evaluate the efectiveness of risk reduction measures in the system design. This paper proposes an approach for using stochastic model checking to evaluate how likely a given sensor failure mode is to lead to a hazardous system failure, taking control logic and sensor-update timing failures into account. In particular we propose two complementary techniques: one for examining short- term consequences of component failures and the other for examining more subtle longer-term consequences (so-called hidden failures). The techniques overcome scaling issues and yield valuable insights into the relative merits of dierent design decisions. The PRISM model checker is used for stochastic analysis of Continuous Time Markov Chain (CTMC) system models. The approach is illustrated on a case study from manufacturing, involving an industrial metal Press. Although relatively simple, the Press exhibits a wide range of different behaviours, including hidden failures and subtle race conditions

Bounded Satisfiability for PCTL

While model checking PCTL for Markov chains is decidable in polynomial-time, the decidability of PCTL satisfiability, as well as its finite model property, are long standing open problems. While general satisfiability is an intriguing challenge from a purely theoretical point of view, we argue that general solutions would not be of interest to practitioners: such solutions could be too big to be implementable or even infinite. Inspired by bounded synthesis techniques, we turn to the more applied problem of seeking models of a bounded size: we restrict our search to implementable -- and therefore reasonably simple -- models. We propose a procedure to decide whether or not a given PCTL formula has an implementable model by reducing it to an SMT problem. We have implemented our techniques and found that they can be applied to the practical problem of sanity checking -- a procedure that allows a system designer to check whether their formula has an unexpectedly small model

Formal Dependability Engineering with MIOA

In this paper, we introduce MIOA, a stochastic process algebra-like specification language with datatypes, as well as a logic intSPDL, and its model checking algorithms. MIOA, which stands for Markovian input/output automata language, is an extension of Lynch's input/automata with Markovian timed transitions.MIOA can serve both as a fully fledged ``stand-alone'' specification language and the semantic model for the architectural dependability modelling and evaluation language Arcade. The logic intSPDL is an extension of the stochastic logic SPDL, to deal with the specialties of MIOA. intSPDL in the context of Arcade can be seen as the semantic model of abstract and complex dependability measures that can be defined in the Arcade framework. We define syntax and semantics of both MIOA and intSPDL, and show examples of applying MIOA and intSPDL in the realm of dependability modelling with Arcade