Model checking for imprecise Markov chains.

We extend probabilistic computational tree logic for expressing properties of Markov chains to imprecise Markov chains, and provide an efficient algorithm for model checking of imprecise Markov chains. Thereby, we provide a formal framework to answer a very wide range of questions about imprecise Markov chains, in a systematic and computationally efficient way

Hitting Times and Probabilities for Imprecise Markov Chains

We consider the problem of characterising expected hitting times and hitting probabilities for imprecise Markov chains. To this end, we consider three distinct ways in which imprecise Markov chains have been defined in the literature: as sets of homogeneous Markov chains, as sets of more general stochastic processes, and as game-theoretic probability models. Our first contribution is that all these different types of imprecise Markov chains have the same lower and upper expected hitting times, and similarly the hitting probabilities are the same for these three types. Moreover, we provide a characterisation of these quantities that directly generalises a similar characterisation for precise, homogeneous Markov chains

Hitting times and probabilities for imprecise Markov chains

We consider the problem of characterising expected hitting times and hitting probabilities for imprecise Markov chains. To this end, we consider three distinct ways in which imprecise Markov chains have been defined in the literature: as sets of homogeneous Markov chains, as sets of more general stochastic processes, and as game-theoretic probability models. Our first contribution is that all these different types of imprecise Markov chains have the same lower and upper expected hitting times, and similarly the hitting probabilities are the same for these three types. Moreover, we provide a characterisation of these quantities that directly generalises a similar characterisation for precise, homogeneous Markov chains

Model Checking for Imprecise Markov Chains

We extend probabilistic computational tree logic for expressing properties of Markov chains to imprecise Markov chains, and provide an efficient algorithm for model checking of imprecise Markov chains. Thereby, we provide a formal framework to answer a very wide range of questions about imprecise Markov chains, in a systematic and computationally efficient way

Limit behaviour for imprecise Markov Chains

An imprecise Markov chain is defined by a closed convex set of transition matrices instead of a unique one for a classical precise Markov chain. These imprecise Markov chains allow us to model situations where we do not have enough information to specify a unique transition matrix, or to approximate the behaviour of non‐stationary Markov chains. We show that there are efficient, dynamic programming‐ like ways to work and reason with these imprecise Markov chains; e.g. to calculate the resulting distribution over the states at any time instant. We prove that this distribution converges in time, similarly to the precise case and under very mild conditions. We thus effectively prove a Perron‐Frobenius theorem for a special class of non‐linear systems

A Recursive Algorithm for Computing Inferences in Imprecise Markov Chains

We present an algorithm that can efficiently compute a broad class of inferences for discrete-time imprecise Markov chains, a generalised type of Markov chains that allows one to take into account partially specified probabilities and other types of model uncertainty. The class of inferences that we consider contains, as special cases, tight lower and upper bounds on expected hitting times, on hitting probabilities and on expectations of functions that are a sum or product of simpler ones. Our algorithm exploits the specific structure that is inherent in all these inferences: they admit a general recursive decomposition. This allows us to achieve a computational complexity that scales linearly in the number of time points on which the inference depends, instead of the exponential scaling that is typical for a naive approach

Imprecise Markov chains and their limit behaviour

When the initial and transition probabilities of a finite Markov chain in discrete time are not well known, we should perform a sensitivity analysis. This can be done by considering as basic uncertainty models the so-called credal sets that these probabilities are known or believed to belong to, and by allowing the probabilities to vary over such sets. This leads to the definition of an imprecise Markov chain. We show that the time evolution of such a system can be studied very efficiently using so-called lower and upper expectations, which are equivalent mathematical representations of credal sets. We also study how the inferred credal set about the state at time n evolves as n goes to infinity: under quite unrestrictive conditions, it converges to a uniquely invariant credal set, regardless of the credal set given for the initial state. This leads to a non-trivial generalisation of the classical Perron-Frobenius Theorem to imprecise Markov chains.Comment: v1: 28 pages, 8 figures; v2: 31 pages, 9 figures, major revision after review: added, modified, and removed material (no results dropped, results added), moved proofs to an appendi

Perturbation bounds and degree of imprecision for uniquely convergent imprecise Markov chains

The effect of perturbations of parameters for uniquely convergent imprecise Markov chains is studied. We provide the maximal distance between the distributions of original and perturbed chain and maximal degree of imprecision, given the imprecision of the initial distribution. The bounds on the errors and degrees of imprecision are found for the distributions at finite time steps, and for the stationary distributions as well.Comment: 20 pages, 2 figure

A classification of invariant distributions and convergence of imprecise Markov chains

We analyse the structure of imprecise Markov chains and study their convergence by means of accessibility relations. We first identify the sets of states, so-called minimal permanent classes, that are the minimal sets capable of containing and preserving the whole probability mass of the chain. These classes generalise the essential classes known from the classical theory. We then define a class of extremal imprecise invariant distributions and show that they are uniquely determined by the values of the upper probability on minimal permanent classes. Moreover, we give conditions for unique convergence to these extremal invariant distributions