Imprecise Continuous-Time Markov Chains

Continuous-time Markov chains are mathematical models that are used to describe the state-evolution of dynamical systems under stochastic uncertainty, and have found widespread applications in various fields. In order to make these models computationally tractable, they rely on a number of assumptions that may not be realistic for the domain of application; in particular, the ability to provide exact numerical parameter assessments, and the applicability of time-homogeneity and the eponymous Markov property. In this work, we extend these models to imprecise continuous-time Markov chains (ICTMC's), which are a robust generalisation that relaxes these assumptions while remaining computationally tractable. More technically, an ICTMC is a set of "precise" continuous-time finite-state stochastic processes, and rather than computing expected values of functions, we seek to compute lower expectations, which are tight lower bounds on the expectations that correspond to such a set of "precise" models. Note that, in contrast to e.g. Bayesian methods, all the elements of such a set are treated on equal grounds; we do not consider a distribution over this set. The first part of this paper develops a formalism for describing continuous-time finite-state stochastic processes that does not require the aforementioned simplifying assumptions. Next, this formalism is used to characterise ICTMC's and to investigate their properties. The concept of lower expectation is then given an alternative operator-theoretic characterisation, by means of a lower transition operator, and the properties of this operator are investigated as well. Finally, we use this lower transition operator to derive tractable algorithms (with polynomial runtime complexity w.r.t. the maximum numerical error) for computing the lower expectation of functions that depend on the state at any finite number of time points

Efficient computation of updated lower expectations for imprecise continuous-time hidden Markov chains

We consider the problem of performing inference with imprecise continuous-time hidden Markov chains, that is, imprecise continuous-time Markov chains that are augmented with random output variables whose distribution depends on the hidden state of the chain. The prefix `imprecise' refers to the fact that we do not consider a classical continuous-time Markov chain, but replace it with a robust extension that allows us to represent various types of model uncertainty, using the theory of imprecise probabilities. The inference problem amounts to computing lower expectations of functions on the state-space of the chain, given observations of the output variables. We develop and investigate this problem with very few assumptions on the output variables; in particular, they can be chosen to be either discrete or continuous random variables. Our main result is a polynomial runtime algorithm to compute the lower expectation of functions on the state-space at any given time-point, given a collection of observations of the output variables

Imprecise continuous-time Markov chains : efficient computational methods with guaranteed error bounds

Imprecise continuous-time Markov chains are a robust type of continuous-time Markov chains that allow for partially specified time-dependent parameters. Computing inferences for them requires the solution of a non-linear differential equation. As there is no general analytical expression for this solution, efficient numerical approximation methods are essential to the applicability of this model. We here improve the uniform approximation method of Krak et al. (2016) in two ways and propose a novel and more efficient adaptive approximation method. For ergodic chains, we also provide a method that allows us to approximate stationary distributions up to any desired maximal error

Using imprecise continuous time Markov chains for assessing the reliability of power networks with common cause failure and non-immediate repair.

We explore how imprecise continuous time Markov chains can improve traditional reliability models based on precise continuous time Markov chains. Specifically, we analyse the reliability of power networks under very weak statistical assumptions, explicitly accounting for non-stationary failure and repair rates and the limited accuracy by which common cause failure rates can be estimated. Bounds on typical quantities of interest are derived, namely the expected time spent in system failure state, as well as the expected number of transitions to that state. A worked numerical example demonstrates the theoretical techniques described. Interestingly, the number of iterations required for convergence is observed to be much lower than current theoretical bounds

Bounding inferences for large-scale continuous-time Markov chains : a new approach based on lumping and imprecise Markov chains

If the state space of a homogeneous continuous-time Markov chain is too large, making inferences becomes computationally infeasible. Fortunately, the state space of such a chain is usually too detailed for the inferences we are interested in, in the sense that a less detailed—smaller—state space suffices to unambiguously formalise the inference. However, in general this so-called lumped state space inhibits computing exact inferences because the corresponding dynamics are unknown and/or intractable to obtain. We address this issue by considering an imprecise continuous-time Markov chain. In this way, we are able to provide guaranteed lower and upper bounds for the inferences of interest, without suffering from the curse of dimensionality

Using imprecise continuous time Markov chains for assessing the reliability of power networks with common cause failure and non-immediate repair

We explore how imprecise continuous time Markov chains can improve traditional reliability models based on precise continuous time Markov chains. Specifically, we analyse the reliability of power networks under very weak statistical assumptions, explicitly accounting for non-stationary failure and repair rates and the limited accuracy by which common cause failure rates can be estimated. Bounds on typical quantities of interest are derived, namely the expected time spent in system failure state, as well as the expected number of transitions to that state. A worked numerical example demonstrates the theoretical techniques described. Interestingly, the number of iterations required for convergence is observed to be much lower than current theoretical bounds

Modelling Spectrum Assignment in a Two-Service Flexi-Grid Optical Link with Imprecise Continuous-Time Markov Chains

Flexi-grid optical networks (Gerstel et al., 2012) are a novel paradigm for managing the capacity of optical fibers more efficiently. The idea is to divide the spectrum in small frequency slices, and to consider an allocation policy that adaptively assigns one or multiple contiguous slices to incoming bandwidth requests, depending on their size. However, as new requests arrive and old requests are served and return resources to the free pool, the spectrum might become fragmented, leading to inefficiency and unfairness. It is therefore necessary to quantify the performance of a given spectrum allocation policy, for example by determining the probability that a bandwidth request is blocked, in the sense that it cannot be allocated because there are not enough contiguous free slices. To determine blocking probabilities for an optical link with traffic requests of two different sizes and a random allocation policy, Kim et al. (2015) use a Markov chain. Unfortunately, the number of states of this Markov chain grows exponentially with the number of available frequency slices, making it infeasible to determine blocking probabilities for large systems. Therefore, Kim et al. (2015) also consider a second Markov chain, with a highly reduced state space and approximate transition rates, to obtain approximations of these blocking probabilities. In this contribution, we first show how to construct such full and reduced-state Markov chains for two other allocation policies, and compare these with the random policy. Next, we introduce a so-called imprecise Markov chain, which has the same reduced state space but imprecise (interval-valued) transition rates, and explain how it can be used to determine guaranteed upper and lower bounds for --- instead of approximations of --- blocking probabilities, for different families of allocation policies

Hitting times and probabilities for imprecise Markov chains

We consider the problem of characterising expected hitting times and hitting probabilities for imprecise Markov chains. To this end, we consider three distinct ways in which imprecise Markov chains have been defined in the literature: as sets of homogeneous Markov chains, as sets of more general stochastic processes, and as game-theoretic probability models. Our first contribution is that all these different types of imprecise Markov chains have the same lower and upper expected hitting times, and similarly the hitting probabilities are the same for these three types. Moreover, we provide a characterisation of these quantities that directly generalises a similar characterisation for precise, homogeneous Markov chains