Hitting Times and Probabilities for Imprecise Markov Chains

We consider the problem of characterising expected hitting times and hitting probabilities for imprecise Markov chains. To this end, we consider three distinct ways in which imprecise Markov chains have been defined in the literature: as sets of homogeneous Markov chains, as sets of more general stochastic processes, and as game-theoretic probability models. Our first contribution is that all these different types of imprecise Markov chains have the same lower and upper expected hitting times, and similarly the hitting probabilities are the same for these three types. Moreover, we provide a characterisation of these quantities that directly generalises a similar characterisation for precise, homogeneous Markov chains

Hitting times and probabilities for imprecise Markov chains

We consider the problem of characterising expected hitting times and hitting probabilities for imprecise Markov chains. To this end, we consider three distinct ways in which imprecise Markov chains have been defined in the literature: as sets of homogeneous Markov chains, as sets of more general stochastic processes, and as game-theoretic probability models. Our first contribution is that all these different types of imprecise Markov chains have the same lower and upper expected hitting times, and similarly the hitting probabilities are the same for these three types. Moreover, we provide a characterisation of these quantities that directly generalises a similar characterisation for precise, homogeneous Markov chains

In search of a global belief model for discrete-time uncertain processes

To model discrete-time uncertain processes, we argue for the use of a global belief model in the form of an upper expectation that satisfies a number of simple and intuitive axioms. We motivate these axioms on the basis of two possible interpretations for this upper expectation: a behavioural interpretation similar to that of Walley’s, and an interpretation in terms of upper envelopes of linear expectations. Subsequently, we show that the most conservative upper expectation satisfying our axioms coincides with a particular version of the game-theoretic upper expectation introduced by Shafer and Vovk. This has two important implications. On the one hand, it guarantees that there is a unique most conservative global belief model satisfying our axioms. On the other hand, it shows that Shafer and Vovk’s model can be given an axiomatic characterisation, thereby providing an alternative motivation for adopting this model, even outside their framework

A Particular Upper Expectation as Global Belief Model for Discrete-Time Finite-State Uncertain Processes

To model discrete-time finite-state uncertain processes, we argue for the use of a global belief model in the form of an upper expectation that is the most conservative one under a set of basic axioms. Our motivation for these axioms, which describe how local and global belief models should be related, is based on two possible interpretations for an upper expectation: a behavioural one similar to Walley's, and an interpretation in terms of upper envelopes of linear expectations. We show that the most conservative upper expectation satisfying our axioms, that is, our model of choice, coincides with a particular version of the game-theoretic upper expectation introduced by Shafer and Vovk. This has two important implications: it guarantees that there is a unique most conservative global belief model satisfying our axioms; and it shows that Shafer and Vovk's model can be given an axiomatic characterisation and thereby provides an alternative motivation for adopting this model, even outside their game-theoretic framework. Finally, we relate our model to the upper expectation resulting from a traditional measure-theoretic approach. We show that this measure-theoretic upper expectation also satisfies the proposed axioms, which implies that it is dominated by our model or, equivalently, the game-theoretic model. Moreover, if all local models are precise, all three models coincide.Comment: Extension of the conference paper `In Search of a Global Belief Model for Discrete-Time Uncertain Processes

Average Behaviour in Discrete-Time Imprecise Markov Chains: A Study of Weak Ergodicity

We study the limit behaviour of upper and lower bounds on expected time averages in imprecise Markov chains; a generalised type of Markov chain where the local dynamics, traditionally characterised by transition probabilities, are now represented by sets of `plausible' transition probabilities. Our first main result is a necessary and sufficient condition under which these upper and lower bounds, called upper and lower expected time averages, will converge as time progresses towards infinity to limit values that do not depend on the process' initial state. Our condition is considerably weaker than that needed for ergodic behaviour; a similar notion which demands that marginal upper and lower expectations of functions at a single time instant converge to so-called limit-or steady state-upper and lower expectations. For this reason, we refer to our notion as `weak ergodicity'. Our second main result shows that, as far as this weakly ergodic behaviour is concerned, one should not worry about which type of independence assumption to adopt-epistemic irrelevance, complete independence or repetition independence. The characterisation of weak ergodicity as well as the limit values of upper and lower expected time averages do not depend on such a choice. Notably, this type of robustness is not exhibited by the notion of ergodicity and the related inferences of limit upper and lower expectations. Finally, though limit upper and lower expectations are often used to provide approximate information about the limit behaviour of time averages, we show that such an approximation is sub-optimal and that it can be significantly improved by directly using upper and lower expected time averages.Comment: arXiv admin note: substantial text overlap with arXiv:2002.0566

Average behaviour of imprecise Markov chains : a single pointwise ergodic theorem for six different models

We study the average behaviour of imprecise Markov chains; a generalised type of Markov chain where local probabilities are partially specified, and where structural assumptions such as Markovianity are weakened. In particular, we prove a pointwise ergodic theorem that provides (strictly) almost sure bounds on the long term average of any real function of the state of such an imprecise Markov chain. Compared to an earlier ergodic theorem by De Cooman et al. (2006), our result requires weaker conditions, provides tighter bounds, and applies to six different types of models

Global upper expectations for discrete-time stochastic processes : in practice, they are all the same!

We consider three different types of global uncertainty models for discrete-time stochastic processes: measure-theoretic upper expectations, game-theoretic upper expectations and axiomatic upper expectations. The last two are known to be identical. We show that they coincide with measure-theoretic upper expectations on two distinct domains: monotone pointwise limits of finitary gambles, and bounded below Borel-measurable variables. We argue that these domains cover most practical inferences, and that therefore, in practice, it does not matter which model is used

Game-theoretic upper expectations for discrete-time finite-state uncertain processes

Game-theoretic upper expectations are joint (global) probability models that mathematically describe the behaviour of uncertain processes in terms of supermartingales; capital processes corresponding to available betting strategies. Compared to (the more common) measure-theoretic expectation functionals, they are not bounded to restrictive assumptions such as measurability or precision, yet succeed in preserving, or even generalising many of their fundamental properties. We focus on a discrete-time setting where local state spaces are finite and, in this specific context, build on the existing work of Shafer and Vovk; the main developers of the framework of game-theoretic upper expectations. In a first part, we study Shafer and Vovk's characterisation of a local upper expectation and show how it is related to Walley's behavioural notion of coherence. The second part consists in a study of game-theoretic upper expectations on a more global level, where several alternative definitions, as well as a broad range of properties are derived, e.g. the law of iterated upper expectations, compatibility with local models, coherence properties,... Our main contribution, however, concerns the continuity behaviour of these operators. We prove continuity with respect to non-increasing sequences of so-called lower cuts and continuity with respect to non-increasing sequences of finitary functions. We moreover show that the game-theoretic upper expectation is uniquely determined by its values on the domain of bounded below limits of finitary functions, and additionally show that, for any such limit, the limiting sequence can be constructed in such a way that the game-theoretic upper expectation is continuous with respect to this particular sequence.Comment: arXiv admin note: text overlap with arXiv:1902.0940