536 research outputs found
Two-state imprecise Markov chains for statistical modelling of two-state non-Markovian processes.
This paper proposes a method for fitting a two-state
imprecise Markov chain to time series data from a twostate
non-Markovian process. Such non-Markovian
processes are common in practical applications. We
focus on how to fit modelling parameters based on
data from a process where time to transition is not exponentially
distributed, thereby violating the Markov
assumption. We do so by first fitting a many-state (i.e.
having more than two states) Markov chain to the
data, through its associated phase-type distribution.
Then, we lump the process to a two-state imprecise
Markov chain. In practical applications, a two-state imprecise
Markov chain might be more convenient than
a many-state Markov chain, as we have closed analytic
expressions for typical quantities of interest (including
the lower and upper expectation of any function of
the state at any point in time). A numerical example
demonstrates how the entire inference process (fitting
and prediction) can be done using Markov chain Monte
Carlo, for a given set of prior distributions on the parameters.
In particular, we numerically identify the set
of posterior densities and posterior lower and upper
expectations on all model parameters and predictive
quantities. We compare our inferences under a range
of sample sizes and model assumptions.
Keywords: imprecise Markov chain, estimation, reliability,
Markov assumption, MCM
Hitting times and probabilities for imprecise Markov chains
We consider the problem of characterising expected hitting times and hitting probabilities for imprecise Markov chains. To this end, we consider three distinct ways in which imprecise Markov chains have been defined in the literature: as sets of homogeneous Markov chains, as sets of more general stochastic processes, and as game-theoretic probability models. Our first contribution is that all these different types of imprecise Markov chains have the same lower and upper expected hitting times, and similarly the hitting probabilities are the same for these three types. Moreover, we provide a characterisation of these quantities that directly generalises a similar characterisation for precise, homogeneous Markov chains
A Recursive Algorithm for Computing Inferences in Imprecise Markov Chains
We present an algorithm that can efficiently compute a broad class of
inferences for discrete-time imprecise Markov chains, a generalised type of
Markov chains that allows one to take into account partially specified
probabilities and other types of model uncertainty. The class of inferences
that we consider contains, as special cases, tight lower and upper bounds on
expected hitting times, on hitting probabilities and on expectations of
functions that are a sum or product of simpler ones. Our algorithm exploits the
specific structure that is inherent in all these inferences: they admit a
general recursive decomposition. This allows us to achieve a computational
complexity that scales linearly in the number of time points on which the
inference depends, instead of the exponential scaling that is typical for a
naive approach
Formal analysis techniques for gossiping protocols
We give a survey of formal verification techniques that can be used to corroborate existing experimental results for gossiping protocols in a rigorous manner. We present properties of interest for gossiping protocols and discuss how various formal evaluation techniques can be employed to predict them
Abstractions of stochastic hybrid systems
Many control systems have large, infinite state space that can not be easily abstracted. One method to analyse and verify these systems is reachability analysis. It is frequently used for air traffic control and power plants. Because of lack of complete information about the environment or unpredicted changes, the stochastic approach is a viable alternative. In this paper, different ways of introducing rechability under uncertainty are presented. A new concept of stochastic bisimulation is introduced and its connection with the reachability analysis is established. The work is mainly motivated by safety critical situations in air traffic control (like collision detection and avoidance) and formal tools are based on stochastic analysis
Imprecise Continuous-Time Markov Chains
Continuous-time Markov chains are mathematical models that are used to
describe the state-evolution of dynamical systems under stochastic uncertainty,
and have found widespread applications in various fields. In order to make
these models computationally tractable, they rely on a number of assumptions
that may not be realistic for the domain of application; in particular, the
ability to provide exact numerical parameter assessments, and the applicability
of time-homogeneity and the eponymous Markov property. In this work, we extend
these models to imprecise continuous-time Markov chains (ICTMC's), which are a
robust generalisation that relaxes these assumptions while remaining
computationally tractable.
More technically, an ICTMC is a set of "precise" continuous-time finite-state
stochastic processes, and rather than computing expected values of functions,
we seek to compute lower expectations, which are tight lower bounds on the
expectations that correspond to such a set of "precise" models. Note that, in
contrast to e.g. Bayesian methods, all the elements of such a set are treated
on equal grounds; we do not consider a distribution over this set.
The first part of this paper develops a formalism for describing
continuous-time finite-state stochastic processes that does not require the
aforementioned simplifying assumptions. Next, this formalism is used to
characterise ICTMC's and to investigate their properties. The concept of lower
expectation is then given an alternative operator-theoretic characterisation,
by means of a lower transition operator, and the properties of this operator
are investigated as well. Finally, we use this lower transition operator to
derive tractable algorithms (with polynomial runtime complexity w.r.t. the
maximum numerical error) for computing the lower expectation of functions that
depend on the state at any finite number of time points
Algorithmic Methods in Queues and in the Exploration of Point Processes
This is a review of methodology for the algorithmic study of some useful models
in point process and queueing theory, as discussed in three lectures at the Summer
Institute at Sozopol, Bulgaria. We provide references to sources where the extensive
details of this work are found. For future investigation, some open problems and
new methodological approaches are proposed
Hitting Times and Probabilities for Imprecise Markov Chains
We consider the problem of characterising expected hitting times and hitting
probabilities for imprecise Markov chains. To this end, we consider three
distinct ways in which imprecise Markov chains have been defined in the
literature: as sets of homogeneous Markov chains, as sets of more general
stochastic processes, and as game-theoretic probability models. Our first
contribution is that all these different types of imprecise Markov chains have
the same lower and upper expected hitting times, and similarly the hitting
probabilities are the same for these three types. Moreover, we provide a
characterisation of these quantities that directly generalises a similar
characterisation for precise, homogeneous Markov chains
Modelling and estimation for random fields
Caption title.Includes bibliographical references (p. [21]-[22]).Supported by Air Force Office of Scientific Research. AFOSR-89-0276-C Supported by the Army Research Office. DAAL03-92-G-0115Sanjoy K. Mitter
- …