536 research outputs found

    Two-state imprecise Markov chains for statistical modelling of two-state non-Markovian processes.

    Get PDF
    This paper proposes a method for fitting a two-state imprecise Markov chain to time series data from a twostate non-Markovian process. Such non-Markovian processes are common in practical applications. We focus on how to fit modelling parameters based on data from a process where time to transition is not exponentially distributed, thereby violating the Markov assumption. We do so by first fitting a many-state (i.e. having more than two states) Markov chain to the data, through its associated phase-type distribution. Then, we lump the process to a two-state imprecise Markov chain. In practical applications, a two-state imprecise Markov chain might be more convenient than a many-state Markov chain, as we have closed analytic expressions for typical quantities of interest (including the lower and upper expectation of any function of the state at any point in time). A numerical example demonstrates how the entire inference process (fitting and prediction) can be done using Markov chain Monte Carlo, for a given set of prior distributions on the parameters. In particular, we numerically identify the set of posterior densities and posterior lower and upper expectations on all model parameters and predictive quantities. We compare our inferences under a range of sample sizes and model assumptions. Keywords: imprecise Markov chain, estimation, reliability, Markov assumption, MCM

    Hitting times and probabilities for imprecise Markov chains

    Get PDF
    We consider the problem of characterising expected hitting times and hitting probabilities for imprecise Markov chains. To this end, we consider three distinct ways in which imprecise Markov chains have been defined in the literature: as sets of homogeneous Markov chains, as sets of more general stochastic processes, and as game-theoretic probability models. Our first contribution is that all these different types of imprecise Markov chains have the same lower and upper expected hitting times, and similarly the hitting probabilities are the same for these three types. Moreover, we provide a characterisation of these quantities that directly generalises a similar characterisation for precise, homogeneous Markov chains

    A Recursive Algorithm for Computing Inferences in Imprecise Markov Chains

    Full text link
    We present an algorithm that can efficiently compute a broad class of inferences for discrete-time imprecise Markov chains, a generalised type of Markov chains that allows one to take into account partially specified probabilities and other types of model uncertainty. The class of inferences that we consider contains, as special cases, tight lower and upper bounds on expected hitting times, on hitting probabilities and on expectations of functions that are a sum or product of simpler ones. Our algorithm exploits the specific structure that is inherent in all these inferences: they admit a general recursive decomposition. This allows us to achieve a computational complexity that scales linearly in the number of time points on which the inference depends, instead of the exponential scaling that is typical for a naive approach

    Formal analysis techniques for gossiping protocols

    Get PDF
    We give a survey of formal verification techniques that can be used to corroborate existing experimental results for gossiping protocols in a rigorous manner. We present properties of interest for gossiping protocols and discuss how various formal evaluation techniques can be employed to predict them

    Abstractions of stochastic hybrid systems

    Get PDF
    Many control systems have large, infinite state space that can not be easily abstracted. One method to analyse and verify these systems is reachability analysis. It is frequently used for air traffic control and power plants. Because of lack of complete information about the environment or unpredicted changes, the stochastic approach is a viable alternative. In this paper, different ways of introducing rechability under uncertainty are presented. A new concept of stochastic bisimulation is introduced and its connection with the reachability analysis is established. The work is mainly motivated by safety critical situations in air traffic control (like collision detection and avoidance) and formal tools are based on stochastic analysis

    Imprecise Continuous-Time Markov Chains

    Get PDF
    Continuous-time Markov chains are mathematical models that are used to describe the state-evolution of dynamical systems under stochastic uncertainty, and have found widespread applications in various fields. In order to make these models computationally tractable, they rely on a number of assumptions that may not be realistic for the domain of application; in particular, the ability to provide exact numerical parameter assessments, and the applicability of time-homogeneity and the eponymous Markov property. In this work, we extend these models to imprecise continuous-time Markov chains (ICTMC's), which are a robust generalisation that relaxes these assumptions while remaining computationally tractable. More technically, an ICTMC is a set of "precise" continuous-time finite-state stochastic processes, and rather than computing expected values of functions, we seek to compute lower expectations, which are tight lower bounds on the expectations that correspond to such a set of "precise" models. Note that, in contrast to e.g. Bayesian methods, all the elements of such a set are treated on equal grounds; we do not consider a distribution over this set. The first part of this paper develops a formalism for describing continuous-time finite-state stochastic processes that does not require the aforementioned simplifying assumptions. Next, this formalism is used to characterise ICTMC's and to investigate their properties. The concept of lower expectation is then given an alternative operator-theoretic characterisation, by means of a lower transition operator, and the properties of this operator are investigated as well. Finally, we use this lower transition operator to derive tractable algorithms (with polynomial runtime complexity w.r.t. the maximum numerical error) for computing the lower expectation of functions that depend on the state at any finite number of time points

    Algorithmic Methods in Queues and in the Exploration of Point Processes

    Get PDF
    This is a review of methodology for the algorithmic study of some useful models in point process and queueing theory, as discussed in three lectures at the Summer Institute at Sozopol, Bulgaria. We provide references to sources where the extensive details of this work are found. For future investigation, some open problems and new methodological approaches are proposed

    Hitting Times and Probabilities for Imprecise Markov Chains

    Get PDF
    We consider the problem of characterising expected hitting times and hitting probabilities for imprecise Markov chains. To this end, we consider three distinct ways in which imprecise Markov chains have been defined in the literature: as sets of homogeneous Markov chains, as sets of more general stochastic processes, and as game-theoretic probability models. Our first contribution is that all these different types of imprecise Markov chains have the same lower and upper expected hitting times, and similarly the hitting probabilities are the same for these three types. Moreover, we provide a characterisation of these quantities that directly generalises a similar characterisation for precise, homogeneous Markov chains

    Modelling and estimation for random fields

    Get PDF
    Caption title.Includes bibliographical references (p. [21]-[22]).Supported by Air Force Office of Scientific Research. AFOSR-89-0276-C Supported by the Army Research Office. DAAL03-92-G-0115Sanjoy K. Mitter
    corecore