1,889 research outputs found
A Faster-Than Relation for Semi-Markov Decision Processes
When modeling concurrent or cyber-physical systems, non-functional
requirements such as time are important to consider. In order to improve the
timing aspects of a model, it is necessary to have some notion of what it means
for a process to be faster than another, which can guide the stepwise
refinement of the model. To this end we study a faster-than relation for
semi-Markov decision processes and compare it to standard notions for relating
systems. We consider the compositional aspects of this relation, and show that
the faster-than relation is not a precongruence with respect to parallel
composition, hence giving rise to so-called parallel timing anomalies. We take
the first steps toward understanding this problem by identifying decidable
conditions sufficient to avoid parallel timing anomalies in the absence of
non-determinism.Comment: In Proceedings QAPL 2019, arXiv:2001.0616
Verification and Control of Partially Observable Probabilistic Real-Time Systems
We propose automated techniques for the verification and control of
probabilistic real-time systems that are only partially observable. To formally
model such systems, we define an extension of probabilistic timed automata in
which local states are partially visible to an observer or controller. We give
a probabilistic temporal logic that can express a range of quantitative
properties of these models, relating to the probability of an event's
occurrence or the expected value of a reward measure. We then propose
techniques to either verify that such a property holds or to synthesise a
controller for the model which makes it true. Our approach is based on an
integer discretisation of the model's dense-time behaviour and a grid-based
abstraction of the uncountable belief space induced by partial observability.
The latter is necessarily approximate since the underlying problem is
undecidable, however we show how both lower and upper bounds on numerical
results can be generated. We illustrate the effectiveness of the approach by
implementing it in the PRISM model checker and applying it to several case
studies, from the domains of computer security and task scheduling
Flexible and practical modeling of animal telemetry data: hidden Markov models and extensions
We discuss hidden Markov-type models for fitting a variety of multistate random walks to wildlife movement data. Discrete-time hidden Markov models (HMMs) achieve considerable computational gains by focusing on observations that are regularly spaced in time, and for which the measurement error is negligible. These conditions are often met, in particular for data related to terrestrial animals, so that a likelihood-based HMM approach is feasible. We describe a number of extensions of HMMs for animal movement modeling, including more flexible state transition models and individual random effects (fitted in a non-Bayesian framework). In particular we consider so-called hidden semi-Markov models, which may substantially improve the goodness of fit and provide important insights into the behavioral state switching dynamics. To showcase the expediency of these methods, we consider an application of a hierarchical hidden semi-Markov model to multiple bison movement paths
A tool for model-checking Markov chains
Markov chains are widely used in the context of the performance and reliability modeling of various systems. Model checking of such chains with respect to a given (branching) temporal logic formula has been proposed for both discrete [34, 10] and continuous time settings [7, 12]. In this paper, we describe a prototype model checker for discrete and continuous-time Markov chains, the Erlangen-Twente Markov Chain Checker EĆMC2, where properties are expressed in appropriate extensions of CTL. We illustrate the general benefits of this approach and discuss the structure of the tool. Furthermore, we report on successful applications of the tool to some examples, highlighting lessons learned during the development and application of EĆMC2
Verification and control of partially observable probabilistic systems
We present automated techniques for the verification and control of partially observable, probabilistic systems for both discrete and dense models of time. For the discrete-time case, we formally model these systems using partially observable Markov decision processes; for dense time, we propose an extension of probabilistic timed automata in which local states are partially visible to an observer or controller. We give probabilistic temporal logics that can express a range of quantitative properties of these models, relating to the probability of an eventās occurrence or the expected value of a reward measure. We then propose techniques to either verify that such a property holds or synthesise a controller for the model which makes it true. Our approach is based on a grid-based abstraction of the uncountable belief space induced by partial observability and, for dense-time models, an integer discretisation of real-time behaviour. The former is necessarily approximate since the underlying problem is undecidable, however we show how both lower and upper bounds on numerical results can be generated. We illustrate the effectiveness of the approach by implementing it in the PRISM model checker and applying it to several case studies from the domains of task and network scheduling, computer security and planning
- ā¦