877 research outputs found
Multi-objective Robust Strategy Synthesis for Interval Markov Decision Processes
Interval Markov decision processes (IMDPs) generalise classical MDPs by
having interval-valued transition probabilities. They provide a powerful
modelling tool for probabilistic systems with an additional variation or
uncertainty that prevents the knowledge of the exact transition probabilities.
In this paper, we consider the problem of multi-objective robust strategy
synthesis for interval MDPs, where the aim is to find a robust strategy that
guarantees the satisfaction of multiple properties at the same time in face of
the transition probability uncertainty. We first show that this problem is
PSPACE-hard. Then, we provide a value iteration-based decision algorithm to
approximate the Pareto set of achievable points. We finally demonstrate the
practical effectiveness of our proposed approaches by applying them on several
case studies using a prototypical tool.Comment: This article is a full version of a paper accepted to the Conference
on Quantitative Evaluation of SysTems (QEST) 201
Parameter Synthesis for Markov Models
Markov chain analysis is a key technique in reliability engineering. A
practical obstacle is that all probabilities in Markov models need to be known.
However, system quantities such as failure rates or packet loss ratios, etc.
are often not---or only partially---known. This motivates considering
parametric models with transitions labeled with functions over parameters.
Whereas traditional Markov chain analysis evaluates a reliability metric for a
single, fixed set of probabilities, analysing parametric Markov models focuses
on synthesising parameter values that establish a given reliability or
performance specification . Examples are: what component failure rates
ensure the probability of a system breakdown to be below 0.00000001?, or which
failure rates maximise reliability? This paper presents various analysis
algorithms for parametric Markov chains and Markov decision processes. We focus
on three problems: (a) do all parameter values within a given region satisfy
?, (b) which regions satisfy and which ones do not?, and (c)
an approximate version of (b) focusing on covering a large fraction of all
possible parameter values. We give a detailed account of the various
algorithms, present a software tool realising these techniques, and report on
an extensive experimental evaluation on benchmarks that span a wide range of
applications.Comment: 38 page
Verification and control of partially observable probabilistic systems
We present automated techniques for the verification and control of partially observable, probabilistic systems for both discrete and dense models of time. For the discrete-time case, we formally model these systems using partially observable Markov decision processes; for dense time, we propose an extension of probabilistic timed automata in which local states are partially visible to an observer or controller. We give probabilistic temporal logics that can express a range of quantitative properties of these models, relating to the probability of an event’s occurrence or the expected value of a reward measure. We then propose techniques to either verify that such a property holds or synthesise a controller for the model which makes it true. Our approach is based on a grid-based abstraction of the uncountable belief space induced by partial observability and, for dense-time models, an integer discretisation of real-time behaviour. The former is necessarily approximate since the underlying problem is undecidable, however we show how both lower and upper bounds on numerical results can be generated. We illustrate the effectiveness of the approach by implementing it in the PRISM model checker and applying it to several case studies from the domains of task and network scheduling, computer security and planning
Parametric LTL on Markov Chains
This paper is concerned with the verification of finite Markov chains against
parametrized LTL (pLTL) formulas. In pLTL, the until-modality is equipped with
a bound that contains variables; e.g., asserts that
holds within time steps, where is a variable on natural
numbers. The central problem studied in this paper is to determine the set of
parameter valuations for which the probability to
satisfy pLTL-formula in a Markov chain meets a given threshold , where is a comparison on reals and a probability. As for pLTL
determining the emptiness of is undecidable, we consider
several logic fragments. We consider parametric reachability properties, a
sub-logic of pLTL restricted to next and , parametric B\"uchi
properties and finally, a maximal subclass of pLTL for which emptiness of is decidable.Comment: TCS Track B 201
- …