1,626 research outputs found
Probabilistic Reachability for Parametric Markov Models
Abstract. Given a parametric Markov model, we consider the problem of computing the formula expressing the probability of reaching a given set of states. To attack this principal problem, Daws has suggested to first convert the Markov chain into a finite automaton, from which a regular expression is computed. Afterwards, this expression is evaluated to a closed form expression representing the reachability probability. This paper investigates how this idea can be turned into an effective procedure. It turns out that the bottleneck lies in an exponential growth of the regular expression relative to the number of states. We therefore proceed differently, by tightly intertwining the regular expression computation with its evaluation. This allows us to arrive at an effective method that avoids the exponential blow up in most practical cases. We give a detailed account of the approach, also extending to parametric models with rewards and with non-determinism. Experimental evidence is provided, illustrating that our implementation provides meaningful insights on non-trivial models.
Probabilistic Reachability for Parametric Markov Models
Abstract. Given a parametric Markov model, we consider the problem of computing the rational function expressing the probability of reaching a given set of states. To attack this principal problem, Daws has suggested to first convert the Markov chain into a finite automaton, from which a regular expression is computed. Afterwards, this expression is evaluated to a closed form function representing the reachability probability. This paper investigates how this idea can be turned into an effective procedure. It turns out that the bottleneck lies in the growth of the regular expression relative to the number of states (nΘ(logn)). We therefore proceed differently, by tightly intertwining the regular expression computation with its evaluation. This allows us to arrive at an effective method that avoids this blow up in most practical cases. We give a detailed account of the approach, also extending to parametric models with rewards and with non-determinism. Experimental evidence is provided, illustrating that our implementation provides meaningful insights on non-trivial models.
Parameter-Independent Strategies for pMDPs via POMDPs
Markov Decision Processes (MDPs) are a popular class of models suitable for
solving control decision problems in probabilistic reactive systems. We
consider parametric MDPs (pMDPs) that include parameters in some of the
transition probabilities to account for stochastic uncertainties of the
environment such as noise or input disturbances.
We study pMDPs with reachability objectives where the parameter values are
unknown and impossible to measure directly during execution, but there is a
probability distribution known over the parameter values. We study for the
first time computing parameter-independent strategies that are expectation
optimal, i.e., optimize the expected reachability probability under the
probability distribution over the parameters. We present an encoding of our
problem to partially observable MDPs (POMDPs), i.e., a reduction of our problem
to computing optimal strategies in POMDPs.
We evaluate our method experimentally on several benchmarks: a motivating
(repeated) learner model; a series of benchmarks of varying configurations of a
robot moving on a grid; and a consensus protocol.Comment: Extended version of a QEST 2018 pape
Bounded Model Checking for Probabilistic Programs
In this paper we investigate the applicability of standard model checking
approaches to verifying properties in probabilistic programming. As the
operational model for a standard probabilistic program is a potentially
infinite parametric Markov decision process, no direct adaption of existing
techniques is possible. Therefore, we propose an on-the-fly approach where the
operational model is successively created and verified via a step-wise
execution of the program. This approach enables to take key features of many
probabilistic programs into account: nondeterminism and conditioning. We
discuss the restrictions and demonstrate the scalability on several benchmarks
Reachability in Parametric Interval Markov Chains using Constraints
Parametric Interval Markov Chains (pIMCs) are a specification formalism that
extend Markov Chains (MCs) and Interval Markov Chains (IMCs) by taking into
account imprecision in the transition probability values: transitions in pIMCs
are labeled with parametric intervals of probabilities. In this work, we study
the difference between pIMCs and other Markov Chain abstractions models and
investigate the two usual semantics for IMCs: once-and-for-all and
at-every-step. In particular, we prove that both semantics agree on the
maximal/minimal reachability probabilities of a given IMC. We then investigate
solutions to several parameter synthesis problems in the context of pIMCs --
consistency, qualitative reachability and quantitative reachability -- that
rely on constraint encodings. Finally, we propose a prototype implementation of
our constraint encodings with promising results
Accelerating Parametric Probabilistic Verification
We present a novel method for computing reachability probabilities of
parametric discrete-time Markov chains whose transition probabilities are
fractions of polynomials over a set of parameters. Our algorithm is based on
two key ingredients: a graph decomposition into strongly connected subgraphs
combined with a novel factorization strategy for polynomials. Experimental
evaluations show that these approaches can lead to a speed-up of up to several
orders of magnitude in comparison to existing approache
Parametric LTL on Markov Chains
This paper is concerned with the verification of finite Markov chains against
parametrized LTL (pLTL) formulas. In pLTL, the until-modality is equipped with
a bound that contains variables; e.g., asserts that
holds within time steps, where is a variable on natural
numbers. The central problem studied in this paper is to determine the set of
parameter valuations for which the probability to
satisfy pLTL-formula in a Markov chain meets a given threshold , where is a comparison on reals and a probability. As for pLTL
determining the emptiness of is undecidable, we consider
several logic fragments. We consider parametric reachability properties, a
sub-logic of pLTL restricted to next and , parametric B\"uchi
properties and finally, a maximal subclass of pLTL for which emptiness of is decidable.Comment: TCS Track B 201
- …