332 research outputs found
Model Checking Probabilistic Pushdown Automata
We consider the model checking problem for probabilistic pushdown automata
(pPDA) and properties expressible in various probabilistic logics. We start
with properties that can be formulated as instances of a generalized random
walk problem. We prove that both qualitative and quantitative model checking
for this class of properties and pPDA is decidable. Then we show that model
checking for the qualitative fragment of the logic PCTL and pPDA is also
decidable. Moreover, we develop an error-tolerant model checking algorithm for
PCTL and the subclass of stateless pPDA. Finally, we consider the class of
omega-regular properties and show that both qualitative and quantitative model
checking for pPDA is decidable
Tableaux for Policy Synthesis for MDPs with PCTL* Constraints
Markov decision processes (MDPs) are the standard formalism for modelling
sequential decision making in stochastic environments. Policy synthesis
addresses the problem of how to control or limit the decisions an agent makes
so that a given specification is met. In this paper we consider PCTL*, the
probabilistic counterpart of CTL*, as the specification language. Because in
general the policy synthesis problem for PCTL* is undecidable, we restrict to
policies whose execution history memory is finitely bounded a priori.
Surprisingly, no algorithm for policy synthesis for this natural and
expressive framework has been developed so far. We close this gap and describe
a tableau-based algorithm that, given an MDP and a PCTL* specification, derives
in a non-deterministic way a system of (possibly nonlinear) equalities and
inequalities. The solutions of this system, if any, describe the desired
(stochastic) policies.
Our main result in this paper is the correctness of our method, i.e.,
soundness, completeness and termination.Comment: This is a long version of a conference paper published at TABLEAUX
2017. It contains proofs of the main results and fixes a bug. See the
footnote on page 1 for detail
Polynomial Time Algorithms for Branching Markov Decision Processes and Probabilistic Min(Max) Polynomial Bellman Equations
We show that one can approximate the least fixed point solution for a
multivariate system of monotone probabilistic max(min) polynomial equations,
referred to as maxPPSs (and minPPSs, respectively), in time polynomial in both
the encoding size of the system of equations and in log(1/epsilon), where
epsilon > 0 is the desired additive error bound of the solution. (The model of
computation is the standard Turing machine model.) We establish this result
using a generalization of Newton's method which applies to maxPPSs and minPPSs,
even though the underlying functions are only piecewise-differentiable. This
generalizes our recent work which provided a P-time algorithm for purely
probabilistic PPSs.
These equations form the Bellman optimality equations for several important
classes of infinite-state Markov Decision Processes (MDPs). Thus, as a
corollary, we obtain the first polynomial time algorithms for computing to
within arbitrary desired precision the optimal value vector for several classes
of infinite-state MDPs which arise as extensions of classic, and heavily
studied, purely stochastic processes. These include both the problem of
maximizing and mininizing the termination (extinction) probability of
multi-type branching MDPs, stochastic context-free MDPs, and 1-exit Recursive
MDPs.
Furthermore, we also show that we can compute in P-time an epsilon-optimal
policy for both maximizing and minimizing branching, context-free, and
1-exit-Recursive MDPs, for any given desired epsilon > 0. This is despite the
fact that actually computing optimal strategies is Sqrt-Sum-hard and
PosSLP-hard in this setting.
We also derive, as an easy consequence of these results, an FNP upper bound
on the complexity of computing the value (within arbitrary desired precision)
of branching simple stochastic games (BSSGs)
Reachability in Parametric Interval Markov Chains using Constraints
Parametric Interval Markov Chains (pIMCs) are a specification formalism that
extend Markov Chains (MCs) and Interval Markov Chains (IMCs) by taking into
account imprecision in the transition probability values: transitions in pIMCs
are labeled with parametric intervals of probabilities. In this work, we study
the difference between pIMCs and other Markov Chain abstractions models and
investigate the two usual semantics for IMCs: once-and-for-all and
at-every-step. In particular, we prove that both semantics agree on the
maximal/minimal reachability probabilities of a given IMC. We then investigate
solutions to several parameter synthesis problems in the context of pIMCs --
consistency, qualitative reachability and quantitative reachability -- that
rely on constraint encodings. Finally, we propose a prototype implementation of
our constraint encodings with promising results
On finitely ambiguous B\"uchi automata
Unambiguous B\"uchi automata, i.e. B\"uchi automata allowing only one
accepting run per word, are a useful restriction of B\"uchi automata that is
well-suited for probabilistic model-checking. In this paper we propose a more
permissive variant, namely finitely ambiguous B\"uchi automata, a
generalisation where each word has at most accepting runs, for some fixed
. We adapt existing notions and results concerning finite and bounded
ambiguity of finite automata to the setting of -languages and present a
translation from arbitrary nondeterministic B\"uchi automata with states to
finitely ambiguous automata with at most states and at most accepting
runs per word
Quantitative multi-objective verification for probabilistic systems
We present a verification framework for analysing multiple quantitative objectives of systems that exhibit both nondeterministic and stochastic behaviour. These systems are modelled as probabilistic automata, enriched with cost or reward structures that capture, for example, energy usage or performance metrics. Quantitative properties of these models are expressed in a specification language that incorporates probabilistic safety and liveness properties, expected total cost or reward, and supports multiple objectives of these types. We propose and implement an efficient verification framework for such properties and then present two distinct applications of it: firstly, controller synthesis subject to multiple quantitative objectives; and, secondly, quantitative compositional verification. The practical applicability of both approaches is illustrated with experimental results from several large case studies
Decision Problems for Nash Equilibria in Stochastic Games
We analyse the computational complexity of finding Nash equilibria in
stochastic multiplayer games with -regular objectives. While the
existence of an equilibrium whose payoff falls into a certain interval may be
undecidable, we single out several decidable restrictions of the problem.
First, restricting the search space to stationary, or pure stationary,
equilibria results in problems that are typically contained in PSPACE and NP,
respectively. Second, we show that the existence of an equilibrium with a
binary payoff (i.e. an equilibrium where each player either wins or loses with
probability 1) is decidable. We also establish that the existence of a Nash
equilibrium with a certain binary payoff entails the existence of an
equilibrium with the same payoff in pure, finite-state strategies.Comment: 22 pages, revised versio
From LTL and Limit-Deterministic B\"uchi Automata to Deterministic Parity Automata
Controller synthesis for general linear temporal logic (LTL) objectives is a
challenging task. The standard approach involves translating the LTL objective
into a deterministic parity automaton (DPA) by means of the Safra-Piterman
construction. One of the challenges is the size of the DPA, which often grows
very fast in practice, and can reach double exponential size in the length of
the LTL formula. In this paper we describe a single exponential translation
from limit-deterministic B\"uchi automata (LDBA) to DPA, and show that it can
be concatenated with a recent efficient translation from LTL to LDBA to yield a
double exponential, \enquote{Safraless} LTL-to-DPA construction. We also report
on an implementation, a comparison with the SPOT library, and performance on
several sets of formulas, including instances from the 2016 SyntComp
competition
Computing Quantiles in Markov Reward Models
Probabilistic model checking mainly concentrates on techniques for reasoning
about the probabilities of certain path properties or expected values of
certain random variables. For the quantitative system analysis, however, there
is also another type of interesting performance measure, namely quantiles. A
typical quantile query takes as input a lower probability bound p and a
reachability property. The task is then to compute the minimal reward bound r
such that with probability at least p the target set will be reached before the
accumulated reward exceeds r. Quantiles are well-known from mathematical
statistics, but to the best of our knowledge they have not been addressed by
the model checking community so far.
In this paper, we study the complexity of quantile queries for until
properties in discrete-time finite-state Markov decision processes with
non-negative rewards on states. We show that qualitative quantile queries can
be evaluated in polynomial time and present an exponential algorithm for the
evaluation of quantitative quantile queries. For the special case of Markov
chains, we show that quantitative quantile queries can be evaluated in time
polynomial in the size of the chain and the maximum reward.Comment: 17 pages, 1 figure; typo in example correcte
- …