16 research outputs found
Qualitative Reachability in Stochastic BPA Games
We consider a class of infinite-state stochastic games generated by stateless
pushdown automata (or, equivalently, 1-exit recursive state machines), where
the winning objective is specified by a regular set of target configurations
and a qualitative probability constraint `>0' or `=1'. The goal of one player
is to maximize the probability of reaching the target set so that the
constraint is satisfied, while the other player aims at the opposite. We show
that the winner in such games can be determined in PTIME for the `>0'
constraint, and both in NP and coNP for the `=1' constraint. Further, we prove
that the winning regions for both players are regular, and we design algorithms
which compute the associated finite-state automata. Finally, we show that
winning strategies can be synthesized effectively.Comment: Submitted to Information and Computation. 48 pages, 3 figure
Optimal Strategies in Infinite-state Stochastic Reachability Games
We consider perfect-information reachability stochastic games for 2 players
on infinite graphs. We identify a subclass of such games, and prove two
interesting properties of it: first, Player Max always has optimal strategies
in games from this subclass, and second, these games are strongly determined.
The subclass is defined by the property that the set of all values can only
have one accumulation point -- 0. Our results nicely mirror recent results for
finitely-branching games, where, on the contrary, Player Min always has optimal
strategies. However, our proof methods are substantially different, because the
roles of the players are not symmetric. We also do not restrict the branching
of the games. Finally, we apply our results in the context of recently studied
One-Counter stochastic games
Solving Stochastic B\"uchi Games on Infinite Arenas with a Finite Attractor
We consider games played on an infinite probabilistic arena where the first
player aims at satisfying generalized B\"uchi objectives almost surely, i.e.,
with probability one. We provide a fixpoint characterization of the winning
sets and associated winning strategies in the case where the arena satisfies
the finite-attractor property. From this we directly deduce the decidability of
these games on probabilistic lossy channel systems.Comment: In Proceedings QAPL 2013, arXiv:1306.241
Approximating the Termination Value of One-Counter MDPs and Stochastic Games
One-counter MDPs (OC-MDPs) and one-counter simple stochastic games (OC-SSGs)
are 1-player, and 2-player turn-based zero-sum, stochastic games played on the
transition graph of classic one-counter automata (equivalently, pushdown
automata with a 1-letter stack alphabet). A key objective for the analysis and
verification of these games is the termination objective, where the players aim
to maximize (minimize, respectively) the probability of hitting counter value
0, starting at a given control state and given counter value. Recently, we
studied qualitative decision problems ("is the optimal termination value = 1?")
for OC-MDPs (and OC-SSGs) and showed them to be decidable in P-time (in NP and
coNP, respectively). However, quantitative decision and approximation problems
("is the optimal termination value ? p", or "approximate the termination value
within epsilon") are far more challenging. This is so in part because optimal
strategies may not exist, and because even when they do exist they can have a
highly non-trivial structure. It thus remained open even whether any of these
quantitative termination problems are computable. In this paper we show that
all quantitative approximation problems for the termination value for OC-MDPs
and OC-SSGs are computable. Specifically, given a OC-SSG, and given epsilon >
0, we can compute a value v that approximates the value of the OC-SSG
termination game within additive error epsilon, and furthermore we can compute
epsilon-optimal strategies for both players in the game. A key ingredient in
our proofs is a subtle martingale, derived from solving certain LPs that we can
associate with a maximizing OC-MDP. An application of Azuma's inequality on
these martingales yields a computable bound for the "wealth" at which a "rich
person's strategy" becomes epsilon-optimal for OC-MDPs.Comment: 35 pages, 1 figure, full version of a paper presented at ICALP 2011,
invited for submission to Information and Computatio
Taming denumerable Markov decision processes with decisiveness
Decisiveness has proven to be an elegant concept for denumerable Markov
chains: it is general enough to encompass several natural classes of
denumerable Markov chains, and is a sufficient condition for simple qualitative
and approximate quantitative model checking algorithms to exist. In this paper,
we explore how to extend the notion of decisiveness to Markov decision
processes. Compared to Markov chains, the extra non-determinism can be resolved
in an adversarial or cooperative way, yielding two natural notions of
decisiveness. We then explore whether these notions yield model checking
procedures concerning the infimum and supremum probabilities of reachability
properties
Qualitative Analysis of Concurrent Mean-payoff Games
We consider concurrent games played by two-players on a finite-state graph,
where in every round the players simultaneously choose a move, and the current
state along with the joint moves determine the successor state. We study a
fundamental objective, namely, mean-payoff objective, where a reward is
associated to each transition, and the goal of player 1 is to maximize the
long-run average of the rewards, and the objective of player 2 is strictly the
opposite. The path constraint for player 1 could be qualitative, i.e., the
mean-payoff is the maximal reward, or arbitrarily close to it; or quantitative,
i.e., a given threshold between the minimal and maximal reward. We consider the
computation of the almost-sure (resp. positive) winning sets, where player 1
can ensure that the path constraint is satisfied with probability 1 (resp.
positive probability). Our main results for qualitative path constraints are as
follows: (1) we establish qualitative determinacy results that show that for
every state either player 1 has a strategy to ensure almost-sure (resp.
positive) winning against all player-2 strategies, or player 2 has a spoiling
strategy to falsify almost-sure (resp. positive) winning against all player-1
strategies; (2) we present optimal strategy complexity results that precisely
characterize the classes of strategies required for almost-sure and positive
winning for both players; and (3) we present quadratic time algorithms to
compute the almost-sure and the positive winning sets, matching the best known
bound of algorithms for much simpler problems (such as reachability
objectives). For quantitative constraints we show that a polynomial time
solution for the almost-sure or the positive winning set would imply a solution
to a long-standing open problem (the value problem for turn-based deterministic
mean-payoff games) that is not known to be solvable in polynomial time
Probabilistic modal {\mu}-calculus with independent product
The probabilistic modal {\mu}-calculus is a fixed-point logic designed for
expressing properties of probabilistic labeled transition systems (PLTS's). Two
equivalent semantics have been studied for this logic, both assigning to each
state a value in the interval [0,1] representing the probability that the
property expressed by the formula holds at the state. One semantics is
denotational and the other is a game semantics, specified in terms of
two-player stochastic parity games. A shortcoming of the probabilistic modal
{\mu}-calculus is the lack of expressiveness required to encode other important
temporal logics for PLTS's such as Probabilistic Computation Tree Logic (PCTL).
To address this limitation we extend the logic with a new pair of operators:
independent product and coproduct. The resulting logic, called probabilistic
modal {\mu}-calculus with independent product, can encode many properties of
interest and subsumes the qualitative fragment of PCTL. The main contribution
of this paper is the definition of an appropriate game semantics for this
extended probabilistic {\mu}-calculus. This relies on the definition of a new
class of games which generalize standard two-player stochastic (parity) games
by allowing a play to be split into concurrent subplays, each continuing their
evolution independently. Our main technical result is the equivalence of the
two semantics. The proof is carried out in ZFC set theory extended with
Martin's Axiom at an uncountable cardinal
Hyperplane Separation Technique for Multidimensional Mean-Payoff Games
We consider both finite-state game graphs and recursive game graphs (or
pushdown game graphs), that can model the control flow of sequential programs
with recursion, with multi-dimensional mean-payoff objectives. In pushdown
games two types of strategies are relevant: global strategies, that depend on
the entire global history; and modular strategies, that have only local memory
and thus do not depend on the context of invocation. We present solutions to
several fundamental algorithmic questions and our main contributions are as
follows: (1) We show that finite-state multi-dimensional mean-payoff games can
be solved in polynomial time if the number of dimensions and the maximal
absolute value of the weight is fixed; whereas if the number of dimensions is
arbitrary, then problem is already known to be coNP-complete. (2) We show that
pushdown graphs with multi-dimensional mean-payoff objectives can be solved in
polynomial time. (3) For pushdown games under global strategies both single and
multi-dimensional mean-payoff objectives problems are known to be undecidable,
and we show that under modular strategies the multi-dimensional problem is also
undecidable (whereas under modular strategies the single dimensional problem is
NP-complete). We show that if the number of modules, the number of exits, and
the maximal absolute value of the weight is fixed, then pushdown games under
modular strategies with single dimensional mean-payoff objectives can be solved
in polynomial time, and if either of the number of exits or the number of
modules is not bounded, then the problem is NP-hard. (4) Finally we show that a
fixed parameter tractable algorithm for finite-state multi-dimensional
mean-payoff games or pushdown games under modular strategies with
single-dimensional mean-payoff objectives would imply the solution of the
long-standing open problem of fixed parameter tractability of parity games.Comment: arXiv admin note: text overlap with arXiv:1201.282