5,218 research outputs found
Algorithms and Conditional Lower Bounds for Planning Problems
We consider planning problems for graphs, Markov decision processes (MDPs),
and games on graphs. While graphs represent the most basic planning model, MDPs
represent interaction with nature and games on graphs represent interaction
with an adversarial environment. We consider two planning problems where there
are k different target sets, and the problems are as follows: (a) the coverage
problem asks whether there is a plan for each individual target set, and (b)
the sequential target reachability problem asks whether the targets can be
reached in sequence. For the coverage problem, we present a linear-time
algorithm for graphs and quadratic conditional lower bound for MDPs and games
on graphs. For the sequential target problem, we present a linear-time
algorithm for graphs, a sub-quadratic algorithm for MDPs, and a quadratic
conditional lower bound for games on graphs. Our results with conditional lower
bounds establish (i) model-separation results showing that for the coverage
problem MDPs and games on graphs are harder than graphs and for the sequential
reachability problem games on graphs are harder than MDPs and graphs; (ii)
objective-separation results showing that for MDPs the coverage problem is
harder than the sequential target problem.Comment: Accepted at ICAPS'1
Termination Criteria for Solving Concurrent Safety and Reachability Games
We consider concurrent games played on graphs. At every round of a game, each
player simultaneously and independently selects a move; the moves jointly
determine the transition to a successor state. Two basic objectives are the
safety objective to stay forever in a given set of states, and its dual, the
reachability objective to reach a given set of states. We present in this paper
a strategy improvement algorithm for computing the value of a concurrent safety
game, that is, the maximal probability with which player~1 can enforce the
safety objective. The algorithm yields a sequence of player-1 strategies which
ensure probabilities of winning that converge monotonically to the value of the
safety game.
Our result is significant because the strategy improvement algorithm
provides, for the first time, a way to approximate the value of a concurrent
safety game from below. Since a value iteration algorithm, or a strategy
improvement algorithm for reachability games, can be used to approximate the
same value from above, the combination of both algorithms yields a method for
computing a converging sequence of upper and lower bounds for the values of
concurrent reachability and safety games. Previous methods could approximate
the values of these games only from one direction, and as no rates of
convergence are known, they did not provide a practical way to solve these
games
Reconfiguration in bounded bandwidth and treedepth
We show that several reconfiguration problems known to be PSPACE-complete
remain so even when limited to graphs of bounded bandwidth. The essential step
is noticing the similarity to very limited string rewriting systems, whose
ability to directly simulate Turing Machines is classically known. This
resolves a question posed open in [Bonsma P., 2012]. On the other hand, we show
that a large class of reconfiguration problems becomes tractable on graphs of
bounded treedepth, and that this result is in some sense tight.Comment: 14 page
Discounting in Games across Time Scales
We introduce two-level discounted games played by two players on a
perfect-information stochastic game graph. The upper level game is a discounted
game and the lower level game is an undiscounted reachability game. Two-level
games model hierarchical and sequential decision making under uncertainty
across different time scales. We show the existence of pure memoryless optimal
strategies for both players and an ordered field property for such games. We
show that if there is only one player (Markov decision processes), then the
values can be computed in polynomial time. It follows that whether the value of
a player is equal to a given rational constant in two-level discounted games
can be decided in NP intersected coNP. We also give an alternate strategy
improvement algorithm to compute the value
Quantification of reachable attractors in asynchronous discrete dynamics
Motivation: Models of discrete concurrent systems often lead to huge and
complex state transition graphs that represent their dynamics. This makes
difficult to analyse dynamical properties. In particular, for logical models of
biological regulatory networks, it is of real interest to study attractors and
their reachability from specific initial conditions, i.e. to assess the
potential asymptotical behaviours of the system. Beyond the identification of
the reachable attractors, we propose to quantify this reachability.
Results: Relying on the structure of the state transition graph, we estimate
the probability of each attractor reachable from a given initial condition or
from a portion of the state space. First, we present a quasi-exact solution
with an original algorithm called Firefront, based on the exhaustive
exploration of the reachable state space. Then, we introduce an adapted version
of Monte Carlo simulation algorithm, termed Avatar, better suited to larger
models. Firefront and Avatar methods are validated and compared to other
related approaches, using as test cases logical models of synthetic and
biological networks.
Availability: Both algorithms are implemented as Perl scripts that can be
freely downloaded from http://compbio.igc.gulbenkian.pt/nmd/node/59 along with
Supplementary Material.Comment: 19 pages, 2 figures, 2 algorithms and 2 table
Average Case Analysis of the Classical Algorithm for Markov Decision Processes with B\"uchi Objectives
We consider Markov decision processes (MDPs) with -regular
specifications given as parity objectives. We consider the problem of computing
the set of almost-sure winning vertices from where the objective can be ensured
with probability 1. The algorithms for the computation of the almost-sure
winning set for parity objectives iteratively use the solutions for the
almost-sure winning set for B\"uchi objectives (a special case of parity
objectives). We study for the first time the average case complexity of the
classical algorithm for computing almost-sure winning vertices for MDPs with
B\"uchi objectives. Our contributions are as follows: First, we show that for
MDPs with constant out-degree the expected number of iterations is at most
logarithmic and the average case running time is linear (as compared to the
worst case linear number of iterations and quadratic time complexity). Second,
we show that for general MDPs the expected number of iterations is constant and
the average case running time is linear (again as compared to the worst case
linear number of iterations and quadratic time complexity). Finally we also
show that given all graphs are equally likely, the probability that the
classical algorithm requires more than constant number of iterations is
exponentially small
Optimal Strategies in Infinite-state Stochastic Reachability Games
We consider perfect-information reachability stochastic games for 2 players
on infinite graphs. We identify a subclass of such games, and prove two
interesting properties of it: first, Player Max always has optimal strategies
in games from this subclass, and second, these games are strongly determined.
The subclass is defined by the property that the set of all values can only
have one accumulation point -- 0. Our results nicely mirror recent results for
finitely-branching games, where, on the contrary, Player Min always has optimal
strategies. However, our proof methods are substantially different, because the
roles of the players are not symmetric. We also do not restrict the branching
of the games. Finally, we apply our results in the context of recently studied
One-Counter stochastic games
- …