11 research outputs found
What is known about the Value 1 Problem for Probabilistic Automata?
The value 1 problem is a decision problem for probabilistic automata over
finite words: are there words accepted by the automaton with arbitrarily high
probability? Although undecidable, this problem attracted a lot of attention
over the last few years. The aim of this paper is to review and relate the
results pertaining to the value 1 problem. In particular, several algorithms
have been proposed to partially solve this problem. We show the relations
between them, leading to the following conclusion: the Markov Monoid Algorithm
is the most correct algorithm known to (partially) solve the value 1 problem
The Complexity of POMDPs with Long-run Average Objectives
We study the problem of approximation of optimal values in
partially-observable Markov decision processes (POMDPs) with long-run average
objectives. POMDPs are a standard model for dynamic systems with probabilistic
and nondeterministic behavior in uncertain environments. In long-run average
objectives rewards are associated with every transition of the POMDP and the
payoff is the long-run average of the rewards along the executions of the
POMDP. We establish strategy complexity and computational complexity results.
Our main result shows that finite-memory strategies suffice for approximation
of optimal values, and the related decision problem is recursively enumerable
complete
Profinite Techniques for Probabilistic Automata and the Markov Monoid Algorithm
We consider the value 1 problem for probabilistic automata over finite words:
it asks whether a given probabilistic automaton accepts words with probability
arbitrarily close to 1. This problem is known to be undecidable. However,
different algorithms have been proposed to partially solve it; it has been
recently shown that the Markov Monoid algorithm, based on algebra, is the most
correct algorithm so far. The first contribution of this paper is to give a
characterisation of the Markov Monoid algorithm. The second contribution is to
develop a profinite theory for probabilistic automata, called the prostochastic
theory. This new framework gives a topological account of the value 1 problem,
which in this context is cast as an emptiness problem. The above
characterisation is reformulated using the prostochastic theory, allowing us to
give a simple and modular proof.Comment: Conference version: STACS'2016, Symposium on Theoretical Aspects of
Computer Science Journal version: TCS'2017, Theoretical Computer Scienc
IST Austria Technical Report
We consider probabilistic automata on infinite words with acceptance defined by parity conditions. We consider three qualitative decision problems: (i) the positive decision problem asks whether there is a word that is accepted with positive probability; (ii) the almost decision problem asks whether there is a word that is accepted with probability 1; and (iii) the limit decision problem asks whether for every ε > 0 there is a word that is accepted with probability at least 1 − ε. We unify and generalize several decidability results for probabilistic automata over infinite words, and identify a robust (closed under union and intersection) subclass of probabilistic automata for which all the qualitative decision problems are decidable for parity conditions. We also show that if the input words are restricted to lasso shape words, then the positive and almost problems are decidable for all probabilistic automata with parity conditions
Qualitative Analysis of Concurrent Mean-payoff Games
We consider concurrent games played by two-players on a finite-state graph,
where in every round the players simultaneously choose a move, and the current
state along with the joint moves determine the successor state. We study a
fundamental objective, namely, mean-payoff objective, where a reward is
associated to each transition, and the goal of player 1 is to maximize the
long-run average of the rewards, and the objective of player 2 is strictly the
opposite. The path constraint for player 1 could be qualitative, i.e., the
mean-payoff is the maximal reward, or arbitrarily close to it; or quantitative,
i.e., a given threshold between the minimal and maximal reward. We consider the
computation of the almost-sure (resp. positive) winning sets, where player 1
can ensure that the path constraint is satisfied with probability 1 (resp.
positive probability). Our main results for qualitative path constraints are as
follows: (1) we establish qualitative determinacy results that show that for
every state either player 1 has a strategy to ensure almost-sure (resp.
positive) winning against all player-2 strategies, or player 2 has a spoiling
strategy to falsify almost-sure (resp. positive) winning against all player-1
strategies; (2) we present optimal strategy complexity results that precisely
characterize the classes of strategies required for almost-sure and positive
winning for both players; and (3) we present quadratic time algorithms to
compute the almost-sure and the positive winning sets, matching the best known
bound of algorithms for much simpler problems (such as reachability
objectives). For quantitative constraints we show that a polynomial time
solution for the almost-sure or the positive winning set would imply a solution
to a long-standing open problem (the value problem for turn-based deterministic
mean-payoff games) that is not known to be solvable in polynomial time
Deciding the value 1 problem for probabilistic leaktight automata
The value 1 problem is a decision problem for probabilistic automata over
finite words: given a probabilistic automaton, are there words accepted with
probability arbitrarily close to 1? This problem was proved undecidable
recently; to overcome this, several classes of probabilistic automata of
different nature were proposed, for which the value 1 problem has been shown
decidable. In this paper, we introduce yet another class of probabilistic
automata, called leaktight automata, which strictly subsumes all classes of
probabilistic automata whose value 1 problem is known to be decidable. We prove
that for leaktight automata, the value 1 problem is decidable (in fact,
PSPACE-complete) by constructing a saturation algorithm based on the
computation of a monoid abstracting the behaviours of the automaton. We rely on
algebraic techniques developed by Simon to prove that this abstraction is
complete. Furthermore, we adapt this saturation algorithm to decide whether an
automaton is leaktight. Finally, we show a reduction allowing to extend our
decidability results from finite words to infinite ones, implying that the
value 1 problem for probabilistic leaktight parity automata is decidable
IST Austria Technical Report
Evolution occurs in populations of reproducing individuals. The structure of the population affects the outcome of the evolutionary process. Evolutionary graph theory is a powerful approach to study this phenomenon. There are two graphs. The interaction graph specifies who interacts with whom in the context of evolution.The replacement graph specifies who competes with whom for reproduction.
The vertices of the two graphs are the same, and each vertex corresponds to an individual of the population. A key quantity is the fixation probability of a new mutant. It is defined as the probability that a newly introduced mutant (on a single vertex) generates a lineage of offspring which eventually takes over the entire population of resident individuals. The basic computational questions are as follows: (i) the qualitative question asks whether the fixation probability is positive; and (ii) the quantitative approximation question asks for an approximation of the fixation probability.
Our main results are:
(1) We show that the qualitative question is NP-complete and the quantitative approximation question is #P-hard in the special case when the interaction and the replacement graphs coincide and even with the restriction that the resident individuals do not reproduce (which corresponds to an invading population taking over an empty structure).
(2) We show that in general the qualitative question is PSPACE-complete and the quantitative approximation question is PSPACE-hard and can be solved in exponential time
The Value 1 Problem Under Finite-memory Strategies for Concurrent Mean-payoff Games
We consider concurrent mean-payoff games, a very well-studied class of
two-player (player 1 vs player 2) zero-sum games on finite-state graphs where
every transition is assigned a reward between 0 and 1, and the payoff function
is the long-run average of the rewards. The value is the maximal expected
payoff that player 1 can guarantee against all strategies of player 2. We
consider the computation of the set of states with value 1 under finite-memory
strategies for player 1, and our main results for the problem are as follows:
(1) we present a polynomial-time algorithm; (2) we show that whenever there is
a finite-memory strategy, there is a stationary strategy that does not need
memory at all; and (3) we present an optimal bound (which is double
exponential) on the patience of stationary strategies (where patience of a
distribution is the inverse of the smallest positive probability and represents
a complexity measure of a stationary strategy)
IST Austria Technical Report
We consider concurrent mean-payoff games, a very well-studied class of two-player (player 1 vs player 2) zero-sum games on finite-state graphs where every transition is assigned a reward between 0 and 1, and the payoff function is the long-run average of the rewards. The value is the maximal expected payoff that player 1 can guarantee against all strategies of player 2. We consider the computation of the set of states with value 1 under finite-memory strategies for player 1, and our main results for the problem are as follows: (1) we present a polynomial-time algorithm; (2) we show that whenever there is a finite-memory strategy, there is a stationary strategy that does not need memory at all; and (3) we present an optimal bound (which is double exponential) on the patience of stationary strategies (where patience of a distribution is the inverse of the smallest positive probability and represents a complexity measure of a stationary strategy)