Equilibria, Fixed Points, and Complexity Classes

Many models from a variety of areas involve the computation of an equilibrium or fixed point of some kind. Examples include Nash equilibria in games; market equilibria; computing optimal strategies and the values of competitive games (stochastic and other games); stable configurations of neural networks; analysing basic stochastic models for evolution like branching processes and for language like stochastic context-free grammars; and models that incorporate the basic primitives of probability and recursion like recursive Markov chains. It is not known whether these problems can be solved in polynomial time. There are certain common computational principles underlying different types of equilibria, which are captured by the complexity classes PLS, PPAD, and FIXP. Representative complete problems for these classes are respectively, pure Nash equilibria in games where they are guaranteed to exist, (mixed) Nash equilibria in 2-player normal form games, and (mixed) Nash equilibria in normal form games with 3 (or more) players. This paper reviews the underlying computational principles and the corresponding classes

Markovian binary trees subject to catastrophes: computation of the extinction probability

This thesis is focused on the study of the probability that a certain Markovian binary tree becomes extinct. Diefferently from most of the existing literature, this study has been conducted through a matrix approach instead of probabilistic. A Markovian binary tree (MBT) is a model suitable for processes of evolution involving a population of individuals, wherein each of them evolves independently from the others. At every moment, the whole population can be partitioned into a finite number of typologies. During its life, an individual evolves and various events may happen, according to established probabilistic rules, in particular: - an individual can change its type and continue its life; - an individual can generate another individual and continue its life, the newborn individual will evolve independently from its parent; - an individual can die. An MBT is said to be extinct whenever there are no more individuals alive. In particular, in our analysis, we are interested in computing the probability that a certain tree will become extinct conditioned that the starting population consists of only one individual. This problem has been tackled with a renewed interest in the last years, since the vector of extinction probabilities was identied as the minimal nonnegative solution of a certain quadratic vector equation. Various algorithms have been proposed for computing such a solution, and a survey of them is reported in the thesis. In this work we deal with an extension of this model, in fact, beside the process of the population, we introduce a parallel independent process of catastrophes. When a catastrophe happens it influences every individual alive at the moment, who may die or survive with a probability depending on their current type. Also in this case we are interested in computing the extinction probability vector, however the challenge is much harder since the quadratic vector equation employed in the classical MBT case is not true anymore. In a recent work, Hautphenne et al. pointed out the importance of a parameter, whose positivity or negativity plays the discriminating role between the processes that will survive forever with a certain positive probability and those which will become extinct almost surely. The problem is that even only the computation of this parameter is beyond our possibilities. In fact, it is showed in the thesis how its computation of is equivalent to the computation of the maximal Lyapunov exponent of a random dynamical system, a problem that is well known to be hard apart from special cases. Hautphenne et al. proposed an upper and a lower bound for such a parameter by using probabilistic arguments. In this thesis, by using matrix properties, we provide different expressions for the value of the parameter and we derive some upper and lower bounds. We conclude the thesis by showing with numerical experiment when the various bounds are accurate or not. It becomes evident that the upper bound provided by Hautphenne et al. is still the best upper bound available, on the other hand the lower bound we suggested works better than the one available before

Polynomial Time Algorithms for Branching Markov Decision Processes and Probabilistic Min(Max) Polynomial Bellman Equations

We show that one can approximate the least fixed point solution for a multivariate system of monotone probabilistic max(min) polynomial equations, referred to as maxPPSs (and minPPSs, respectively), in time polynomial in both the encoding size of the system of equations and in log(1/epsilon), where epsilon > 0 is the desired additive error bound of the solution. (The model of computation is the standard Turing machine model.) We establish this result using a generalization of Newton's method which applies to maxPPSs and minPPSs, even though the underlying functions are only piecewise-differentiable. This generalizes our recent work which provided a P-time algorithm for purely probabilistic PPSs. These equations form the Bellman optimality equations for several important classes of infinite-state Markov Decision Processes (MDPs). Thus, as a corollary, we obtain the first polynomial time algorithms for computing to within arbitrary desired precision the optimal value vector for several classes of infinite-state MDPs which arise as extensions of classic, and heavily studied, purely stochastic processes. These include both the problem of maximizing and mininizing the termination (extinction) probability of multi-type branching MDPs, stochastic context-free MDPs, and 1-exit Recursive MDPs. Furthermore, we also show that we can compute in P-time an epsilon-optimal policy for both maximizing and minimizing branching, context-free, and 1-exit-Recursive MDPs, for any given desired epsilon > 0. This is despite the fact that actually computing optimal strategies is Sqrt-Sum-hard and PosSLP-hard in this setting. We also derive, as an easy consequence of these results, an FNP upper bound on the complexity of computing the value (within arbitrary desired precision) of branching simple stochastic games (BSSGs)

Reachability for Branching Concurrent Stochastic Games

We give polynomial time algorithms for deciding almost-sure and limit-sure reachability in Branching Concurrent Stochastic Games (BCSGs). These are a class of infinite-state imperfect-information stochastic games that generalize both finite-state concurrent stochastic reachability games ([L. de Alfaro et al., 2007]) and branching simple stochastic reachability games ([K. Etessami et al., 2018])

Greatest Fixed Points of Probabilistic Min/Max Polynomial Equations, and Reachability for Branching Markov Decision Processes?

We give polynomial time algorithms for quantitative (and qualitative) reachability analysis for Branching Markov Decision Processes (BMDPs). Specifically, given a BMDP, and given an initial population, where the objective of the controller is to maximize (or minimize) the probability of eventually reaching a population that contains an object of a desired (or undesired) type, we give algorithms for approximating the supremum (infimum) reachability probability, within desired precision epsilon > 0, in time polynomial in the encoding size of the BMDP and in log(1/epsilon). We furthermore give P-time algorithms for computing epsilon-optimal strategies for both maximization and minimization of reachability probabilities. We also give P-time algorithms for all associated qualitative analysis problems, namely: deciding whether the optimal (supremum or infimum) reachability probabilities are 0 or 1. Prior to this paper, approximation of optimal reachability probabilities for BMDPs was not even known to be decidable. Our algorithms exploit the following basic fact: we show that for any BMDP, its maximum (minimum) non-reachability probabilities are given by the greatest fixed point (GFP) solution g* in [0,1]^n of a corresponding monotone max (min) Probabilistic Polynomial System of equations (max/min-PPS), x=P(x), which are the Bellman optimality equations for a BMDP with non-reachability objectives. We show how to compute the GFP of max/min PPSs to desired precision in P-time. We also study more general Branching Simple Stochastic Games (BSSGs) with (non-)reachability objectives. We show that: (1) the value of these games is captured by the GFP of a corresponding max-minPPS; (2) the quantitative problem of approximating the value is in TFNP; and (3) the qualitative problems associated with the value are all solvable in P-time