On Zero-Sum Two Person Perfect Information Stochastic Games

A zero-sum two person Perfect Information Stochastic game (PISG) under limiting average payoff has a value and both the maximiser and the minimiser have optimal pure stationary strategies. Firstly we form the matrix of undiscounted payoffs corresponding to each pair of pure stationary strategies (for each initial state) of the two players and prove that this matrix has a pure saddle point. Then by using the results by Derman [1] we prove the existence of optimal pure stationary strategy pair of the players. A crude but finite step algorithm is given to compute such an optimal pure stationary strategy pair of the players.Comment: arXiv admin note: text overlap with arXiv:2201.0017

Algorithms for uniform optimal strategies in two-player zero-sum stochastic games with perfect information

In stochastic games with perfect information, in each state at most one player has more than one action available. We propose two algorithms which find the uniform optimal strategies for zero-sum two-player stochastic games with perfect information. Such strategies are optimal for the long term average criterion as well. We prove the convergence for one algorithm, which presents a higher complexity than the other one, for which we provide numerical analysis.Dans les jeux stochastiques à information parfaite, dans chaque etat, au plus, un joueur a plus d'une action disponibles. Nous proposons deux algorithmes qui trouvent les stratégies uniformément optimales pour les jeux stochastiques à somme nulle avec deux joueurs et information parfaite. Ces stratégies sont aussi optimales pour le critère de la moyenne à long terme. Nous prouvons la convergence pour un algorithme, qui a une plus grande complexité que l'autre, pour lequel nous offrons une analyse numérique

Finite-Step Algorithms for Single-Controller and Perfect Information Stochastic Games

Abstract. After a brief survey of iterative algorithms for general stochas-tic games, we concentrate on finite-step algorithms for two special classes of stochastic games. They are Single-Controller Stochastic Games and Per-fect Information Stochastic Games. In the case of single-controller games, the transition probabilities depend on the actions of the same player in all states. In perfect information stochastic games, one of the players has exactly one action in each state. Single-controller zero-sum games are effi-ciently solved by linear programming. Non-zero-sum single-controller stochastic games are reducible to linear complementary problems (LCP). In the discounted case they can be modified to fit into the so-called LCPs of Eave’s class L. In the undiscounted case the LCP’s are reducible to Lemke’s copositive plus class. In either case Lemke’s algorithm can be used to find a Nash equilibrium. In the case of discounted zero-sum perfect informa-tion stochastic games, a policy improvement algorithm is presented. Many other classes of stochastic games with orderfield property still await efficient finite-step algorithms. 1

Integer Programming Approaches to Stochastic Games Arising in Paired Kidney Exchange and Industrial Organization

We investigate three different problems in this dissertation. The first two problems are related to games arising in paired kidney exchange, and the third is rooted in a computational branch of the industrial organization literature. We provide more details on these problems in the following. End-stage renal disease (ESRD), the final stage of chronic kidney disease, is the ninth-leading cause of death in the United States, where it afflicts more than a half million patients, and costs more than forty billion dollars indirect expenses annually. Transplantation is the preferred treatment for ESRD; unfortunately, there is a severe shortage of transplantable kidneys. Kidney exchange is a growing approach to alleviate the shortage of kidneys for transplantation, and the United States is considering creating a national kidney exchange program since such a program provides more and better transplants. A major challenge to establish a national kidney exchange program is the lack of incentives for transplant centers to participate in such a program. To overcome this issue, the kidney transplant community has recently proposed a payment strategy framework that incentivizes transplant centers to participate in a national program. Absent from this debate is a careful investigation of how to design these incentives. We develop a principal-agent model to analyze these incentives and find an equilibrium payment strategy. We develop a mixed-integer bilinear bilevel program to compute an equilibrium payment strategy. We show that this bilevel program can be solved as a mixed-integer linear program. We calibrate our model and provide several data-driven insights about advantages of a national kidney exchange program. We shed light on several controversial policy questions about an equilibrium payment strategy. In particular, we demonstrate that there exists a ``win-win'' payment strategy that could result in saving thousands of lives and billions of dollars annually. Consensus stopping games are a class of stochastic games that arises in the context of kidney exchange. Specifically, the problem of finding a socially optimal pure stationary equilibrium of a consensus stopping game is adapted to value a given kidney exchange. However, computational difficulties have limited its applicability. We show that a consensus stopping game may have many pure stationary equilibria, which in turn raises the question of equilibrium selection. Given an objective criterion, we study the problem of finding a best pure stationary equilibrium for the game, which we show to be NP-hard. We characterize the pure stationary equilibria, show that they form an independence system, and develop several families of valid inequalities. We then solve the equilibrium selection problem as a mixed-integer linear program (MILP) by a branch-and-cut approach. Our computational results demonstrate the advantages of our approach over a commercial solver. Industrial organization is an area of economics that studies firms and markets. Currently, a class of stochastic games are adopted to model behaviors of firms in a market. However, inherent challenges in computability of stationary equilibria have restricted its applicability. To overcome this challenge, we develop several characterizations of stationary equilibria for the class of stochastic games