Optimality in different strategy classes in zero-sum stochastic games

We present a complete picture of the relationship between the existence of 0-optimal strategies and c-optimal strategies, epsilon > 0, in the classes of stationary, Markov and history dependent strategies

Stochastic games with non-observable actions

We examine n-player stochastic games. These are dynamic games where a play evolves in stages along a finite set of states; at each stage players independently have to choose actions in the present state and these choices determine a stage payoff to each player as well as a transition to a new state where actions have to be chosen at the next stage. For each player the infinite sequence of his stage payoffs is evaluated by taking the limiting average. Normally stochastic games are examined under the condition of full monitoring, i.e. At any stage each player observes the present state and the actions chosen by all players. This paper is a first attempt towards understanding under what circumstances equilibria could exist in n-player stochastic games without full monitoring. We demonstrate the non-existence of ?-equilibria in n-player stochastic games, with respect to the average reward, when at each stage each player is able to observe the present state, his own action, his own payoff, and the payoffs of the other players, but is unable to observe the actions of them. For this purpose, we present and examine a counterexample with 3 players. If we further drop the assumption that the players can observe the payoffs of the others, then counterexamples already exist in games with only 2 players

Stationary Strategies in Zero-Sum Stochastic Games

We deal with zero-sum stochastic games. We demonstrate the importance of stationary strategies by showing that stationary strategies are better (in terms of the rewards they guarantee for a player, against any strategy of his opponent) than (1) pure strategies (even history-dependent ones), (2) strategies which may use only a finite number of different mixed actions in any state, and (3) strategies with finite recall. Examples are given to clarify the issues

Finite-Step Algorithms for Single-Controller and Perfect Information Stochastic Games

Abstract. After a brief survey of iterative algorithms for general stochas-tic games, we concentrate on finite-step algorithms for two special classes of stochastic games. They are Single-Controller Stochastic Games and Per-fect Information Stochastic Games. In the case of single-controller games, the transition probabilities depend on the actions of the same player in all states. In perfect information stochastic games, one of the players has exactly one action in each state. Single-controller zero-sum games are effi-ciently solved by linear programming. Non-zero-sum single-controller stochastic games are reducible to linear complementary problems (LCP). In the discounted case they can be modified to fit into the so-called LCPs of Eave’s class L. In the undiscounted case the LCP’s are reducible to Lemke’s copositive plus class. In either case Lemke’s algorithm can be used to find a Nash equilibrium. In the case of discounted zero-sum perfect informa-tion stochastic games, a policy improvement algorithm is presented. Many other classes of stochastic games with orderfield property still await efficient finite-step algorithms. 1

Nonzero-sum Stochastic Games

This paper treats of stochastic games. We focus on nonzero-sum games and provide a detailed survey of selected recent results. In Section 1, we consider stochastic Markov games. A correlation of strategies of the players, involving ``public signals'', is described, and a correlated equilibrium theorem proved recently by Nowak and Raghavan for discounted stochastic games with general state space is presented. We also report an extension of this result to a class of undiscounted stochastic games, satisfying some uniform ergodicity condition. Stopping games are related to stochastic Markov games. In Section 2, we describe a version of Dynkin's game related to observation of a Markov process with random assignment mechanism of states to the players. Some recent contributions of the second author in this area are reported. The paper also contains a brief overview of the theory of nonzero-sum stochastic games and stopping games which is very far from being complete

The bad match; a total reward stochastic game

For two person zero sum stochastic games we introduce a new criterion for evaluating streams of payoffs. When the players use this criterion we call such games total reward stochastic games. It is unknown whether total reward stochastic games, with the property that the average value is zero for each initial state, always have a value. We examine an example of such a total reward stochastic game in which one of the players can playe-optimal only by using history dependent strategies