Nonzero-sum Stochastic Games

This paper treats of stochastic games. We focus on nonzero-sum games and provide a detailed survey of selected recent results. In Section 1, we consider stochastic Markov games. A correlation of strategies of the players, involving ``public signals'', is described, and a correlated equilibrium theorem proved recently by Nowak and Raghavan for discounted stochastic games with general state space is presented. We also report an extension of this result to a class of undiscounted stochastic games, satisfying some uniform ergodicity condition. Stopping games are related to stochastic Markov games. In Section 2, we describe a version of Dynkin's game related to observation of a Markov process with random assignment mechanism of states to the players. Some recent contributions of the second author in this area are reported. The paper also contains a brief overview of the theory of nonzero-sum stochastic games and stopping games which is very far from being complete.average payoff stochastic games, correlated stationary equilibria, nonzero-sum games, stopping time, stopping games

Stochastic games with endogenous transitions

We introduce a stochastic game in which transition probabilities depend on the history of the play, i.e., the players' past action choices. To solve this new type of game under the limiting average reward criterion, we determine the set of jointly-convergent pure-strategy rewards which can be supported by equilibria involving threats. We examine the following setting for motivational and expository purposes. Each period, two agents exploiting a fishery choose between catching with restraint or without. The fish stock is in either of two\ud states, High or Low, and in the latter each action pair yields lower payoffs. Restraint is harmless to the fish, but it is a dominated strategy in each stage game. Absence of restraint damages the resource, i.e., the less restraint the agents show, the higher the probablities that Low occurs at the next stage of the play. This state may even become 'absorbing', i.e., transitions to High become impossible

Optimality in different strategy classes in zero-sum stochastic games

We present a complete picture of the relationship between the existence of 0-optimal strategies and c-optimal strategies, epsilon > 0, in the classes of stationary, Markov and history dependent strategies

On the computation of large sets of rewards in ETP-ESP-games with communicating states

Games with endogenous transition probabilities and endogenous stage payoffs (or ETP-ESP-games) are stochastic games in which both the transition probabilities and the payo¤s at any stage are continuous functions of the relative frequencies of all action combinations chosen in the past. We present methods to compute large sets of jointly-convergent pure-strategy rewards in ETP-ESP-games with communicating states. Such sets are useful in determining feasible rewards in a game. They are also instrumental in obtaining the set of (Nash) equilibrium rewards

On the maxmin value of stochastic games with imperfect monitoring

We study zero-sum stochastic games in which players do not observe the actions of the opponent. Rather, they observe a stochastic signal that may depend on the state, and on the pair of actions chosen by the players. We assume each player observes the state and his own action. We propose a candidate for the max-min value, which does not depend on the information structure of player 2. We prove that player 2 can defend the proposed max-min value, and that in absorbing games player 1 can guarantee it. Analogous results hold for the min-max value. This paper thereby unites several results due to Coulomb.Stochastic games; partial monitoring; value

Stochastic Games on a Product State Space

We examine product-games, which are n-player stochastic games satisfying: (1) the state space is a product S(1)Ãâ¦ÃS(n); (2) the action space of any player i only depends of the i-th coordinate of the state; (3) the transition probability of moving from s(i) ∈ S(i) to t(i) ∈S(i), on the i-th coordinate S(i) of the state space, only depends on the action chosen by player i. So, as far as the actions and the transitions are concerned, every player i can play on the i-th coordinate of the product-game without interference of the other players. No condition is imposed on the payoff structure of the game. We focus on product-games with an aperiodic transition structure, for which we present an approach based on so-called communicating states. For the general n-player case, we establish the existence of 0-equilibria, which makes product-games one of the first classes within n-player stochastic games with such a result. In addition, for the special case of two-player zero-sum games of this type, we show that both players have stationary 0-optimal strategies. Both proofs are constructive by nature.Economics (Jel: A)

A Nested Family of kk-total Effective Rewards for Positional Games

We consider Gillette's two-person zero-sum stochastic games with perfect information. For each k \in \ZZ_+ we introduce an effective reward function, called $k$-total. For $k = 0$ and $1$ this function is known as {\it mean payoff} and {\it total reward}, respectively. We restrict our attention to the deterministic case. For all $k$, we prove the existence of a saddle point which can be realized by uniformly optimal pure stationary strategies. We also demonstrate that $k$-total reward games can be embedded into $(k+1)$-total reward games

Nonzero-sum Stochastic Games

This paper treats of stochastic games. We focus on nonzero-sum games and provide a detailed survey of selected recent results. In Section 1, we consider stochastic Markov games. A correlation of strategies of the players, involving ``public signals'', is described, and a correlated equilibrium theorem proved recently by Nowak and Raghavan for discounted stochastic games with general state space is presented. We also report an extension of this result to a class of undiscounted stochastic games, satisfying some uniform ergodicity condition. Stopping games are related to stochastic Markov games. In Section 2, we describe a version of Dynkin's game related to observation of a Markov process with random assignment mechanism of states to the players. Some recent contributions of the second author in this area are reported. The paper also contains a brief overview of the theory of nonzero-sum stochastic games and stopping games which is very far from being complete