Approximate solutions of continuous-time stochastic games

The paper is concerned with a zero-sum continuous-time stochastic differential game with a dynamics controlled by a Markov process and a terminal payoff. The value function of the original game is estimated using the value function of a model game. The dynamics of the model game differs from the original one. The general result applied to differential games yields the approximation of value function of differential game by the solution of countable system of ODEs.Comment: 23 page

Avoiding the Curse of Dimensionality in Dynamic Stochastic Games

Continuous-time stochastic games with a finite number of states have substantial computational and conceptual advantages over the more common discrete-time model. In particular, continuous time avoids a curse of dimensionality and speeds up computations by orders of magnitude in games with more than a few state variables. The continuous-time approach opens the way to analyze more complex and realistic stochastic games than is feasible in discrete-time models.

An update on continuous-time stochastic games of fixed duration

This paper shows that continuous-time stochastic games of fixed duration need not possess equilibria in Markov strategies. The example requires payoffs and transitions to depend on time in a continuous but irregular (almost nowhere almost differentiable) way. This example offers a correction to the erroneous construction presented previously in Levy (Dyn Games Appl 3(2):279–312, 2013. https://doi.org/10.1007/s13235-012-0067-2)

Avoiding the Curse of Dimensionality in Dynamic Stochastic Games

Discrete-time stochastic games with a finite number of states have been widely applied to study the strategic interactions among forward-looking players in dynamic environments. These games suffer from a curse of dimensionality when the cost of computing players\u27 expectations over all possible future states increases exponentially in the number of state variables. We explore the alternative of continuous-time stochastic games with a finite number of states and argue that continuous time may have substantial advantages. In particular, under widely used laws of motion, continuous time avoids the curse of dimensionality in computing expectations, thereby speeding up the computations by orders of magnitude in games with more than a few state variables. This much smaller computational burden greatly extends the range and richness of applications of stochastic games

Approximate public-signal correlated equilibria for nonzero-sum differential games

We construct an approximate public-signal correlated equilibrium for a nonzero-sum differential game in the class of stochastic strategies with memory. The construction is based on a solution of an auxiliary nonzero-sum continuous-time stochastic game. This class of games includes stochastic differential games and continuous-time Markov games. Moreover, we study the limit of approximate equilibrium outcomes in the case when the auxiliary stochastic games tend to the original deterministic one. We show that it lies in the convex hull of the set of equilibrium values provided by deterministic punishment strategies.Comment: 35 page

MeGARA: Menu-based Game Abstraction and Abstraction Refinement of Markov Automata

Markov automata combine continuous time, probabilistic transitions, and nondeterminism in a single model. They represent an important and powerful way to model a wide range of complex real-life systems. However, such models tend to be large and difficult to handle, making abstraction and abstraction refinement necessary. In this paper we present an abstraction and abstraction refinement technique for Markov automata, based on the game-based and menu-based abstraction of probabilistic automata. First experiments show that a significant reduction in size is possible using abstraction.Comment: In Proceedings QAPL 2014, arXiv:1406.156

Avoiding the Curse of Dimensionality in Dynamic Stochastic Games

Discrete-time stochastic games with a finite number of states have been widely ap- plied to study the strategic interactions among forward-looking players in dynamic en- vironments. However, these games suffer from a "curse of dimensionality" since the cost of computing players' expectations over all possible future states increases exponentially in the number of state variables. We explore the alternative of continuous-time stochas- tic games with a finite number of states, and show that continuous time has substantial computational and conceptual advantages. Most important, continuous time avoids the curse of dimensionality, thereby speeding up the computations by orders of magnitude in games with more than a few state variables. Overall, the continuous-time approach opens the way to analyze more complex and realistic stochastic games than currently feasible.