Operator approach to values of stochastic games with varying stage duration

We study the links between the values of stochastic games with varying stage duration $h$, the corresponding Shapley operators $\bf{T}$ and ${\bf{T}}\_h$and the solution of $\dot f\_t = ({\bf{T}} - Id )f\_t$. Considering general non expansive maps we establish two kinds of results, under both the discounted or the finite length framework, that apply to the class of "exact" stochastic games. First, for a fixed length or discount factor, the value converges as the stage duration go to 0. Second, the asymptotic behavior of the value as the length goes to infinity, or as the discount factor goes to 0, does not depend on the stage duration. In addition, these properties imply the existence of the value of the finite length or discounted continuous time game (associated to a continuous time jointly controlled Markov process), as the limit of the value of any time discretization with vanishing mesh.Comment: 22 pages, International Journal of Game Theory, Springer Verlag, 201

A uniform Tauberian theorem in optimal control

In an optimal control framework, we consider the value $V_T(x)$ of the problem starting from state $x$ with finite horizon $T$, as well as the value $V_\lambda(x)$ of the $\lambda$-discounted problem starting from $x$. We prove that uniform convergence (on the set of states) of the values $V_T(\cdot)$ as $T$ tends to infinity is equivalent to uniform convergence of the values $V_\lambda(\cdot)$ as $\lambda$ tends to 0, and that the limits are identical. An example is also provided to show that the result does not hold for pointwise convergence. This work is an extension, using similar techniques, of a related result in a discrete-time framework \cite{LehSys}.Comment: 14 page

Asymptotic Properties of Optimal Trajectories in Dynamic Programming

We prove in a dynamic programming framework that uniform convergence of the finite horizon values implies that asymptotically the average accumulated payoff is constant on optimal trajectories. We analyze and discuss several possible extensions to two-person games.Comment: 9 page

Definable Zero-Sum Stochastic Games

International audienceDefinable zero-sum stochastic games involve a finite number of states and action sets, reward and transition functions that are definable in an o-minimal structure. Prominent examples of such games are finite, semi-algebraic or globally subanalytic stochastic games. We prove that the Shapley operator of any definable stochastic game with separable transition and reward functions is definable in the same structure. Definability in the same structure does not hold systematically: we provide a counterexample of a stochastic game with semi-algebraic data yielding a non semi-algebraic but globally subanalytic Shapley operator. %Showing the definability of the Shapley operator in full generality appears thus as a complex and challenging issue. } Our definability results on Shapley operators are used to prove that any separable definable game has a uniform value; in the case of polynomially bounded structures we also provide convergence rates. Using an approximation procedure, we actually establish that general zero-sum games with separable definable transition functions have a uniform value. These results highlight the key role played by the tame structure of transition functions. As particular cases of our main results, we obtain that stochastic games with polynomial transitions, definable games with finite actions on one side, definable games with perfect information or switching controls have a uniform value. Applications to nonlinear maps arising in risk sensitive control and Perron-Frobenius theory are also given

Evolution equations in discrete and continuous time for nonexpansive operators in Banach spaces

We consider some discrete and continuous dynamics in a Banach space involving a non expansive operator $J$ and a corresponding family of strictly contracting operators $\Phi(\lambda,x):=\lambda J(\frac{1-\lambda}{\lambda}x)$ for $\lambda\in]0,1]$. Our motivation comes from the study of two-player zero-sum repeated games, where the value of the $n$-stage game (resp. the value of the $\lambda$-discounted game) satisfies the relation $v_n=\Phi(\frac{1}{n},v_{n-1})$ (resp. $v_\lambda=\Phi(\lambda,v_\lambda)$) where $J$ is the Shapley operator of the game. We study the evolution equation $u'(t)=J(u(t))-u(t)$ as well as associated Eulerian schemes, establishing a new exponential formula and a Kobayashi-like inequality for such trajectories. We prove that the solution of the non-autonomous evolution equation $u'(t)=\Phi(\bm{\lambda}(t),u(t))-u(t)$ has the same asymptotic behavior (even when it diverges) as the sequence $v_n$ (resp. as the family $v_\lambda$) when $\bm{\lambda}(t)=1/t$ (resp. when $\bm{\lambda}(t)$ converges slowly enough to 0).Comment: 28 pages To appear in ESAIM:COC

Iterated monotonic nonexpansive operators and asymptotic properties of zero-sum stochastic games

We consider an operator \Ps defined on a set of real valued functions and satisfying two properties of monotonicity and additive homogeneity. This is motivated by the case of zero sum stochastic games, for which the Shapley operator is monotone and additively homogeneous. We study the asymptotic of the trajectories defined by v_n=\frac{\Ps^n(0)}{n} ($n\in N , n \rightarrow \infty$) and v_\lambda=\lambda\Ps\left(\frac{1-\lambda}{\lambda}v_\lambda\right) ($\lambda \in (0,1], \lambda \rightarrow 0$). Examining the iterates of \Ps, we exhibit analytical conditions on the operator that imply that $v_n$ and $v_\lambda$ have at most one accumulation point for the uniform norm. In particular this establishes the uniform convergence of $v_n$ and $v_\lambda$ to the same limit for a large subclass of the class of games where only one player control the transitions. We also study the general case of two players controlling the transitions, giving a sufficient condition for convergence.ou

Existence of the limit value of two person zero-sum discounted repeated games via comparison theorems

Abstract We give new proofs of existence of the limit of the discounted values for two person zerosum games in the three following frameworks: absorbing, recursive, incomplete information. The idea of these new proofs is to use some comparison criteria