General limit value in Dynamic Programming

We consider a dynamic programming problem with arbitrary state space and bounded rewards. Is it possible to define in an unique way a limit value for the problem, where the "patience" of the decision-maker tends to infinity ? We consider, for each evaluation $\theta$ (a probability distribution over positive integers) the value function $v_{\theta}$ of the problem where the weight of any stage $t$ is given by $\theta_t$, and we investigate the uniform convergence of a sequence $(v_{\theta^k})_k$ when the "impatience" of the evaluations vanishes, in the sense that $\sum_{t} |\theta^k_{t}-\theta^k_{t+1}| \rightarrow_{k \to \infty} 0$. We prove that this uniform convergence happens if and only if the metric space ${v_{\theta^k}, k\geq 1}$ is totally bounded. Moreover there exists a particular function $v^*$, independent of the particular chosen sequence $({\theta^k})_k$, such that any limit point of such sequence of value functions is precisely $v^*$. Consequently, while speaking of uniform convergence of the value functions, $v^*$ may be considered as the unique possible limit when the patience of the decision-maker tends to infinity. The result applies in particular to discounted payoffs when the discount factor vanishes, as well as to average payoffs where the number of stages goes to infinity, and also to models with stochastic transitions. We present tractable corollaries, and we discuss counterexamples and a conjecture

Continuous bounded cocycles

Let $G$ be a minimal locally compact groupoid with compact metrizable unit space and let $E$ be a continuous $G$-Hilbert bundle. We show that a bounded continuous cocycle c: G\ra r^*E is necessarily a continuous coboundary. This is a groupoid version of a classical theorem of Gottschalk and Hedlund.Comment: 14 pages, presented at the EU-NCG4 Conference, Bucharest 201

The value of Repeated Games with an informed controller

We consider the general model of zero-sum repeated games (or stochastic games with signals), and assume that one of the players is fully informed and controls the transitions of the state variable. We prove the existence of the uniform value, generalizing several results of the literature. A preliminary existence result is obtained for a certain class of stochastic games played with pure strategies

Random walks on Bratteli diagrams

In the eighties, A. Connes and E. J. Woods made a connection between hyperfinite von Neumann algebras and Poisson boundaries of time dependent random walks. The present paper explains this connection and gives a detailed proof of two theorems quoted there: the construction of a large class of states on a hyperfinite von Neumann algebra (due to A. Connes) and the ergodic decomposition of a Markov measure via harmonic functions (a classical result in probability theory). The crux of the first theorem is a model for conditional expectations on finite dimensional C*-algebras. The proof of the second theorem hinges on the notion of cotransition probability.Comment: 18 pages, written version of a talk given at the Operator Theory 26th Conference, Timisoara 201