4,212 research outputs found
Game-theoretical control with continuous action sets
Motivated by the recent applications of game-theoretical learning techniques
to the design of distributed control systems, we study a class of control
problems that can be formulated as potential games with continuous action sets,
and we propose an actor-critic reinforcement learning algorithm that provably
converges to equilibrium in this class of problems. The method employed is to
analyse the learning process under study through a mean-field dynamical system
that evolves in an infinite-dimensional function space (the space of
probability distributions over the players' continuous controls). To do so, we
extend the theory of finite-dimensional two-timescale stochastic approximation
to an infinite-dimensional, Banach space setting, and we prove that the
continuous dynamics of the process converge to equilibrium in the case of
potential games. These results combine to give a provably-convergent learning
algorithm in which players do not need to keep track of the controls selected
by the other agents.Comment: 19 page
Measure-valued limits of interacting particle systems with k-nary interactions. II, Finite-dimensional limits
It is shown that Markov chains in Z+d describing k-nary interacting particles of d different types approximate (in the continuous state limit) Markov processes on R+d having pseudo-differential generators p (x,i (/x)) with symbols p (x,) depending polynomially (degree k) on x. This approximation can be used to prove existence and non-explosion results for the latter processes. Our general scheme of continuous state (or finite-dimensional measure-valued) limits to processes of k-nary interaction yields a unified description of these limits for a large variety of models that are intensively studied in different domains of natural science from interacting particles in statistical mechanics (e.g. coagulation-fragmentation processes) to evolutionary games and multidimensional birth and death processes from biology and social sciences
- …