Stochastic Approximation, Momentum, and Nash Play

Main objects here are normal-form games, featuring uncertainty and noncooperative players who entertain local visions, form local approximations, and hesitate in making large, swift adjustments. For the purpose of reaching Nash equilibrium, or learning such play, we advocate and illustrate an algorithm that combines stochastic gradient projection with the heavyball method. What emerges is a coupled, constrained, second-order stochastic process. Some friction feeds into and stabilizes myopic approximations. Convergence to Nash play obtains under seemingly weak and natural conditions, an important one being that accumulated marginal payoffs remains bounded above.Noncooperative games; Nash equilibrium; stochastic programming and approximation; the heavy ball method.

Convexity, Differential Equations, and Games

Theoretical and experimental studies of noncooperative games increasingly recognize Nash equilibrium as a limiting outcome of players‘ repeated interaction. This note, while sharing that view, illustrates and advocates combined use of convex optimization and differential equations, the purpose being to render equilibrium both plausible and stable.noncooperative games, Nash equilibrium, repeated play, differential equations, stability.

Higher-Order Uncoupled Dynamics Do Not Lead to Nash Equilibrium \unicode{x2014} Except When They Do

The framework of multi-agent learning explores the dynamics of how individual agent strategies evolve in response to the evolving strategies of other agents. Of particular interest is whether or not agent strategies converge to well known solution concepts such as Nash Equilibrium (NE). Most ``fixed order'' learning dynamics restrict an agent's underlying state to be its own strategy. In ``higher order'' learning, agent dynamics can include auxiliary states that can capture phenomena such as path dependencies. We introduce higher-order gradient play dynamics that resemble projected gradient ascent with auxiliary states. The dynamics are ``payoff based'' in that each agent's dynamics depend on its own evolving payoff. While these payoffs depend on the strategies of other agents in a game setting, agent dynamics do not depend explicitly on the nature of the game or the strategies of other agents. In this sense, dynamics are ``uncoupled'' since an agent's dynamics do not depend explicitly on the utility functions of other agents. We first show that for any specific game with an isolated completely mixed-strategy NE, there exist higher-order gradient play dynamics that lead (locally) to that NE, both for the specific game and nearby games with perturbed utility functions. Conversely, we show that for any higher-order gradient play dynamics, there exists a game with a unique isolated completely mixed-strategy NE for which the dynamics do not lead to NE. These results build on prior work that showed that uncoupled fixed-order learning cannot lead to NE in certain instances, whereas higher-order variants can. Finally, we consider the mixed-strategy equilibrium associated with coordination games. While higher-order gradient play can converge to such equilibria, we show such dynamics must be inherently internally unstable

Higher Order Game Dynamics

Continuous-time game dynamics are typically first order systems where payoffs determine the growth rate of the players' strategy shares. In this paper, we investigate what happens beyond first order by viewing payoffs as higher order forces of change, specifying e.g. the acceleration of the players' evolution instead of its velocity (a viewpoint which emerges naturally when it comes to aggregating empirical data of past instances of play). To that end, we derive a wide class of higher order game dynamics, generalizing first order imitative dynamics, and, in particular, the replicator dynamics. We show that strictly dominated strategies become extinct in n-th order payoff-monotonic dynamics n orders as fast as in the corresponding first order dynamics; furthermore, in stark contrast to first order, weakly dominated strategies also become extinct for n>1. All in all, higher order payoff-monotonic dynamics lead to the elimination of weakly dominated strategies, followed by the iterated deletion of strictly dominated strategies, thus providing a dynamic justification of the well-known epistemic rationalizability process of Dekel and Fudenberg (1990). Finally, we also establish a higher order analogue of the folk theorem of evolutionary game theory, and we show that con- vergence to strict equilibria in n-th order dynamics is n orders as fast as in first order.Comment: 32 pages, 6 figures; to appear in the Journal of Economic Theory. Updated material on the microfoundations of the dynamic

Higher Order Games Dynamics

Le PDF est la version auteurContinuous-time game dynamics are typically first order systems where payoffs determine the growth rate of the playersʼ strategy shares. In this paper, we investigate what happens beyond first order by viewing payoffs as higher order forces of change, specifying e.g. the acceleration of the playersʼ evolution instead of its velocity (a viewpoint which emerges naturally when it comes to aggregating empirical data of past instances of play). To that end, we derive a wide class of higher order game dynamics, generalizing first order imitative dynamics, and, in particular, the replicator dynamics. We show that strictly dominated strategies become extinct in n-th order payoff-monotonic dynamics n orders as fast as in the corresponding first order dynamics; furthermore, in stark contrast to first order, weakly dominated strategies also become extinct for n⩾2. All in all, higher order payoff-monotonic dynamics lead to the elimination of weakly dominated strategies, followed by the iterated deletion of strictly dominated strategies, thus providing a dynamic justification of the well-known epistemic rationalizability process of Dekel and Fudenberg [7]. Finally, we also establish a higher order analogue of the folk theorem of evolutionary game theory, and we show that convergence to strict equilibria in n-th order dynamics is n orders as fast as in first order

Newtonian Mechanics and Nash Play

Nash equilibrium leaves the impression that each player foresees perfectly and responds optimally. Must human-like, rational agents really acquire both these faculties? This paper argues that in some instances neither is ever needed. For the argument repeated play is modelled here as a constrained, decentralized, second-order process driven by noncoordinated pursuit of better payoffs. Some friction feeds into — and stabilizes — a fairly myopic mode of behavior. Convergence to equilibrium therefore obtains under weak and natural conditions. An important condition is that accumulation of marginal payoffs, along the path of play, yields a sum which is bounded above