Fictitious play and- no-cycling conditions

We investigate the paths of pure strategy profiles induced by the fictitious play process. We present rules that such paths must follow. Using these rules we prove that every non-degenerate 2*3 game has the continuous fictitious play property, that is, every continuous fictitious play process, independent of initial actions and beliefs, approaches equilibrium in such games.

Two Essays On Economic Theory

My thesis consists of two separate essays on economic theory. The title of the first essay (chapter 1) is Learning and Nash Equilibria in 3 x 3 Symmetric Games and the title of the second essay (chapter 2) is General Equilibrium Theory with Monopolistic Competition: An Introductory Analysis. ;Chapter 1 explores the dynamic implications of learning models by studying fictitious play in 3 x 3 symmetric games. The basic model consists of two persons playing a symmetric normal form game with only three pure strategies repeatedly, and choosing their strategies in each period to maximize their expected payoffs in the stage game. After each play of the game each person forms his belief about his opponent\u27s next strategy choice according to rules defined by fictitious play. It is shown that for a reasonably wide class of 3 x 3 symmetric games the sequence of beliefs generated by fictitious play inevitably converges to a mixed-strategy Nash equilibrium. For games that do not belong to this class, the limiting outcomes of fictitious play with identical initial beliefs are characterized. Finally, replacing fictitious play with another more sophisticated learning process yields stronger convergence results for fictitious play. These results provide useful references for future work in this area.;Chapter 2 provides an introduction to general equilibrium theory with monopolistic competition. A model of an economy with monopolistic competitive firms is constructed and studied. Each monopolistic firm perceives a demand function for its output which satisfies certain consistency condition in equilibrium. Three basic questions in equilibrium theory are addressed in the context of the model: existence, uniqueness and pareto compatibility of monopolistic competitive equilibria. Using fairly standard assumptions, it is shown that an infinite number of monopolistic competitive equilibria exists which are supported by different systems of perceived demand functions. Fixing the perceived demand functions, however, the number of equilibria (with different prices and allocations) is generically finite. Also, different equilibria supported by different systems of perceived demand functions are in general pareto incompatible

A Generalized Training Approach for Multiagent Learning

This paper investigates a population-based training regime based on game-theoretic principles called Policy-Spaced Response Oracles (PSRO). PSRO is general in the sense that it (1) encompasses well-known algorithms such as fictitious play and double oracle as special cases, and (2) in principle applies to general-sum, many-player games. Despite this, prior studies of PSRO have been focused on two-player zero-sum games, a regime wherein Nash equilibria are tractably computable. In moving from two-player zero-sum games to more general settings, computation of Nash equilibria quickly becomes infeasible. Here, we extend the theoretical underpinnings of PSRO by considering an alternative solution concept, $\alpha$-Rank, which is unique (thus faces no equilibrium selection issues, unlike Nash) and applies readily to general-sum, many-player settings. We establish convergence guarantees in several games classes, and identify links between Nash equilibria and $\alpha$-Rank. We demonstrate the competitive performance of $\alpha$-Rank-based PSRO against an exact Nash solver-based PSRO in 2-player Kuhn and Leduc Poker. We then go beyond the reach of prior PSRO applications by considering 3- to 5-player poker games, yielding instances where $\alpha$-Rank achieves faster convergence than approximate Nash solvers, thus establishing it as a favorable general games solver. We also carry out an initial empirical validation in MuJoCo soccer, illustrating the feasibility of the proposed approach in another complex domain

Deep Reinforcement Learning from Self-Play in Imperfect-Information Games

Many real-world applications can be described as large-scale games of imperfect information. To deal with these challenging domains, prior work has focused on computing Nash equilibria in a handcrafted abstraction of the domain. In this paper we introduce the first scalable end-to-end approach to learning approximate Nash equilibria without prior domain knowledge. Our method combines fictitious self-play with deep reinforcement learning. When applied to Leduc poker, Neural Fictitious Self-Play (NFSP) approached a Nash equilibrium, whereas common reinforcement learning methods diverged. In Limit Texas Holdem, a poker game of real-world scale, NFSP learnt a strategy that approached the performance of state-of-the-art, superhuman algorithms based on significant domain expertise.Comment: updated version, incorporating conference feedbac

No-regret Dynamics and Fictitious Play

Potential based no-regret dynamics are shown to be related to fictitious play. Roughly, these are epsilon-best reply dynamics where epsilon is the maximal regret, which vanishes with time. This allows for alternative and sometimes much shorter proofs of known results on convergence of no-regret dynamics to the set of Nash equilibria