1,728 research outputs found
From Weak Learning to Strong Learning in Fictitious Play Type Algorithms
The paper studies the highly prototypical Fictitious Play (FP) algorithm, as
well as a broad class of learning processes based on best-response dynamics,
that we refer to as FP-type algorithms. A well-known shortcoming of FP is that,
while players may learn an equilibrium strategy in some abstract sense, there
are no guarantees that the period-by-period strategies generated by the
algorithm actually converge to equilibrium themselves. This issue is
fundamentally related to the discontinuous nature of the best response
correspondence and is inherited by many FP-type algorithms. Not only does it
cause problems in the interpretation of such algorithms as a mechanism for
economic and social learning, but it also greatly diminishes the practical
value of these algorithms for use in distributed control. We refer to forms of
learning in which players learn equilibria in some abstract sense only (to be
defined more precisely in the paper) as weak learning, and we refer to forms of
learning where players' period-by-period strategies converge to equilibrium as
strong learning. An approach is presented for modifying an FP-type algorithm
that achieves weak learning in order to construct a variant that achieves
strong learning. Theoretical convergence results are proved.Comment: 22 page
On Robustness Properties in Empirical Centroid Fictitious Play
Empirical Centroid Fictitious Play (ECFP) is a generalization of the
well-known Fictitious Play (FP) algorithm designed for implementation in
large-scale games. In ECFP, the set of players is subdivided into equivalence
classes with players in the same class possessing similar properties. Players
choose a next-stage action by tracking and responding to aggregate statistics
related to each equivalence class. This setup alleviates the difficult task of
tracking and responding to the statistical behavior of every individual player,
as is the case in traditional FP. Aside from ECFP, many useful modifications
have been proposed to classical FP, e.g., rules allowing for network-based
implementation, increased computational efficiency, and stronger forms of
learning. Such modifications tend to be of great practical value; however,
their effectiveness relies heavily on two fundamental properties of FP:
robustness to alterations in the empirical distribution step size process, and
robustness to best-response perturbations. The main contribution of the paper
is to show that similar robustness properties also hold for the ECFP algorithm.
This result serves as a first step in enabling practical modifications to ECFP,
similar to those already developed for FP.Comment: Submitted for publication. Initial Submission: Mar. 201
Self-tuning experience weighted attraction learning in games
Self-tuning experience weighted attraction (EWA) is a one-parameter theory of learning in
games. It addresses a criticism that an earlier model (EWA) has too many parameters, by
fixing some parameters at plausible values and replacing others with functions of experience
so that they no longer need to be estimated. Consequently, it is econometrically simpler
than the popular weighted fictitious play and reinforcement learning models.
The functions of experience which replace free parameters “self-tune” over time, adjusting
in a way that selects a sensible learning rule to capture subjects’ choice dynamics. For
instance, the self-tuning EWA model can turn from a weighted fictitious play into an averaging
reinforcement learning as subjects equilibrate and learn to ignore inferior foregone
payoffs. The theory was tested on seven different games, and compared to the earlier parametric
EWA model and a one-parameter stochastic equilibrium theory (QRE). Self-tuning
EWA does as well as EWA in predicting behavior in new games, even though it has fewer
parameters, and fits reliably better than the QRE equilibrium benchmark
Game-theoretical control with continuous action sets
Motivated by the recent applications of game-theoretical learning techniques
to the design of distributed control systems, we study a class of control
problems that can be formulated as potential games with continuous action sets,
and we propose an actor-critic reinforcement learning algorithm that provably
converges to equilibrium in this class of problems. The method employed is to
analyse the learning process under study through a mean-field dynamical system
that evolves in an infinite-dimensional function space (the space of
probability distributions over the players' continuous controls). To do so, we
extend the theory of finite-dimensional two-timescale stochastic approximation
to an infinite-dimensional, Banach space setting, and we prove that the
continuous dynamics of the process converge to equilibrium in the case of
potential games. These results combine to give a provably-convergent learning
algorithm in which players do not need to keep track of the controls selected
by the other agents.Comment: 19 page
Collective states in social systems with interacting learning agents
We consider a social system of interacting heterogeneous agents with learning
abilities, a model close to Random Field Ising Models, where the random field
corresponds to the idiosyncratic willingness to pay. Given a fixed price,
agents decide repeatedly whether to buy or not a unit of a good, so as to
maximize their expected utilities. We show that the equilibrium reached by the
system depends on the nature of the information agents use to estimate their
expected utilities.Comment: 18 pages, 26 figure
Payoff Performance of Fictitious Play
We investigate how well continuous-time fictitious play in two-player games
performs in terms of average payoff, particularly compared to Nash equilibrium
payoff. We show that in many games, fictitious play outperforms Nash
equilibrium on average or even at all times, and moreover that any game is
linearly equivalent to one in which this is the case. Conversely, we provide
conditions under which Nash equilibrium payoff dominates fictitious play
payoff. A key step in our analysis is to show that fictitious play dynamics
asymptotically converges the set of coarse correlated equilibria (a fact which
is implicit in the literature).Comment: 16 pages, 4 figure
- …