114 research outputs found
AWESOME: A General Multiagent Learning Algorithm that Converges in Self-Play and Learns a Best Response Against Stationary Opponents
A satisfactory multiagent learning algorithm should, {\em at a minimum},
learn to play optimally against stationary opponents and converge to a Nash
equilibrium in self-play. The algorithm that has come closest, WoLF-IGA, has
been proven to have these two properties in 2-player 2-action repeated
games--assuming that the opponent's (mixed) strategy is observable. In this
paper we present AWESOME, the first algorithm that is guaranteed to have these
two properties in {\em all} repeated (finite) games. It requires only that the
other players' actual actions (not their strategies) can be observed at each
step. It also learns to play optimally against opponents that {\em eventually
become} stationary. The basic idea behind AWESOME ({\em Adapt When Everybody is
Stationary, Otherwise Move to Equilibrium}) is to try to adapt to the others'
strategies when they appear stationary, but otherwise to retreat to a
precomputed equilibrium strategy. The techniques used to prove the properties
of AWESOME are fundamentally different from those used for previous algorithms,
and may help in analyzing other multiagent learning algorithms also
BL-WoLF: A Framework For Loss-Bounded Learnability In Zero-Sum Games
We present BL-WoLF, a framework for learnability in repeated zero-sum games
where the cost of learning is measured by the losses the learning agent accrues
(rather than the number of rounds). The game is adversarially chosen from some
family that the learner knows. The opponent knows the game and the learner's
learning strategy. The learner tries to either not accrue losses, or to quickly
learn about the game so as to avoid future losses (this is consistent with the
Win or Learn Fast (WoLF) principle; BL stands for ``bounded loss''). Our
framework allows for both probabilistic and approximate learning. The resultant
notion of {\em BL-WoLF}-learnability can be applied to any class of games, and
allows us to measure the inherent disadvantage to a player that does not know
which game in the class it is in. We present {\em guaranteed
BL-WoLF-learnability} results for families of games with deterministic payoffs
and families of games with stochastic payoffs. We demonstrate that these
families are {\em guaranteed approximately BL-WoLF-learnable} with lower cost.
We then demonstrate families of games (both stochastic and deterministic) that
are not guaranteed BL-WoLF-learnable. We show that those families,
nevertheless, are {\em BL-WoLF-learnable}. To prove these results, we use a key
lemma which we derive
Settling Some Open Problems on 2-Player Symmetric Nash Equilibria
Over the years, researchers have studied the complexity of several decision
versions of Nash equilibrium in (symmetric) two-player games (bimatrix games).
To the best of our knowledge, the last remaining open problem of this sort is
the following; it was stated by Papadimitriou in 2007: find a non-symmetric
Nash equilibrium (NE) in a symmetric game. We show that this problem is
NP-complete and the problem of counting the number of non-symmetric NE in a
symmetric game is #P-complete.
In 2005, Kannan and Theobald defined the "rank of a bimatrix game"
represented by matrices (A, B) to be rank(A+B) and asked whether a NE can be
computed in rank 1 games in polynomial time. Observe that the rank 0 case is
precisely the zero sum case, for which a polynomial time algorithm follows from
von Neumann's reduction of such games to linear programming. In 2011, Adsul et.
al. obtained an algorithm for rank 1 games; however, it does not solve the case
of symmetric rank 1 games. We resolve this problem
- …