Self-tuning experience weighted attraction learning in games

Self-tuning experience weighted attraction (EWA) is a one-parameter theory of learning in games. It addresses a criticism that an earlier model (EWA) has too many parameters, by fixing some parameters at plausible values and replacing others with functions of experience so that they no longer need to be estimated. Consequently, it is econometrically simpler than the popular weighted fictitious play and reinforcement learning models. The functions of experience which replace free parameters “self-tune” over time, adjusting in a way that selects a sensible learning rule to capture subjects’ choice dynamics. For instance, the self-tuning EWA model can turn from a weighted fictitious play into an averaging reinforcement learning as subjects equilibrate and learn to ignore inferior foregone payoffs. The theory was tested on seven different games, and compared to the earlier parametric EWA model and a one-parameter stochastic equilibrium theory (QRE). Self-tuning EWA does as well as EWA in predicting behavior in new games, even though it has fewer parameters, and fits reliably better than the QRE equilibrium benchmark

Individual Differences in EWA Learning with Partial Payoff Information

We extend experience-weighted attraction (EWA) learning to games in which only the set of possible foregone payoffs from unchosen strategies are known, and estimate parameters separately for each player to study heterogeneity. We assume players estimate unknown foregone payoffs from a strategy, by substituting the last payoff actually received from that strategy, by clairvoyantly guessing the actual foregone payoff, or by averaging the set of possible foregone payoffs conditional on the actual outcomes. All three assumptions improve predictive accuracy of EWA. Individual parameter estimates suggest that players cluster into two separate subgroups (which differ from traditional reinforcement and belief learning)

Games of incomplete information without common knowledge priors

We relax the assumption that priors are common knowledge, in the standard model of games of incomplete information. We make the realistic assumption that the players are boundedly rational: they base their actions on finite-order belief hierarchies. When the different layers of beliefs are independent of each other, we can retain Harsányi’s type-space, and we can define straightforward generalizations of Bayesian Nash Equilibrium (BNE) and Rationalizability in our context. Since neither of these concepts is quite satisfactory, we propose a hybrid concept, Mirage Equilibrium, providing us with a practical tool to work with inconsistent belief hierarchies. When the different layers of beliefs are correlated, we must enlarge the type-space to include the parametric beliefs. This presents us with the difficulty of the inherent openness of finite belief subspaces. Appealing to bounded rationality once more, we posit that the players believe that their opponent holds a belief hierarchy one layer shorter than they do and we provide alternative generalizations of BNE and Rationalizability. Finally, we show that, when beliefs are degenerate point beliefs, the definition of Mirage Equilibrium coincides with that of the generalized BNE.inconsistent beliefs, games of incomplete information, finite belief hierarchy

A Projective Simulation Scheme for Partially-Observable Multi-Agent Systems

We introduce a kind of partial observability to the projective simulation (PS) learning method. It is done by adding a belief projection operator and an observability parameter to the original framework of the efficiency of the PS model. I provide theoretical formulations, network representations, and situated scenarios derived from the invasion toy problem as a starting point for some multi-agent PS models.Comment: 28 pages, 21 figure

A Note on the Equivalence of Rationalizability Concepts in Generalized Nice Games

Moulin (1984) describes the class of nice games for which the solution concept of point-rationalizability coincides with iterated elimination of strongly dominated strategies. As a consequence nice games have the desirable property that all rationalizability concepts determine the same strategic solution. However, nice games are characterized by rather strong assumptions. For example, only single-valued best responses are admitted and the individual strategy sets have to be convex and compact subsets of the real line R1. This note shows that equivalence of all rationalizability concepts can be extended to multi-valued best response correspondences. The surprising finding is that equivalence does not hold for individual strategy sets that are compact and convex subsets of Rn with n>1.

Once Beaten, Never Again: Imitation in Two-Player Potential Games

We show that in symmetric two-player exact potential games, the simple decision rule "imitate-if-better" cannot be beaten by any strategy in a repeated game by more than the maximal payoff difference of the one-period game. Our results apply to many interesting games including examples like 2x2 games, Cournot duopoly, price competition, public goods games, common pool resource games, and minimum effort coordination games.Imitate-the-best, learning, exact potential games, symmetric games, relative payoffs, zero-sum games