Repeated Relationships with Limits on Information Processing

Many important strategic problems are characterized by repeated interactions among agents. There is a large literature in game theory and economics illustrating how considerations of future interactions can provide incentives for cooperation that would not be possible in one-shot interactions. Much of the work in repeated games assumes public monitoring: players observe precisely the same thing at each stage of the game. It is well-understood that even slight deviations from public monitoring increase dramatically the difficulty the problems players face in coordinating their actions. Repeated games with private monitoring incorporate differences in what players observe at each stage. Equilibria in repeated games with private monitoring, however, often seem unrealistic; the equilibrium strategies may be highly complex and very sensitive to the fine details of the stochastic relationship between players’ actions and observations. Furthermore, there is no realistic story about how players might arrive at their equilibrium strategies. We propose an alternative approach to understanding how people cooperate. Each player is endowed with a mental system that processes information: a mental system consists of a number of psychological states and a transition function between states that depends on observations made. In this world, a strategy is just a function from states to actions. Our framework has the following desirable properties: (i) players restrict attention to a relatively small set of simple strategies. (ii) the number of strategies that players compare is small enough that players might ultimately learn which perform well. We find that some mental systems allow agents to cooperate under a broad set of parameters, while others are not conducive to cooperation.Repeated Games, Private Monitoring, Mental States

Coordinated Multi-Agent Imitation Learning

We study the problem of imitation learning from demonstrations of multiple coordinating agents. One key challenge in this setting is that learning a good model of coordination can be difficult, since coordination is often implicit in the demonstrations and must be inferred as a latent variable. We propose a joint approach that simultaneously learns a latent coordination model along with the individual policies. In particular, our method integrates unsupervised structure learning with conventional imitation learning. We illustrate the power of our approach on a difficult problem of learning multiple policies for fine-grained behavior modeling in team sports, where different players occupy different roles in the coordinated team strategy. We show that having a coordination model to infer the roles of players yields substantially improved imitation loss compared to conventional baselines.Comment: International Conference on Machine Learning 201

Congestion, equilibrium and learning: The minority game

The minority game is a simple congestion game in which the players' main goal is to choose among two options the one that is adopted by the smallest number of players. We characterize the set of Nash equilibria and the limiting behavior of several well-known learning processes in the minority game with an arbitrary odd number of players. Interestingly, different learning processes provide considerably different predictions

Dynamics in atomic signaling games

We study an atomic signaling game under stochastic evolutionary dynamics. There is a finite number of players who repeatedly update from a finite number of available languages/signaling strategies. Players imitate the most fit agents with high probability or mutate with low probability. We analyze the long-run distribution of states and show that, for sufficiently small mutation probability, its support is limited to efficient communication systems. We find that this behavior is insensitive to the particular choice of evolutionary dynamic, a property that is due to the game having a potential structure with a potential function corresponding to average fitness. Consequently, the model supports conclusions similar to those found in the literature on language competition. That is, we show that efficient languages eventually predominate the society while reproducing the empirical phenomenon of linguistic drift. The emergence of efficiency in the atomic case can be contrasted with results for non-atomic signaling games that establish the non-negligible possibility of convergence, under replicator dynamics, to states of unbounded efficiency loss

Equilibrium Selection and the Rate of Convergence in Coordination Games with Simultaneous Play

We apply the dynamic stochastic framework proposed in the recent evolutionary literature to a class of coordination games played simultaneously by the entire population. In these games, payoffs whence best replies are determined by a summary statistic of the population strategy profile. We demonstrate that with simultaneous play, the equilibrium selection depends crucially on how best responses to the summary statistic remain piece-wise constant. In fact, all the strict Nash equilibria in the underlying stage game can be declared stochastically stable depending on how the best response mapping generates piece-wise constant best responses. Furthermore, we show that if the best response mapping is sufficiently asymmetric, the expected waiting time until the unique stochastically stable state is reached is of the same order as the mutation rate, even in the limit as the population size grows to infinity.equilibrium selection; stochastic stability; waiting time; rate of convergence

Information Structure Design in Team Decision Problems

We consider a problem of information structure design in team decision problems and team games. We propose simple, scalable greedy algorithms for adding a set of extra information links to optimize team performance and resilience to non-cooperative and adversarial agents. We show via a simple counterexample that the set function mapping additional information links to team performance is in general not supermodular. Although this implies that the greedy algorithm is not accompanied by worst-case performance guarantees, we illustrate through numerical experiments that it can produce effective and often optimal or near optimal information structure modifications

Deterministic Equations for Stochastic Spatial Evolutionary Games

Spatial evolutionary games model individuals who are distributed in a spatial domain and update their strategies upon playing a normal form game with their neighbors. We derive integro-differential equations as deterministic approximations of the microscopic updating stochastic processes. This generalizes the known mean-field ordinary differential equations and provide a powerful tool to investigate the spatial effects in populations evolution. The deterministic equations allow to identify many interesting features of the evolution of strategy profiles in a population, such as standing and traveling waves, and pattern formation, especially in replicator-type evolutions

On the Formation of Interaction Networks in Social Coordination Games

There are many situations where two interacting individuals can benefit from coordinating their actions. We examine the endogenous choice of partners in such social coordination games and the implications for resulting play. We model the interaction pattern as a network where individuals periodically have the discretion to add or sever links to other players. A player chooses whether to add or sever a link based on the (prospective) partner's past behavior. With such endogenous interaction patterns we see multiple stochastically stable states of play, including some that involve play of equilibria in the coordination game that are neither efficient nor risk dominant.