Evolutionary stable strategies in networked games: the influence of topology

Evolutionary game theory is used to model the evolution of competing strategies in a population of players. Evolutionary stability of a strategy is a dynamic equilibrium, in which any competing mutated strategy would be wiped out from a population. If a strategy is weak evolutionarily stable, the competing strategy may manage to survive within the network. Understanding the network-related factors that affect the evolutionary stability of a strategy would be critical in making accurate predictions about the behaviour of a strategy in a real-world strategic decision making environment. In this work, we evaluate the effect of network topology on the evolutionary stability of a strategy. We focus on two well-known strategies known as the Zero-determinant strategy and the Pavlov strategy. Zero-determinant strategies have been shown to be evolutionarily unstable in a well-mixed population of players. We identify that the Zero-determinant strategy may survive, and may even dominate in a population of players connected through a non-homogeneous network. We introduce the concept of `topological stability' to denote this phenomenon. We argue that not only the network topology, but also the evolutionary process applied and the initial distribution of strategies are critical in determining the evolutionary stability of strategies. Further, we observe that topological stability could affect other well-known strategies as well, such as the general cooperator strategy and the cooperator strategy. Our observations suggest that the variation of evolutionary stability due to topological stability of strategies may be more prevalent in the social context of strategic evolution, in comparison to the biological context

Analyzing Social Network Structures in the Iterated Prisoner's Dilemma with Choice and Refusal

The Iterated Prisoner's Dilemma with Choice and Refusal (IPD/CR) is an extension of the Iterated Prisoner's Dilemma with evolution that allows players to choose and to refuse their game partners. From individual behaviors, behavioral population structures emerge. In this report, we examine one particular IPD/CR environment and document the social network methods used to identify population behaviors found within this complex adaptive system. In contrast to the standard homogeneous population of nice cooperators, we have also found metastable populations of mixed strategies within this environment. In particular, the social networks of interesting populations and their evolution are examined.Comment: 37 pages, uuencoded gzip'd Postscript (1.1Mb when gunzip'd) also available via WWW at http://www.cs.wisc.edu/~smucker/ipd-cr/ipd-cr.htm

On Rational Delegations in Liquid Democracy

Liquid democracy is a proxy voting method where proxies are delegable. We propose and study a game-theoretic model of liquid democracy to address the following question: when is it rational for a voter to delegate her vote? We study the existence of pure-strategy Nash equilibria in this model, and how group accuracy is affected by them. We complement these theoretical results by means of agent-based simulations to study the effects of delegations on group's accuracy on variously structured social networks.Comment: 17 pages, 3 figures. This paper (without Appendix) appears in the proceedings of AAAI'1

Imitation and Efficient Contagion

In this paper we study the conditions under which efficient behavior can spread from a finite initial seed group to an infinite population living on a network. We formulate conditions on payoffs and network structure under which overall contagion occurs in arbitrary regular networks. Central in this process is the communication pattern among players who are confronted with the same decision, i.e. who are at the same distance from the initial seed group. The extent to which these agents interact among themselves (rather than with players who already have faced or subsequently will face the decision problem) is critical in the Prisoner’s Dilemma. In the Coordination Game the key element is the cohesion of the efficient cluster, a property which is different from the one identified in the Prisoner’s Dilemma. Additional results are obtained when we distinguish the interaction and information neighborhoods. Specifically, we find that contagion tends to be favored by fast neighborhood growth if an assumption of conservative behavior is made. We discuss our findings in relation to the notions of clustering, transitivity and cohesion.imitation, contagion, regular graphs, local interaction game

Defecting or not defecting: how to "read" human behavior during cooperative games by EEG measurements

Understanding the neural mechanisms responsible for human social interactions is difficult, since the brain activities of two or more individuals have to be examined simultaneously and correlated with the observed social patterns. We introduce the concept of hyper-brain network, a connectivity pattern representing at once the information flow among the cortical regions of a single brain as well as the relations among the areas of two distinct brains. Graph analysis of hyper-brain networks constructed from the EEG scanning of 26 couples of individuals playing the Iterated Prisoner's Dilemma reveals the possibility to predict non-cooperative interactions during the decision-making phase. The hyper-brain networks of two-defector couples have significantly less inter-brain links and overall higher modularity - i.e. the tendency to form two separate subgraphs - than couples playing cooperative or tit-for-tat strategies. The decision to defect can be "read" in advance by evaluating the changes of connectivity pattern in the hyper-brain network

Defection and extortion as unexpected catalysts of unconditional cooperation in structured populations

We study the evolution of cooperation in the spatial prisoner's dilemma game, where besides unconditional cooperation and defection, tit-for-tat, win-stay-lose-shift and extortion are the five competing strategies. While pairwise imitation fails to sustain unconditional cooperation and extortion regardless of game parametrization, myopic updating gives rise to the coexistence of all five strategies if the temptation to defect is sufficiently large or if the degree distribution of the interaction network is heterogeneous. This counterintuitive evolutionary outcome emerges as a result of an unexpected chain of strategy invasions. Firstly, defectors emerge and coarsen spontaneously among players adopting win-stay-lose-shift. Secondly, extortioners and players adopting tit-for-tat emerge and spread via neutral drift among the emerged defectors. And lastly, among the extortioners, cooperators become viable too. These recurrent evolutionary invasions yield a five-strategy phase that is stable irrespective of the system size and the structure of the interaction network, and they reveal the most unexpected mechanism that stabilizes extortion and cooperation in an evolutionary setting.Comment: 7 two-column pages, 5 figures; accepted for publication in Scientific Reports [related work available at http://arxiv.org/abs/1401.8294

Learning with Opponent-Learning Awareness

Multi-agent settings are quickly gathering importance in machine learning. This includes a plethora of recent work on deep multi-agent reinforcement learning, but also can be extended to hierarchical RL, generative adversarial networks and decentralised optimisation. In all these settings the presence of multiple learning agents renders the training problem non-stationary and often leads to unstable training or undesired final results. We present Learning with Opponent-Learning Awareness (LOLA), a method in which each agent shapes the anticipated learning of the other agents in the environment. The LOLA learning rule includes a term that accounts for the impact of one agent's policy on the anticipated parameter update of the other agents. Results show that the encounter of two LOLA agents leads to the emergence of tit-for-tat and therefore cooperation in the iterated prisoners' dilemma, while independent learning does not. In this domain, LOLA also receives higher payouts compared to a naive learner, and is robust against exploitation by higher order gradient-based methods. Applied to repeated matching pennies, LOLA agents converge to the Nash equilibrium. In a round robin tournament we show that LOLA agents successfully shape the learning of a range of multi-agent learning algorithms from literature, resulting in the highest average returns on the IPD. We also show that the LOLA update rule can be efficiently calculated using an extension of the policy gradient estimator, making the method suitable for model-free RL. The method thus scales to large parameter and input spaces and nonlinear function approximators. We apply LOLA to a grid world task with an embedded social dilemma using recurrent policies and opponent modelling. By explicitly considering the learning of the other agent, LOLA agents learn to cooperate out of self-interest. The code is at github.com/alshedivat/lola

Oscillatory dynamics in evolutionary games are suppressed by heterogeneous adaptation rates of players

Game dynamics in which three or more strategies are cyclically competitive, as represented by the rock-scissors-paper game, have attracted practical and theoretical interests. In evolutionary dynamics, cyclic competition results in oscillatory dynamics of densities of individual strategists. In finite-size populations, it is known that oscillations blow up until all but one strategies are eradicated if without mutation. In the present paper, we formalize replicator dynamics with players that have different adaptation rates. We show analytically and numerically that the heterogeneous adaptation rate suppresses the oscillation amplitude. In social dilemma games with cyclically competing strategies and homogeneous adaptation rates, altruistic strategies are often relatively weak and cannot survive in finite-size populations. In such situations, heterogeneous adaptation rates save coexistence of different strategies and hence promote altruism. When one strategy dominates the others without cyclic competition, fast adaptors earn more than slow adaptors. When not, mixture of fast and slow adaptors stabilizes population dynamics, and slow adaptation does not imply inefficiency for a player.Comment: 4 figure