Partner Selection for the Emergence of Cooperation in Multi-Agent Systems Using Reinforcement Learning

Social dilemmas have been widely studied to explain how humans are able to cooperate in society. Considerable effort has been invested in designing artificial agents for social dilemmas that incorporate explicit agent motivations that are chosen to favor coordinated or cooperative responses. The prevalence of this general approach points towards the importance of achieving an understanding of both an agent's internal design and external environment dynamics that facilitate cooperative behavior. In this paper, we investigate how partner selection can promote cooperative behavior between agents who are trained to maximize a purely selfish objective function. Our experiments reveal that agents trained with this dynamic learn a strategy that retaliates against defectors while promoting cooperation with other agents resulting in a prosocial society.Comment:

Cooperation, Norms, and Revolutions: A Unified Game-Theoretical Approach

Cooperation is of utmost importance to society as a whole, but is often challenged by individual self-interests. While game theory has studied this problem extensively, there is little work on interactions within and across groups with different preferences or beliefs. Yet, people from different social or cultural backgrounds often meet and interact. This can yield conflict, since behavior that is considered cooperative by one population might be perceived as non-cooperative from the viewpoint of another. To understand the dynamics and outcome of the competitive interactions within and between groups, we study game-dynamical replicator equations for multiple populations with incompatible interests and different power (be this due to different population sizes, material resources, social capital, or other factors). These equations allow us to address various important questions: For example, can cooperation in the prisoner's dilemma be promoted, when two interacting groups have different preferences? Under what conditions can costly punishment, or other mechanisms, foster the evolution of norms? When does cooperation fail, leading to antagonistic behavior, conflict, or even revolutions? And what incentives are needed to reach peaceful agreements between groups with conflicting interests? Our detailed quantitative analysis reveals a large variety of interesting results, which are relevant for society, law and economics, and have implications for the evolution of language and culture as well

Coevolutionary games - a mini review

Prevalence of cooperation within groups of selfish individuals is puzzling in that it contradicts with the basic premise of natural selection. Favoring players with higher fitness, the latter is key for understanding the challenges faced by cooperators when competing with defectors. Evolutionary game theory provides a competent theoretical framework for addressing the subtleties of cooperation in such situations, which are known as social dilemmas. Recent advances point towards the fact that the evolution of strategies alone may be insufficient to fully exploit the benefits offered by cooperative behavior. Indeed, while spatial structure and heterogeneity, for example, have been recognized as potent promoters of cooperation, coevolutionary rules can extend the potentials of such entities further, and even more importantly, lead to the understanding of their emergence. The introduction of coevolutionary rules to evolutionary games implies, that besides the evolution of strategies, another property may simultaneously be subject to evolution as well. Coevolutionary rules may affect the interaction network, the reproduction capability of players, their reputation, mobility or age. Here we review recent works on evolutionary games incorporating coevolutionary rules, as well as give a didactic description of potential pitfalls and misconceptions associated with the subject. In addition, we briefly outline directions for future research that we feel are promising, thereby particularly focusing on dynamical effects of coevolutionary rules on the evolution of cooperation, which are still widely open to research and thus hold promise of exciting new discoveries.Comment: 24 two-column pages, 10 figures; accepted for publication in BioSystem

Leaders should not be conformists in evolutionary social dilemmas

The most common assumption in evolutionary game theory is that players should adopt a strategy that warrants the highest payoff. However, recent studies indicate that the spatial selection for cooperation is enhanced if an appropriate fraction of the population chooses the most common rather than the most profitable strategy within the interaction range. Such conformity might be due to herding instincts or crowd behavior in humans and social animals. In a heterogeneous population where individuals differ in their degree, collective influence, or other traits, an unanswered question remains who should conform. Selecting conformists randomly is the simplest choice, but it is neither a realistic nor the optimal one. We show that, regardless of the source of heterogeneity and game parametrization, socially the most favorable outcomes emerge if the masses conform. On the other hand, forcing leaders to conform significantly hinders the constructive interplay between heterogeneity and coordination, leading to evolutionary outcomes that are worse still than if conformists were chosen randomly. We conclude that leaders must be able to create a following for network reciprocity to be optimally augmented by conformity. In the opposite case, when leaders are castrated and made to follow, the failure of coordination impairs the evolution of cooperation.Comment: 7 two-column pages, 4 figures; accepted for publication in Scientific Reports [related work available at arXiv:1412.4113

Beyond pairwise strategy updating in the prisoner's dilemma game

In spatial games players typically alter their strategy by imitating the most successful or one randomly selected neighbor. Since a single neighbor is taken as reference, the information stemming from other neighbors is neglected, which begets the consideration of alternative, possibly more realistic approaches. Here we show that strategy changes inspired not only by the performance of individual neighbors but rather by entire neighborhoods introduce a qualitatively different evolutionary dynamics that is able to support the stable existence of very small cooperative clusters. This leads to phase diagrams that differ significantly from those obtained by means of pairwise strategy updating. In particular, the survivability of cooperators is possible even by high temptations to defect and over a much wider uncertainty range. We support the simulation results by means of pair approximations and analysis of spatial patterns, which jointly highlight the importance of local information for the resolution of social dilemmas.Comment: 9 two-column pages, 5 figures; accepted for publication in Scientific Report

Do Spouses Cooperate? And If Not: Why?

Models of household economics require an understanding of economic interactions in families. Social ties, repetition and reduced strategic uncertainty make social dilemmas in couples a very special case that needs to be empirically studied. In this paper we present results from a large economic experiment with 100 maritally living couples. Participants made decisions in a social dilemma with their partner and with a stranger. We predict behavior in this task with individual and couples' socio-demographic variables, efficiency preferences and couples' marital satisfaction. As opposed to models explaining behavior amongst strangers, the regressions on couples’ decisions highlight clear patterns concerning cooperation behavior which could inspire future household decision-making models.Noncooperative Games; Laboratory, Individual Behavior; Household Production and Intra-household Allocation

Effectiveness of conditional punishment for the evolution of public cooperation

Collective actions, from city marathons to labor strikes, are often mass-driven and subject to the snowball effect. Motivated by this, we study evolutionary advantages of conditional punishment in the spatial public goods game. Unlike unconditional punishers who always impose the same fines on defectors, conditional punishers do so proportionally with the number of other punishers in the group. Phase diagrams in dependence on the punishment fine and cost reveal that the two types of punishers cannot coexist. Spontaneous coarsening of the two strategies leads to an indirect territorial competition with the defectors, which is won by unconditional punishers only if the sanctioning is inexpensive. Otherwise conditional punishers are the victors of the indirect competition, indicating that under more realistic conditions they are indeed the more effective strategy. Both continuous and discontinuous phase transitions as well as tricritical points characterize the complex evolutionary dynamics, which is due to multipoint interactions that are introduced by conditional punishment. We propose indirect territorial competition as a generally applicable mechanism relying on pattern formation, by means of which spatial structure can be utilized by seemingly subordinate strategies to avoid evolutionary extinction

Evolutionary games in the multiverse

Evolutionary game dynamics of two players with two strategies has been studied in great detail. These games have been used to model many biologically relevant scenarios, ranging from social dilemmas in mammals to microbial diversity. Some of these games may in fact take place between a number of individuals and not just between two. Here, we address one-shot games with multiple players. As long as we have only two strategies, many results from two player games can be generalized to multiple players. For games with multiple players and more than two strategies, we show that statements derived for pairwise interactions do no longer hold. For two player games with any number of strategies there can be at most one isolated internal equilibrium. For any number of players $\boldsymbol{d}$ with any number of strategies n, there can be at most (d-1)^(n-1) isolated internal equilibria. Multiplayer games show a great dynamical complexity that cannot be captured based on pairwise interactions. Our results hold for any game and can easily be applied for specific cases, e.g. public goods games or multiplayer stag hunts

Evolutionary Multiplayer Games

Evolutionary game theory has become one of the most diverse and far reaching theories in biology. Applications of this theory range from cell dynamics to social evolution. However, many applications make it clear that inherent non-linearities of natural systems need to be taken into account. One way of introducing such non-linearities into evolutionary games is by the inclusion of multiple players. An example is of social dilemmas, where group benefits could e.g.\ increase less than linear with the number of cooperators. Such multiplayer games can be introduced in all the fields where evolutionary game theory is already well established. However, the inclusion of non-linearities can help to advance the analysis of systems which are known to be complex, e.g. in the case of non-Mendelian inheritance. We review the diachronic theory and applications of multiplayer evolutionary games and present the current state of the field. Our aim is a summary of the theoretical results from well-mixed populations in infinite as well as finite populations. We also discuss examples from three fields where the theory has been successfully applied, ecology, social sciences and population genetics. In closing, we probe certain future directions which can be explored using the complexity of multiplayer games while preserving the promise of simplicity of evolutionary games.Comment: 14 pages, 2 figures, review pape