Higher-Order Uncoupled Dynamics Do Not Lead to Nash Equilibrium \unicode{x2014} Except When They Do

The framework of multi-agent learning explores the dynamics of how individual agent strategies evolve in response to the evolving strategies of other agents. Of particular interest is whether or not agent strategies converge to well known solution concepts such as Nash Equilibrium (NE). Most ``fixed order'' learning dynamics restrict an agent's underlying state to be its own strategy. In ``higher order'' learning, agent dynamics can include auxiliary states that can capture phenomena such as path dependencies. We introduce higher-order gradient play dynamics that resemble projected gradient ascent with auxiliary states. The dynamics are ``payoff based'' in that each agent's dynamics depend on its own evolving payoff. While these payoffs depend on the strategies of other agents in a game setting, agent dynamics do not depend explicitly on the nature of the game or the strategies of other agents. In this sense, dynamics are ``uncoupled'' since an agent's dynamics do not depend explicitly on the utility functions of other agents. We first show that for any specific game with an isolated completely mixed-strategy NE, there exist higher-order gradient play dynamics that lead (locally) to that NE, both for the specific game and nearby games with perturbed utility functions. Conversely, we show that for any higher-order gradient play dynamics, there exists a game with a unique isolated completely mixed-strategy NE for which the dynamics do not lead to NE. These results build on prior work that showed that uncoupled fixed-order learning cannot lead to NE in certain instances, whereas higher-order variants can. Finally, we consider the mixed-strategy equilibrium associated with coordination games. While higher-order gradient play can converge to such equilibria, we show such dynamics must be inherently internally unstable

Universality of weak selection

Weak selection, which means a phenotype is slightly advantageous over another, is an important limiting case in evolutionary biology. Recently it has been introduced into evolutionary game theory. In evolutionary game dynamics, the probability to be imitated or to reproduce depends on the performance in a game. The influence of the game on the stochastic dynamics in finite populations is governed by the intensity of selection. In many models of both unstructured and structured populations, a key assumption allowing analytical calculations is weak selection, which means that all individuals perform approximately equally well. In the weak selection limit many different microscopic evolutionary models have the same or similar properties. How universal is weak selection for those microscopic evolutionary processes? We answer this question by investigating the fixation probability and the average fixation time not only up to linear, but also up to higher orders in selection intensity. We find universal higher order expansions, which allow a rescaling of the selection intensity. With this, we can identify specific models which violate (linear) weak selection results, such as the one--third rule of coordination games in finite but large populations.Comment: 12 pages, 3 figures, accepted for publication in Physical Review

Cyclic game dynamics driven by iterated reasoning

Recent theories from complexity science argue that complex dynamics are ubiquitous in social and economic systems. These claims emerge from the analysis of individually simple agents whose collective behavior is surprisingly complicated. However, economists have argued that iterated reasoning--what you think I think you think--will suppress complex dynamics by stabilizing or accelerating convergence to Nash equilibrium. We report stable and efficient periodic behavior in human groups playing the Mod Game, a multi-player game similar to Rock-Paper-Scissors. The game rewards subjects for thinking exactly one step ahead of others in their group. Groups that play this game exhibit cycles that are inconsistent with any fixed-point solution concept. These cycles are driven by a "hopping" behavior that is consistent with other accounts of iterated reasoning: agents are constrained to about two steps of iterated reasoning and learn an additional one-half step with each session. If higher-order reasoning can be complicit in complex emergent dynamics, then cyclic and chaotic patterns may be endogenous features of real-world social and economic systems.Comment: 8 pages, 4 figures, and supplementary informatio

The Effect of Incentives and Meta-incentives on the Evolution of Cooperation

Although positive incentives for cooperators and/or negative incentives for free-riders in social dilemmas play an important role in maintaining cooperation, there is still the outstanding issue of who should pay the cost of incentives. The second-order free-rider problem, in which players who do not provide the incentives dominate in a game, is a well-known academic challenge. In order to meet this challenge, we devise and analyze a meta-incentive game that integrates positive incentives (rewards) and negative incentives (punishments) with second-order incentives, which are incentives for other players' incentives. The critical assumption of our model is that players who tend to provide incentives to other players for their cooperative or non-cooperative behavior also tend to provide incentives to their incentive behaviors. In this paper, we solve the replicator dynamics for a simple version of the game and analytically categorize the game types into four groups. We find that the second-order free-rider problem is completely resolved without any third-order or higher (meta) incentive under the assumption. To do so, a second-order costly incentive, which is given individually (peer-to-peer) after playing donation games, is needed. The paper concludes that (1) second-order incentives for first-order reward are necessary for cooperative regimes, (2) a system without first-order rewards cannot maintain a cooperative regime, (3) a system with first-order rewards and no incentives for rewards is the worst because it never reaches cooperation, and (4) a system with rewards for incentives is more likely to be a cooperative regime than a system with punishments for incentives when the cost-effect ratio of incentives is sufficiently large. This solution is general and strong in the sense that the game does not need any centralized institution or proactive system for incentives. (authors' abstract

Cooperation driven by mutations in multi-person Prisoner's Dilemma

The n-person Prisoner's Dilemma is a widely used model for populations where individuals interact in groups. The evolutionary stability of populations has been analysed in the literature for the case where mutations in the population may be considered as isolated events. For this case, and assuming simple trigger strategies and many iterations per game, we analyse the rate of convergence to the evolutionarily stable populations. We find that for some values of the payoff parameters of the Prisoner's Dilemma this rate is so low that the assumption, that mutations in the population are infrequent on that timescale, is unreasonable. Furthermore, the problem is compounded as the group size is increased. In order to address this issue, we derive a deterministic approximation of the evolutionary dynamics with explicit, stochastic mutation processes, valid when the population size is large. We then analyse how the evolutionary dynamics depends on the following factors: mutation rate, group size, the value of the payoff parameters, and the structure of the initial population. In order to carry out the simulations for groups of more than just a few individuals, we derive an efficient way of calculating the fitness values. We find that when the mutation rate per individual and generation is very low, the dynamics is characterised by populations which are evolutionarily stable. As the mutation rate is increased, other fixed points with a higher degree of cooperation become stable. For some values of the payoff parameters, the system is characterised by (apparently) stable limit cycles dominated by cooperative behaviour. The parameter regions corresponding to high degree of cooperation grow in size with the mutation rate, and in number with the group size.Comment: 22 pages, 7 figures. Accepted for publication in Journal of Theoretical Biolog

Evolutionary dynamics of higher-order interactions in social networks

We live and cooperate in networks. However, links in networks only allow for pairwise interactions, thus making the framework suitable for dyadic games, but not for games that are played in groups of more than two players. Here, we study the evolutionary dynamics of a public goods game in social systems with higher-order interactions. First, we show that the game on uniform hypergraphs corresponds to the replicator dynamics in the well-mixed limit, providing an exact theoretical foundation to study cooperation in networked groups. Secondly, we unveil how the presence of hubs and the coexistence of interactions in groups of different sizes affects the evolution of cooperation. Finally, we apply the proposed framework to extract the actual dependence of the synergy factor on the size of a group from real-world collaboration data in science and technology. Our work provides a way to implement informed actions to boost cooperation in social groups

Edge effects in game theoretic dynamics of spatially structured tumours

Background: Analysing tumour architecture for metastatic potential usually focuses on phenotypic differences due to cellular morphology or specific genetic mutations, but often ignore the cell's position within the heterogeneous substructure. Similar disregard for local neighborhood structure is common in mathematical models. Methods: We view the dynamics of disease progression as an evolutionary game between cellular phenotypes. A typical assumption in this modeling paradigm is that the probability of a given phenotypic strategy interacting with another depends exclusively on the abundance of those strategies without regard local heterogeneities. We address this limitation by using the Ohtsuki-Nowak transform to introduce spatial structure to the go vs. grow game. Results: We show that spatial structure can promote the invasive (go) strategy. By considering the change in neighbourhood size at a static boundary -- such as a blood-vessel, organ capsule, or basement membrane -- we show an edge effect that allows a tumour without invasive phenotypes in the bulk to have a polyclonal boundary with invasive cells. We present an example of this promotion of invasive (EMT positive) cells in a metastatic colony of prostate adenocarcinoma in bone marrow. Interpretation: Pathologic analyses that do not distinguish between cells in the bulk and cells at a static edge of a tumour can underestimate the number of invasive cells. We expect our approach to extend to other evolutionary game models where interaction neighborhoods change at fixed system boundaries.Comment: 14 pages, 3 figures; restructured abstract, added histology to fig. 1, added fig. 3, discussion of EMT introduced and cancer biology expande