Evolutionary game theory has traditionally assumed that all individuals in a
population interact with each other between reproduction events. We show that
eliminating this restriction by explicitly considering the time scales of
interaction and selection leads to dramatic changes in the outcome of
evolution. Examples include the selection of the inefficient strategy in the
Harmony and Stag-Hunt games, and the disappearance of the coexistence state in
the Snowdrift game. Our results hold for any population size and in the
presence of a background of fitness.Comment: Final version with minor changes, accepted for publication in
  Physical Review Letter

Cuesta, José A.

Pérez Roca, Carlos

Sánchez, Angel

English

arXiv

4 pages, 3 figures.-- PACS nrs.: 02.50.Le, 05.45.−a, 89.65.−s.Evolutionary game theory has traditionally assumed that all individuals in a population interact with each other between reproduction events. We show that eliminating this restriction by explicitly considering the time scales of interaction and selection leads to dramatic changes in the outcome of evolution. Examples include the selection of the inefficient strategy in the Harmony and Stag-Hunt games, and the disappearance of the coexistence state in the Snowdrift game. Our results hold for any population size and in more general situations with additional factors influencing fitness.Work supported by MEC (Spain) Grants No. BFM2003-0180, No. BFM2003-07749-C05-01, No. FIS2004-1001 and No. NAN2004-9087-C03-03 and by Comunidad de Madrid (Spain) Grants No. UC3M-FI-05-007, SIMUMAT-CM and MOSSNOHO-CM.Publicad

Name not available

Time scales in evolutionary dynamics

e-Archivo (Univ. Carlos III de Madrid e-Archivo)

Time Scales in Evolutionary DynamicsCarlos P. Roca,1,* Jose´ A. Cuesta,1,* and Angel Sa´nchez1,2,*1Grupo Interdisciplinar de Sistemas Complejos (GISC), Departamento de Matema´ticas, Universidad Carlos III de Madrid,28911 Legane´s, Madrid, Spain2Instituto de Biocomputacio´n y Fı´sica de Sistemas Complejos (BIFI), Universidad de Zaragoza, 50009 Zaragoza, Spain(Received 23 June 2006; published 12 October 2006)Evolutionary game theory has traditionally assumed that all individuals in a population interact witheach other between reproduction events. We show that eliminating this restriction by explicitly consid-ering the time scales of interaction and selection leads to dramatic changes in the outcome of evolution.Examples include the selection of the inefficient strategy in the Harmony and Stag-Hunt games, and thedisappearance of the coexistence state in the Snowdrift game. Our results hold for any population size andin more general situations with additional factors influencing fitness.DOI: 10.1103/PhysRevLett.97.158701 PACS numbers: 02.50.Le, 05.45.a, 89.65.sEvolutionary game theory is the mathematical frame-work for modeling evolution in biological, social, andeconomical systems [1–3], and is deeply connected todynamical systems theory and statistical mechanics [4–11]. In the standard setup of evolutionary game theory,strategies available for the game are represented by afraction of individuals in the population. Individuals theninteract according to the rules of the game, and the soearned payoffs determine the frequencies of the next gen-eration (i.e., payoffs represent reproductive fitness).Customarily, most evolutionary game studies make theadditional assumption that individuals play many timesand with all other players before reproduction takes place,so that payoffs, equivalently fitness, are given by the meandistribution of types in the population. This is also thesituation for the so-called round-robin tournament, inwhich each individual plays once with every other. Bothhypotheses, common in biological evolution, imply thatselection occurs much more slowly than the interactionbetween individuals. Although recent experimental studiesshow that this may not always be the case in biology [12–14], it is clear that in cultural evolution or social learningthe time scale of selection is much closer to the time scaleof interaction. The effects of this mixing of scales cannotbe disregarded [15], and then it is natural to ask about theconsequences of the above assumption and the effect ofrelaxing it. Though the main field of application of ourwork is social and cultural evolution, we maintain the usuallanguage of evolutionary biology, to avoid introducing newterminology.In this Letter, we show that rapid selection affects evolu-tionary dynamics in such a dramatic way that for somegames it even changes the stability of equilibria. In order tomake explicit the relation between selection and interac-tion time scales, we use discrete-time dynamics. We followMoran dynamics [16], as this is the proper way to describeevolution of discrete generations in the field of populationdynamics [17]. Specifically, we choose the frequency-dependent version of the Moran dynamics introduced by[18], which allows to consider an evolutionary game in thisdynamical context: N individuals interact by playing agame and reproduce by selecting one individual, withprobability proportional to the payoff, to duplicate andsubstitute a randomly chosen individual. The payoff ofevery player is set to zero after each reproduction event,and this two-step cycle is repeated until the populationeventually stabilizes. This stochastic dynamics is discretein both population and time, while keeping the populationsize constant over time. Interestingly, this microscopicdynamics leads to a difference equation that has beenproposed as an adjusted [1] or discrete-time [2] analogousof the replicator equation, widely used in evolutionarygame theory (see [9] for a recent, detailed discussion ofthis issue). Additionally, we note that for social applica-tions, reproduction may be also interpreted as a learningprocess, in which individuals do not die but instead changethe way they behave or their strategies.Time scales enter the dynamics through the interactionstep, affecting the way fitness is obtained. We introduce anew interaction scheme, by allowing an integer number sof randomly chosen pairs of individuals to play consec-utively the game, between reproduction events. Thus, sequals the ratio between selection and interaction timescales. This is the crucial parameter in our model. Thelimit value of s  1 means that both time scales are equal;greater finite values, s > 1, correspond to the selectiontime scale being slower than the interaction time scale,and the limit value of s ! 1 recovers the round-robinprocedure. In fact, the equivalence of the limit s ! 1 tothe round-robin scheme points to the latter being a form of‘‘mean-field’’ theory, in which individuals reproduce soslowly that it makes sense to replace pairwise interactionsby the interaction with the ‘‘average player’’.As for the games, we will consider the important case ofsymmetric 2 2 games, in which the payoffs are given bythe following matrix 121 2a bc d ;(1)1whose rows give the payoff obtained by each strategy whenconfronted with the other or itself, and a, b, c, d > 0. Let nbe the number of individuals using strategy 1, also referredas type 1 individuals. After each reproduction event n maystay the same, increase by one, or decrease by one. Thecorresponding transition probabilities will depend on thefitness earned by each type during the interaction step andon their frequencies. As both quantities will depend ulti-mately on n, we have a Markov process with a tridiagonaltransition matrix (i.e., a birth-death process [19]) whosenonzero coefficients are Pn;n1  nN EF2F1  F2 jn; Pn;n1  N  nN EF1F1  F2 jn; (2)and Pn;n  1 Pn;n1  Pn;n1. Fi is the payoff obtained by all players of type i, and Ejn denotes the expected valueconditioned to a population of n individuals of type 1.We stress that the parameter s enters through the expected values of the relative fitness of each type (2). Indeed, if werestrict ourselves to the limit s ! 1, these expected values are given directly by the pairing probabilities and the payoffscorresponding to each pair EF1F1  F2 jn nn 1a nN  nbnn 1a nN  nb c  N  nN  n 1d (3)as would be obtained by the round-robin scheme. However,as we will see below, finite values of s often lead to resultscompletely different from this special case.The solution to the birth-death process can be obtainedin a standard manner [19]. Denoting by pn the fixationprobability of type 1 (i.e., the probability of ending up in apopulation with all individuals of type 1) when startingfrom a population with n players of this type, we have pn  Pn;n1pn1  Pn;npn  Pn;n1pn1; (4)with p0  0 and pN  1. The solution to this equation isgiven by pnQn=QN; Qn1Xn1j1Yji1Pi;i1Pi;i1; n>1; (5)with Q1  1. As stated above, the interesting case arisesfor finite values of the parameter s. For general s, astraightforward combinatorial analysis of all the possiblesequences of s pairings leads to EF1F1  F2 jn Xsi0Xsij02sijs!nsjn iiN  nsiN  n 1ji!j!s i j!NN  1s2ai bs i j2ai 2dj b cs i j: (6)This lengthy combinatorial expression reduces, in the limitcase s  1 of extremely rapid selection, to Pn;n1  nN  nNN  11 c bc bnN 1NPn;n1  nN  nNN  12bb cc bc bnN 1N:(7)The above equations are the first hint of the effect of timescales. For this extreme case, only the coefficients of theskew diagonal of (1) appear in (7). Therefore, if the timescale of selection equals that of interaction, the evolution-ary outcome of any game will be determined solely by theperformance of each strategy when confronted with theother, and independently of the results when dealing withitself. However, as we will now see, there are anothernontrivial, important differences.To make our study as general as possible, we haveanalyzed all 12 nonequivalent symmetric 2 2 games[20]. These games can be further classified into threecategories, according to their Nash equilibria and theirdynamical behavior under the replicator dynamics withround-robin interaction:(i) There are six games with a > c and b > d, or a < cand b < d. They have a unique Nash equilibrium, cor-responding to the dominant pure strategy. This equilibriumis the global attractor of the round-robin replicatordynamics.(ii) There are three games with a > c and b < d. Theyhave several Nash equilibria, one of them with a mixedstrategy. With the round-robin replicator dynamics, thismixed strategy equilibrium is an unstable point, whichacts as the boundary between the basins of attraction ofthe two pure strategies, which are the attractors.(iii) The remaining three games have a < c and b > d.They have several Nash equilibria, one of them with amixed strategy. This mixed strategy equilibrium is theglobal attractor of the round-robin replicator dynamics.The two pure strategies are unstable in this case.Let us first consider an example of class (i), namely, theHarmony game [21] (a  1, b  0:25, c  0:75, d 0:01). This is a no-conflict game, in which all playersobtain the maximum payoff by following strategy 1. AsFig. 1(a) shows, this is the result for large values of s, witha fixation probability pn 	 1 for almost all n. On the otherhand, Fig. 1(a) also shows that, for small s, strategy 2, i.e.,the inefficient (in the sense of lowest payoff) one, isselected by the dynamics, unless starting from initial con-ditions with almost all individuals of type 1.For class (ii), a good paradigm is the Stag-Hunt game[22] (a  1, b  0:01, c  0:8, d  0:2), which is a co-ordination game: Strategy 1 maximizes the mutual benefit,2whereas strategy 2 minimizes the risk of loss, and theconflict results from having to choose between these twooptions. As Fig. 1(b) reveals, the round-robin result isobtained for large s: both strategies are attractors, withthe basin boundary located at the frequency correspondingto the mixed strategy equilibrium, i.e., x  d b=ac d b 	 0:49. However, for small values of s thisboundary shifts to greater frequency values, thus reflectingan advantage of strategy 2. In the extreme s  1 case thisstrategy becomes the unique attractor.It is interesting to note that Fig. 1 shows that there is nota general crossover at s 	 N. In the Harmony game, theround-robin regime is mostly reached for s ’ 10  N,whereas in the Stag-Hunt game this does not happen untils ’ 100  N.Finally, let us consider the Snowdrift game [23] (a  1,b  0:2, c  1:8, d  0:01) as an example of class (iii).This is also a dilemma game, as each player has to choosebetween strategy 1, which maximizes the population gain,and strategy 2, which gives individuals the maximum pay-off by exploiting the opponent. With round-robin dynamicsboth strategies coexists in the long run, with frequenciescorresponding to the mixed strategy equilibrium. However,our dynamics can never maintain coexistence indefinitely,because by construction one of the absorbing states (allplayers of type 1 or all of type 2) will be reached sooner orlater with probability 1. Nonetheless, it is possible to studythe duration of metastable states by using the mean time ineach population state before absorbtion, tn [19]. Figure 2shows the results for two values of s and a broad range ofinitial conditions. For s large (s  100), the populationstays for a long time near the value corresponding to themixed strategy equilibrium x  d b=a c db 	 0:19, independently of the initial number of type 1individuals. A smaller value of s  10 (not shown) inducesa shift of the metastable equilibrium to smaller values of n,again almost independently of the initial conditions.Finally, for an even smaller value of s (s  5), there isno metastable equilibrium, but a fluctuation towards thex  0 absorbing state, which clearly depends on the initialconditions.Having given examples of all three classes, we willsummarize the rest of our study by saying that the remain-ing 2 2 games behave in a similar way, with rapidselection (small s) favoring in all cases the type that hasthe greatest coefficient in the skew diagonal of the payoffmatrix. For the remaining five games of class (i) this resultsin a reinforcement of the dominant strategy (the Prisoner’sDilemma [24] being a prominent example). The other twogames of class (ii) exhibit once again a displacement of thebasins of attraction, whereas the other two class (iii) gamesdisplay the suppression of the coexistence state in favor ofone of the strategies. We thus see that rapid selection leadsvery generally to outcomes entirely different from those ofround-robin dynamics.It is important to realize that our results do not changequalitatively with the system size. Considering for instancethe Stag-Hunt game, the change in the basins of attractionis practically independent of the population size. The maineffect of working with larger sizes is a steeper transitionbetween the basins of attraction. Indeed, due to the inherentstochasticity of finite population sizes, smaller populationshave a more blurred basin boundary, with points in eachbasin having an increasing nonzero probability of reachingthe other basin [25]. Our results for all other symmetric2 2 games are equally robust. In fact, for very rapidselection, s  1, the limit N ! 1 of the transition proba-bilities, Eq. (7), shows that they depend only on the fre-quencies of both types.It could be argued that in our model only s pairs ofindividuals play in each round, resulting in a very smalleffective population, this being the fundamental cause ofthe reported results. To probe into this issue, we haveintroduced a background of fitness [7,18], so that everyplayer has an intrinsic probability of being selected, re-gardless of the outcome of the game, and thus guaranteeing00.020.040.06(a)0 20 40 60 80 100Number of type 1 individuals, n00.020.040.06Mean time in each state, t n(b)FIG. 2 (color online). Mean time before fixation in theSnowdrift game for s  5 (a) and 100 (b). Initial values of nequal to 20 (), 50 (4), and 80 (+). Curves in (b) overlap.Population size N  100.00.51(a)0 20 40 60 80 100Initial number of type 1 individuals, n00.51Fixation probability of type 1, p n(b)FIG. 1 (color online). Fixation probabilities in the games(a) Harmony and (b) Stag-Hunt for s  1 (), 5 (), 10 (4),100 (+), or ! 1 (). In (a), curves overlap for s  10, 100, and! 1. Population size N  100.3a population of N players. Indeed, in most applications,agents interact through more than one type of game andthere are external contributions to fitness (environmentalfactors, fashions, or media influence in a social context,etc.). Let fb be the normalized fitness background, so thateach individual has a background of fitness sfb=N beforeselection takes place; fb  1 means that the overall fitnesscoming from the game and from the background are ap-proximately equal, for every value of s and N. Figure 3shows the results for the Stag-Hunt game. A small fitnessbackground of fb  0:1 gives fixation probabilities verysimilar to those with fb  0 [Fig. 2(b)]. For larger values,fb  1, the displacement of the basin boundary is smaller,but still perfectly noticeable. And a very large fitnessbackground, fb * 10 (not shown), drives the dynamics torandom selection for every value of s, because in this casethe influence of the game is almost negligible. Again, forthe remaining symmetric 2 2 games, our conclusionsremain valid as well in the presence of a background offitness. Consequently, our results are not merely due to afinite size effect of a small effective population of players.In summary, we have proven that considering indepen-dent interaction and selection time scales leads to highlynontrivial, counterintuitive results. We have demonstratedthe generality of this conclusion by considering all sym-metric 2 2 games and showing that rapid selection maylead to changes of the asymptotically selected equilibria, tochanges of the basins of attraction of equilibria, or tosuppression of long-lived metastable equilibria. This resulthas major implications for applying evolutionary gametheory to model a specific problem, as the assumption ofslow selection and consequently of round-robin dynamicsmay or may not be correct. Indeed, as the example in [15]shows, rapid selection may lead to the understanding ofproblems where Darwinian, individual evolution wasthought not to play a role because round-robin dynamicswas used. We envisage that successful modeling in rapidlychanging environments, such as social or (sub-)culturedynamics, will need a careful consideration of the involvedtime scales along the lines discussed here.We thank R. Toral for a critical reading of the manu-script. Work supported by MEC (Spain) GrantsNo. BFM2003-0180, No. BFM2003-07749-C05-01,No. FIS2004-1001 and No. NAN2004-9087-C03-03 andby Comunidad de Madrid (Spain) Grants No. UC3M-FI-05-007, SIMUMAT-CM and MOSSNOHO-CM.*Electronic address: http://www.gisc.es[1] J. Maynard Smith, Evolution and the Theory of Games(Cambridge University Press, Cambridge, England, 1982).[2] J. Hofbauer and K. Sigmund, Evolutionary Games andPopulation Dynamics (Cambridge University Press,Cambridge, England, 1998).[3] C. F. Camerer, Behavioral Game Theory (PrincetonUniversity, Princeton, NJ, 2003).[4] G. Szabo´ and C. Hauert, Phys. Rev. Lett. 89, 118101(2002).[5] A. Traulsen et al., Phys. Rev. Lett. 93, 028701 (2004).[6] M. G. Zimmermann et al., Phys. Rev. E 69, 065102(2004).[7] J. C. Claussen and A. Traulsen, Phys. Rev. E 71, 025101(2005).[8] F. C. Santos and J. M. Pacheco, Phys. Rev. Lett. 95,098104 (2005).[9] A. Traulsen et al., Phys. Rev. Lett. 95, 238701 (2005).[10] D. Ariosa and H. Fort, Phys. Rev. E 71, 016132 (2005).[11] C. Hauert and G. Szabo´, Am. J. Phys. 73, 405 (2005).[12] A. P. Hendry and M. T. Kinnison, Evolution (Lawrence,Kansas) 53, 1637 (1999).[13] A. P. Hendry et al., Science 290, 516 (2000).[14] T. Yoshida et al., Nature (London) 424, 303 (2003).[15] A. Sa´nchez and J. A. Cuesta, J. Theor. Biol. 235, 233(2005).[16] P. A. P. Moran, The Statistical Processes of EvolutionaryTheory (Clarendon, Oxford, 1962).[17] J. Roughgarden, Theory of Population Genetics andEvolutionary Ecology: An Introduction (MacMillanPublishing Co., New York, 1979).[18] M. A. Nowak et al., Nature (London) 428, 646 (2004).[19] S. Karlin and H. M. Taylor, A First Course in StochasticProcesses (Academic, New York, 1975), 2nd ed.[20] A. Rapoport and M. Guyer, General Systems 11, 203(1966).[21] A. N. Licht, Yale J. Int. Law 24, 61 (1999).[22] B. Skyrms, The Stag Hunt and the Evolution of SocialStructure (Cambridge University Press, Cambridge,England, 2003).[23] R. Sugden, Economics of Rights, Cooperation, and Wel-fare (Palgrave Macmillan, Hampshire, 2004), 2nd ed.[24] R.Axelrod and W. D. Hamilton, Science 211, 1390 (1981).[25] A. Cabrales, Inter. Econ. Rev. 41, 451 (2000).00.51(a)0 20 40 60 80 100Initial number of type 1 individuals, n00.51Fixation probability of type 1, p n(b)FIG. 3 (color online). Fixation probability in the Stag-Huntgame with a background of fitness fb  0:1 (a) and 1 (b). Valuesof s: 5 (), 10 (4), 100 (+). Population size N  100.4

http://dx.doi.org/10.1103/PhysRevLett.97.158701

Time Scales in Evolutionary Dynamics

Abstract

Similar works

Full text

Available Versions

Name not available

Name not available

e-Archivo (Univ. Carlos III de Madrid e-Archivo)