We here establish an upper bound on the probability for deviations of a Markov population process from its mean-field approximation.

Jörgen Weibull

Michel Benaïm

English

Research Papers in Economics

 
 
MEAN-FIELD APPROXIMATION OF STOCHASTIC 
POPULATION PROCESSES IN GAMES 
 
 
 
 
Michel BENAÏM 
Jörgen W. WEIBULL 
 
 
 
 
 
 
May  2009 
 
Cahier n° 2009-48 
 
 
 
                              ECOLE POLYTECHNIQUE                       
CENTRE NATIONAL DE LA RECHERCHE SCIENTIFIQUE 
 
 
DEPARTEMENT D'ECONOMIE 
Route de Saclay 
91128 PALAISEAU CEDEX 
(33) 1 69333033 
http://www.enseignement.polytechnique.fr/economie/  
mailto:chantal.poujouly@polytechnique.edu 
 
 
h
a
l
-
0
0
4
3
5
5
1
5
,
 
v
e
r
s
i
o
n
 
1
 
-
 
2
4
 
N
o
v
 
2
0
0
9Mean-ﬁeld approximation of stochastic population processes in games
Michel Bena¨ ım∗
Institut de Math´ ematique
Universit´ ed eN e u c h ˆ atel, Switzerland
J¨ orgen W. Weibull
Department of Economics
Stockholm School of Economics
May 14, 2009.
Abstract. We here establish an upper bound on the probability for
deviations of a Markov population process from its mean-ﬁeld approximation.
The population consists of n distinct subpopulations of equal size N,w h e r e
each subpopulation is associated with a player role in a ﬁnite n-player game.
At discrete times t =0 ,1/N, 2/N, ... one individual is drawn at random from
the total population to review his or her pure strategy choice. We allow transi-
tion probabilities to depend smoothly on population size N and show that the
probability bound converges exponenially to zero as N →∞ . This generalizes
ar e s u l ti nB e n a ¨ ım and Weibull (2003).
Many population models in game theory hypothesize a continuum of interacting
agents and describe the evolutionary selection process in terms of a system of ordinary
diﬀerential equations. These equations usually concern changes in population shares
associated with the diﬀerent pure strategies in the game, and the changes are viewed
as aggregates of large numbers of individual strategy switches. In this so-called mass-
action interpretation, due to Nash (1950), individuals are randomly and recurrently
drawn to review their own choice of pure strategy, and mixed strategies are popula-
tion distributions over pure strategies, rather than randomizations implemented by
individual players. This population model was not formalized by Nash (1950), but
dynamic population models later emerged in evolutionary biology. Prime examples
are diﬀerent versions of the replicator dynamics, see Taylor and Jonker (1978), Taylor
(1979) and Maynard Smith (1982). For wide classes of such population dynamics,
results have been obtained that establish connections with non-cooperative concepts
such as dominance, Nash equilibrium and strategic stability (see, e.g. Weibull (1995)).
∗Our research was supported by the Swiss National Foundation Grant 200020-120218 and by the
Alice and Knut Wallenberg Foundation.
1
h
a
l
-
0
0
4
3
5
5
1
5
,
 
v
e
r
s
i
o
n
 
1
 
-
 
2
4
 
N
o
v
 
2
0
0
9Mean-ﬁeld approximation of stochastic population processes in games 2
An important question for the relevance of these results is whether these dynamics
are good approximations of the stochastic population processes that arise from indi-
vidual strategy revision in ﬁnite but large populations. In an earlier study, Bena¨ ım
and Weibull (2003), we addressed this question and established approximation re-
sults under the hypothesis that switching rates do not depend on population size.
However, robustness in this respect is desirable, since in applications switching prob-
abilities may well, to some extent, depend on population size, see Fudenberg et al.
(2006), Fudenberg and Imhof (2008) and Example 1 below.1 In this note, we gen-
eralize our previous result to allow for such dependence on population size.2 To the
best of our knowledge, the present result is the most general and powerful approxima-
tion for this class of Markov chains, giving an exponential upper bound on deviation
probabilities in bounded time intervals, a bound that permits asymptotic analysis by
way of the Borel-Cantelli Lemma.3 The present generalization is obtained by ﬁrst
establishing a result for the stochastic process’ deviations from its ﬁnite-population
mean-ﬁeld ﬂow, before going to its deviation from its limit mean-ﬁeld ﬂow.
1. A class of stochastic processes
Consider a ﬁnite n-player game with player roles i ∈ I = {1,...n}, ﬁnite pure strategy
sets Si = {1,...,mi}, set of pure-strategy proﬁles S = ×iSi, mixed-strategy simplices
∆(Si)=
(
xi ∈ R
mi
+ :
X
h∈Si
xih =1
)
,( 1 )
and polyhedron ¤(S)=×i∆(Si) of mixed-strategy proﬁles x =( x1,...,xn). The
polyhedron ¤(S)i st h u sas u b s e to fRm,f o rm =
P
i mi. For each player role i there
is a subpopulation consisting of N individuals. Each individual is at every moment
in time associated with a pure strategy in her strategy set. An individual in pop-
ulation i who is associated with pure strategy h ∈ Si is called an h-strategist.A t
times t ∈ T = {0,δ,2δ,...},w h e r eδ =1 /N, and only then, exactly one individual
has the opportunity to change his or her pure strategy. This individual is randomly
drawn, with equal probability for all nN individuals, and with statistical indepen-
dence between successive draws. With this ﬁxed relationship between population size
and period length, the expected time interval between two successive draws of one
1We are grateful to Drew Fudenberg for raising this issue.
2We also take the opportunity to correct a mistake in the statement of Lemma 1 in our previous
paper. The statement should be “For every T>0 there exists...”, see Proposition 1 below. We are
grateful to Sergiu Hart for spotting the mistake.
3For other stochastic approximation results, see Kurtz (1981), Benveniste et al. (1990), Duﬂo
(1996), Kushner and Yin (1997), Bena¨ ım (1999) and Bena¨ ım and Le Boudec (2008).
h
a
l
-
0
0
4
3
5
5
1
5
,
 
v
e
r
s
i
o
n
 
1
 
-
 
2
4
 
N
o
v
 
2
0
0
9Mean-ﬁeld approximation of stochastic population processes in games 3
and the same individual is n, independently of the population size N.W e w i l l c a l l
the times t ∈ T transition times - the only times when a transition can take place.
For each N ∈ N,l e tXN =
­
XN (t)
®
t∈T be a Markov chain, with ﬁnite state space
¤N(S), deﬁned as follows. First, ¤N(S) is the subset (“grid”) of points x ∈ ¤(S)
such that Nxih is a nonnegative integer for each i ∈ I and h ∈ Si. Secondly, for
every player role i and pair (h,k) ∈ S2
i of pure strategies for that role, there exists a
continuous function phN
ik : ¤(S) → [0,1] such that phN
ik (x)=0i fxik =0a n d
Pr
∙
X
N
i (t +
1
N
)=xi +
1
N
¡
e
h
i − e
k
i
¢
| X
N(t)=x
¸
= p
hN
ik (x)( 2 )
for all i ∈ I, h,k ∈ Si, N ∈ N and x ∈ ¤N(S). Here phN
ik (x) is the conditional prob-
ability, given the current population state x,t h a tak-strategist in player population
i will be drawn for strategy revision and switch to pure strategy h. Thus, for any
v =( v1,...,vn) ∈ Rm:
Pr
∙
X
N(t +
1
N
)=x +
1
N
v | X
N(t)=x
¸
=
½
phN
ik (x)i f vi = eh
i − ek
i and vj =0∀j 6= i
0o t h e r w i s e
For any player role i ∈ I and pure strategy h ∈ Si, and any population size N,
the expected net increase in the subpopulation of h-strategists, from one transition
time to the next, and conditional upon the current state x,i s
F
N
ih(x)=
X
k6=h
p
hN
ik (x) −
X
k6=h
p
kN
ih (x). ( 3 )
It follows from the probability speciﬁcation above that FN
ih : ¤(S) → R is bounded,
with
P
h FN
ih(x) ≡ 0a n dFN
ih(x) ≥ 0i fxih = 0. We view each function FN as a
mapping from E to E,w h e r eE = RM,f o rM = m − n, is the tangent space of the
polyhedron ¤(S).4 We assume that every function FN,f o rN ∈ N, is bounded and
locally Lipschitz continuous, that there for every compact set C ⊂ E exists a common
Lipschitz constant λ for all functions FN, and, moreover, that FN → F uniformly.
Then also F is bounded and locally Lipschitz continuous.
We are interested in deterministic continuous-time approximation of Markov chains
XN in the class deﬁned above, when the population size N is large, and thus the time
interval δ =1 /N between successive transition times is short. The key element for
such approximation is the vector ﬁeld FN : E → E deﬁned above, which, for short
4Recall that ¤(S) is the Cartesian product of n unit simplices, ∆(Si), and that the latter is a
subset of a hyperplan in Rmi. Hence, the dimension of E is M = m − n.
h
a
l
-
0
0
4
3
5
5
1
5
,
 
v
e
r
s
i
o
n
 
1
 
-
 
2
4
 
N
o
v
 
2
0
0
9Mean-ﬁeld approximation of stochastic population processes in games 4
time intervals, gives the expected net increase in each population share during the
time interval, per time unit.5 The associated system of mean-ﬁeld equations,
˙ xih = F
N
ih(x) ∀i ∈ I,h∈ Si,x∈ E (4)
deﬁne this limiting dynamic. In force of the Picard-Lindel¨ of Theorem, the system (4)
of ﬁrst-order ordinary diﬀerential equations has a unique solution through every point
x in E (see, e.g., Hale (1969)). Moreover, as noted above, the sum of all population
shares in each population remains constant over time, and no population share can
turn negative. Hence, the system of equations (4) deﬁnes a solution mapping ξN :
R × ¤(S) → E that leaves each mixed-strategy simplex ∆(Si), and hence also the
polyhedron ¤(S) of mixed-strategy proﬁles, invariant. In other words, the system of
diﬀerential equations determines a solution for all times t ∈ R, and if the initial state
is in ¤(S), then also all future states are in ¤(S).6 We will call ξN the ﬂow induced
by FN. Similarly, let ξ be the ﬂow induced by the limit vector ﬁeld F.
Let ||·||∞ denote the L∞-norm on E = RM.T h e n|| ˆ XN(t)−ξN(t,x)||∞ represents
the deviation of the interpolated Markov chain from the deterministic approximation
solution ξN at time t, measured as the largest deviation in any population share at
time t:
|| ˆ X
N(t) − ξ
N(t,x)||∞ =m a x
i∈I,h∈Si
¯ ¯ ¯ ˆ X
N
ih(t) − ξ
N
ih(t,x)
¯ ¯ ¯ .( 5 )
The random variable
D
N
N(T,x)=m a x
0≤t≤T
|| ˆ X
N(t) − ξ
N(t,x)||∞ (6)
is thus the maximal deviation in any population share, from the ﬂow induced by FN
through x, during a bounded time interval [0,T].
Likewise, the random variable
D
N(T,x)=m a x
0≤t≤T
|| ˆ X
N(t) − ξ(t,x)||∞ (7)
is the maximal deviation in any population share, from the ﬂow ξ induced by F
through x, during the same time interval.
Example 1. Suppose that (a) every individual has the same probability of being
drawn for strategy revision, (b) the revising individual draws another individual in his
5There are N transition times per time unit and N individuals in each player population.
6More exactly: ξ(0,x)=x for all x, ∂
∂tξih(t,x)=Fih [ξ(t,x)] for all i, h, x and t,a n dξi(t,x) ∈
∆(Si) for all i ∈ I, x ∈ ¤(S), and t ∈ R. The time domain of the solution mapping ξ can be taken
to be the whole real line in force of the compactness of ¤(S).
h
a
l
-
0
0
4
3
5
5
1
5
,
 
v
e
r
s
i
o
n
 
1
 
-
 
2
4
 
N
o
v
 
2
0
0
9Mean-ﬁeld approximation of stochastic population processes in games 5
or her own player subpopulation, and (c) depending on the information then available
about her own and the other individual’s payoﬀs, imitates the other individual. Then
( 3 )h o l d sf o ra l lN ≥ 2,w i t h
p
hN
ik (x)=
xik
n
·
Nxih
N − 1
· q
h
ik(x),
where qh
ik (x) is the conditional imitation probability from pure strategy k to pure
strategy h.I fa l lf u n c t i o n sqh
ik : E → [0,1] are Lipschitz continuous, then
F
N
ih (x)=
1
n(1 − 1/N)
"
X
k6=h
xikq
h
ik(x) −
X
k6=h
xikq
k
ih(x)
#
xih
so all functions FN are bounded, have a common Lipschitz constant, and converge
uniformly to the function F deﬁned by setting 1/N =0in this formula.
2. Mean-field approximation of DN
N(T,x)
Let Uk,f o rk ∈ N, be the diﬀerence between the step taken by the Markov chain XN
between periods k and k +1 ,p e rt i m eu n i t ,a n dt h ev e c t o rﬁeld FN at the state:
Uk =
1
δ
£
X
N((k +1 ) δ) − X
N(kδ)
¤
− F
N(X
N(kδ)) , (8)
where δ =1 /N is the length of a period. Let Fk,k ∈ N denote the sigma-ﬁeld
generated by
©
XN(0),...,XN(kδ)
ª
. The following result provides a useful upper
bound on the diﬀerence Uk.L e th.,.i denote the inner product in the tangent space of
the polyhedron of mixed-strategy proﬁles: hx,yi =
PM
i=1 xiyi for any vectors x,y ∈ E.
Lemma 1. Let ΓN =(
√
2+
° °FN° °
2)2.F o ra n yθ ∈ RM:
E(e
hθ,Uki|Fk) ≤ e
ΓN||θ||2
2/2
Proof. By deﬁnition of Uk it is easy to verify that
||Uk||2 ≤ max
i,h,k
||e
h
i − e
k
i||2 +
° °F
N° °
2 =
p
ΓN (9)
Let g(t)=l o gE(ethθ,Uki|Fk). The function g : R → R is convex and satisﬁes g(0) =
g0(0) = 0, g00(t) ≤ ||θ||2
2ΓN. Therefore g(1) ≤ ||θ||2
2ΓN/2.
We are now in a position to state and prove our result for DN
N(T,x):
h
a
l
-
0
0
4
3
5
5
1
5
,
 
v
e
r
s
i
o
n
 
1
 
-
 
2
4
 
N
o
v
 
2
0
0
9Mean-ﬁeld approximation of stochastic population processes in games 6
Lemma 2. For every T>0 there exists a scalar c>0 such that, for any ε>0,a n d
any N large enough:
Pr
£
D
N
N(T,x) ≥ ε | X
N(0) = x
¤
≤ 2Me
−ε2cN ∀x ∈ ¤
N(S)
Proof. In order to prove this, let λN be the Lipschitz constant of FN on the
compact set ¤(S) ⊂ E, with respect to the L∞-norm, let k·k2 denote the L2-norm,
and let
° °FN° °
2 be the maximum of
° °FN (x)
° °
2 on ¤(S). Let Uk,f o rk ∈ N,b ea s
deﬁned above, and let U : R+ → E be the map deﬁned by U(t)=Uk for kδ ≤ t<
(k +1 ) δ. Likewise, let ¯ XN be the continuous-time (right-continuous) step process
generated by the Markov chain XN: ¯ XN(t)i sd e ﬁned for all t ∈ R+ by ¯ XN(t)=
XN(kδ)f o rkδ ≤ t<(k +1 ) δ. Suppose that XN(0) = x ∈ ¤(S). Then
ˆ X
N(t) − x =
Z t
0
£
F
N( ¯ X
N(s)) + U(s)
¤
ds (10)
=
Z t
0
h
F
N( ˆ X
N(s)) + F
N( ¯ X
N(s)) − F
N( ˆ X
N(s)) + U(s)
i
ds .
Since ξN(t,x) − x =
R t
0 FN(ξN(s,x))ds,w eo b t a i n
|| ˆ X
N(t) − ξ
N(t,x)||∞ ≤ λN
∙Z t
0
(|| ˆ X
N(s) − ξ
N(s,x)||∞)ds +2 δT
¸
+ Ψ(T), ( 1 1 )
where
Ψ(T)=m a x
0≤t≤T
||
Z t
0
U(s)ds||∞ .( 1 2 )
Gr¨ onwall’s inequality implies
D
N
N(T,x)=m a x
0≤t≤T
|| ˆ X
N(t) − ξ
N(t,x)||∞ ≤ [Ψ(T)+2 δλNT]e
λNT .( 1 3 )
Thus, for δ ≤ ε
4λNTe−λNT,
Pr
£
D
N
N(T,x) ≥ ε
¤
≤ Pr
h
Ψ(T) ≥
ε
2
e
−λNT
i
. (14)
Our next goal is to estimate the probability on the right-hand side. For k ∈ N,l e t
Zk(θ)=e x p
Ã
k−1 X
i=0
hθ,δUii −
Γ
2
kδ
2||θ||
2
2
!
.( 1 5 )
h
a
l
-
0
0
4
3
5
5
1
5
,
 
v
e
r
s
i
o
n
 
1
 
-
 
2
4
 
N
o
v
 
2
0
0
9Mean-ﬁeld approximation of stochastic population processes in games 7
According to lemma 1, (Zk(θ))k∈N is a supermartingale. Thus, for any β>0
Pr
"
max
0≤k≤n
hθ,
n−1 X
i=0
δUii ≥ β
#
≤ Pr
∙
max
0≤k≤n
Zk(θ) ≥ exp
µ
β −
ΓN
2
||θ||
2
2nδ
2
¶¸
≤ exp
µ
ΓN
2
||θ||
2
2nδ
2 − β
¶
.( 1 6 )
Let u1,...,u M be the canonical basis of E = RM, ε>0, and u = ±ui for some i.S e t
β = ε2/(ΓNnδ2)a n dθ =( β/ε)u.T h e n
Pr
"
max
0≤k≤n
hu,
k−1 X
i=0
δUii ≥ ε
#
=P r
"
max
0≤k≤n
hθ,
k−1 X
i=0
δUii ≥ β
#
(17)
≤ exp(
−ε2
2ΓNnδ2).
It follows that
Pr[Ψ(T) ≥ ε] ≤ 2M exp
µ
−ε2
2δΓNT
¶
.( 1 8 )
Therefore,
Pr
h
Ψ(T) ≥
ε
2
e
−λNT
i
≤ 2M exp
µ
−ε
2e−2λNT
8δΓNT
¶
(19)
=2 M exp
£
−ε
2cNN
¤
,( 2 0 )
where
cN =
e−2λNT
8ΓNT
=
e−2λNT
8T(
√
2+kFNk2)2
Hence, the claim in the lemma holds for any c ∈ (0,γ), where
γ = lim inf
N→∞
e−2λNT
8T(
√
2+kFNk2)2 (21)
is positive by our hypotheses about the sequence
¡
FN¢
.
3. Mean-field approximation of DN(T,x)
We now turn to the stochastic variable DN(T,x). By Gr¨ onwall’s inequality,
lim
N→∞
max
0≤t≤T
||ξ(t,x) − ξ
N(t,x)||∞ =0
h
a
l
-
0
0
4
3
5
5
1
5
,
 
v
e
r
s
i
o
n
 
1
 
-
 
2
4
 
N
o
v
 
2
0
0
9Mean-ﬁeld approximation of stochastic population processes in games 8
Hence, for any ε>0t h e r ee x i s t saNε such that
max
0≤t≤T
||ξ(t,x) − ξ
N(t,x)||∞ <ε / 2
for all N ≥ Nε. Hence, by the Triangle Inequality, for such N,t h ee v e n t
AN =
©
ω ∈ Ω : D
N(T,x) ≥ ε | X
N(0) = x
ª
implies the event
BN =
©
ω ∈ Ω : D
N
N(T,x) ≥ ε/2 | X
N(0) = x
ª
that is, AN ⊂ BN, and hence Pr(AN) ≤ Pr(BN). Our main result follows immedi-
ately from Lemma 2 above:
Proposition 1. For every T>0 there exists a scalar c>0 such that, for any ε>0,
and any N large enough:
Pr
£
D
N(T,x) ≥ ε | X
N(0) = x
¤
≤ 2Me
−ε2cN ∀x ∈ ¤
N(S).
Remark 1. The claim holds for any c ∈ (0,γ/4),w h e r eγ>0 is deﬁn e di ne q u a t i o n
(21).
Remark 2. It is easily veriﬁed that propositions 1-5 in Bena¨ ım and Weibull (2003)
hold also under the new, weaker hypothesis.
References
[1] Bena¨ ım M. (1999): “Dynamics of stochastic approximations”, Le Seminaire de
Probabilit´ es. Lectures Notes in Mathematics, Vol. 1709, 1-68.
[2] Bena¨ ım M. and J.-Y. Le Boudec (2008): “A class of mean-ﬁeld interaction models
for computer and communication systems”, Performance Evaluation 65, 823-838.
[3] Bena¨ ım M. and J. Weibull (2003): “Deterministic approximation of stochastic
evolution in games”, Econometrica 71, 873-903.
[4] Benveniste A., M. M´ etivier and P. Priouret (1990): Stochastic Approximation
and Adaptive Algorithms. Springer Verlag, New York.
[5] Duﬂo M. (1996): Algorithmes Stochastiques. Springer Verlag, New York.
h
a
l
-
0
0
4
3
5
5
1
5
,
 
v
e
r
s
i
o
n
 
1
 
-
 
2
4
 
N
o
v
 
2
0
0
9Mean-ﬁeld approximation of stochastic population processes in games 9
[6] Fudenberg D., M.Nowak, C. Taylor and L. Imhof (2006): “Evolutionary game
dynamics in ﬁnite populations with strong selection and weak mutation”, The-
oretical Population Biology 70, 352-363.
[7] Fudenberg D. and L. Imhof (2008): “Monotone imitation dynamics in large
populations”, Journal of Economic Theory 140, 229-245.
[8] Hale J. (1969): Ordinary Diﬀerential Equations. John Wiley and Sons, New
York.
[9] Kurtz T. (1981): Approximation of Population Processes.S I A M ,C B M S - N S F
36, Philadelphia.
[10] Kushner H. and G. Yin (1997): Stochastic Approximation Algorithms and Ap-
plications. Springer Verlag, New York.
[11] Maynard Smith J. (1982): Evolution and the Theory of Games. Cambridge Uni-
versity Press, Cambridge.
[12] Nash J. (1950): “Non-cooperative games”, Ph.D. thesis in mathematics at
Princeton University.
[13] Taylor P. and L. Jonker (1978): “Evolutionarily stable strategies and game dy-
namics”, Mathematical Biosciences 40, 145-156.
[14] Taylor P. (1979): “Evolutionarily stable strategies with two types of player”,
Journal of Applied Probability 16, 76-83.
[15] Weibull J. (1995): Evolutionary Game Theory. MIT Press, Cambridge MA, USA.
h
a
l
-
0
0
4
3
5
5
1
5
,
 
v
e
r
s
i
o
n
 
1
 
-
 
2
4
 
N
o
v
 
2
0
0
9

Algorithmes Stochastiques.

Approximation of Population Processes.S

Boudec (2008): “A class of mean-ﬁeld interaction models for computer and communication systems”,

Deterministic approximation of stochastic evolution in games”,

Dynamics of stochastic approximations”,

Evolution and the Theory of Games.

Evolutionarily stable strategies and game dynamics”,

Evolutionarily stable strategies with two types of player”,

Evolutionary game dynamics in ﬁnite populations with strong selection and weak mutation”,

Mean-field approximation of stochastic population processes in games

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics