In this paper we prove the existence, uniqueness and stability of the invariant distribution of a random dynamical system in which the admissible family of laws of motion consists of monotone maps from a closed subset of a finite dimensional Euclidean space into itself.

Bhattacharya, Rabi

Majumdar, Mukul

English

Research Papers in Economics

ISSN 1936-5098
CAE Working Paper #08-08
Random Iterates of Monotone Maps
                                                   
by
Rabi Bhattacharya
and
Mukul Majumdar
   
December 2008
                               Editorial Manager(tm) for Review of Economic Design
                                  Manuscript Draft
Manuscript Number: ROED-D-07-00023
Title: Random Iterates of Monotone Maps
Article Type: Special Issue for Dr. Hurwicz
Section/Category: 
Keywords: Random Dynamical Systems, Invariant Distribution, Convergence, Markov Processes
Corresponding Author: Professor Mukul Majumdar, 
Corresponding Author's Institution: Cornell University
First Author: Mukul Majumdar
Order of Authors: Mukul Majumdar; Rabi Bhattacharya
Manuscript Region of Origin: 
Abstract: In this paper we prove the existence,uniqueness and stability of the invariant distribution of a 
random dynamical system in which the admissible family of laws of motion consists of monotone maps from 
a closed subset of a finite dimensional Euclidean space into itself.Random Iterates of Monotone Maps
Rabi Bhattacharya
Department of Mathematics
University of Arizona
Tucson, AZ 95721
and
Mukul Majumdar
Department of Economics
Cornell University
Ithaca, NY 14853
June 7, 2007
1
Manuscript
Click here to download Manuscript: summer2007RIMM.tex1 Introduction
This paper is dedicated to Leonid Hurwicz. Hurwicz (1944) was a contributor to the
literature on stochastic models of growth and cycles. In collaboration with Kenneth
Arrow he also set the tenor of research on multi-sector dynamic models [see Part
III of Arrow and Hurwicz (1977)]. We focus on a class of stochastic or random
dynamic processes that have been of particular interest in the context of optimiza-
tion problems in – to use his terminology – “non-classical” environments. A formal
statement of the main result is in Section 2. But we begin with a few informal
remarks to provide the motivation. The mathematical model of discounted stochas-
tic dynamic programming has become the basic tool in exploring optimal decision
making under uncertainty both at the micro and macro levels. In “classical” models,
by imposing appropriate (strict) convexity, continuity and monotonicity properties
on the primitives (technological constraints involved in specifying the law of motion,
return functions...), one is able to assert that the optimal policy function is monotone
and continuous. Once, however, one attempts to step out of the “classical” envi-
ronment (for example, to allow for a Knightian S-shaped production function that
exhibits an initial phase of increasing returns), the standard proof of continuity of
the optimal policy function fails. Indeed, even in a deterministic non-classical model
of intertemporal optimization, an example of discontinuity (in which the production
function is S-shaped, the return function is linear) was given in Majumdar and Mitra
(1983). However, in a large class of stochastic models one can still prove that there
is an optimal policy function that is monotonic (see Majumdar, Mitra and Nyarko
(1989) for an elaboration of the ﬁner points of selection and a comprehensive account
2of dynamic optimization under uncertainty with non-concave production functions).
This monotonicity property turns out to be crucial in making signiﬁcant progress in
understanding the evolution of an optimal process, and in establishing some long run
convergence properties. The analysis is simpler when the state space is an interval
(in the real line). Exploring the implications of monotonicity (with possible dis-
continuity) when the state space is a closed subset of a ﬁnite dimensional Euclidean
space is the point of departure of this paper. Consider a random dynamical system
(S,Γ,P) where S is the state space (for example, a closed subset of Rk, Γ an appro-
priate family of maps on S into itself and P is a probability measure on (some σ-ﬁeld
of) Γ.
The evolution of the system can be described as follows: initially, the system is
in some state x; an element α1 of Γ is chosen randomly according to the probability
measure P and the system moves to the state X1 = α1(x) in period one. Again,
independently of α1, an element α2 of Γ is chosen according to the probability mea-
sure P and the state of the system in period two is obtained as X2 = α2(α1(x)). In
general, starting from some x in S, one has
Xn+1(x) = αn+1(Xn(x)) (1.1)
where the maps (αn) are independent and identically distributed according to the
measure P. The initial point x can also be chosen (independently of (αn)) as a
random variable X0. The sequence Xn of states obtained in this manner is a Markov
process and has been of particular interest in economics (and other disciplines).
For describing “convergence to a long run steady state”, perhaps the most widely
3used results identify conditions under which there is some time invariant probability
measure π such that, no matter what the initial x0 is, Xn converges in distribution
to π. In this case we say that the (Markov) process is stable in distribution.
2 The Main Result
In this section we extend an important old result of Dubins and Freedman (1966) on
i.i.d. iterations of monotone maps to multi-dimensional state spaces, and improve
upon some recent results in Bhattacharya and Majumdar [(1999), (2007)], by dis-
pensing with the requirement of continuity of the maps. The state spaces of the
Markov process we consider is assumed to be a subset of Rk(k ≥ 1) satisfying the
following assumption:
(A.1) S is either a closed subset of Rk, or a Borel subset which can be made
homeomorphic to a closed subset of Rk, by means of a strictly increasing continuous
map on S into Rk.
It may be noted that every rectangle Xk
j=1Ij, where Ij’s are arbitrary nondegen-
erate sub-intervals of the real line R satisﬁes the assumption (A1). For, an interval
(a,b)(−∞ ≤ a < b ≤ ∞) is homeomorphic to (−∞,∞) by an appropriate strictly
increasing continuous map. An interval (a,b](−∞ ≤ a < b < ∞) is similarly
homeomorphic to (−∞,0], etc.
To deﬁne the Markov process, let Γ be a set of measurable monotone maps γ =
(γ1,γ2,...,γk) on S into S, under the partial order: x≤ y if xj ≤ yj for 1 ≤ j ≤ k; x
= (x1,...,xk), y = (y1,y2,...,yk)ǫRk (or S). That is, either γ is monotone increasing:
4γ(x) ≤ γ(y) if x ≤ y, or γ is monotone decreasing: γ(y) ≤ γ(x) if x ≤ y; x, y
ǫ S. Let Γ be endowed with a σ-ﬁeld C, and let Q be a probability measure on
(Γ,C). Consider a sequence of i.i.d. maps {αn : n ≥ 1} with common distribution
Q, deﬁned on a probability space (Ω,ℑ,P). For purposes of measurability, assume
that the map (γ,x) → γ(x) on ΓXS into S is measurable with respect to the product
σ-ﬁeld C ⊗ B(S) on ΓXS and the Borel σ-ﬁeld B(S) on S. For each y ǫ S, deﬁne
the Markov process {Xn : n ≥ 0} by
X0 = y, X1 = α1X0,...,Xn = αnXn−1 = αnαn−1...α1X0, (2.1)
where αnαn−1...α1 denotes composition of maps in the indicated order. In general,
X0 can be any random variable with values in S, independent of the sequence {αn :
n ≥ 1}. The transition probability of the Markov process is p(x,B) = P(α1x ǫ
B) = Q({γǫΓ : γx ǫ B}). In general, the n-step transition probability is given by
the distribution of Xn(x) ≡ αnαn−1...α1x, and is denoted by p(n)(x,.). It may also
be expressed as
p
(n)(x,B) = Q
n({γ ǫ Γ
n : ˜ γ x ǫ B}), (xǫS, BǫB(S)), n ≥ 1, (2.2)
where Qn is the product probability on the product space (Γn,C⊗n), and ˜ γ is the
composition
˜ γ x = γnγn−1...γ1x (γ = (γ1,γ2,...,γn)ǫΓ
n). (2.3)
Recall that π is an invariant probability for the Markov process, or for the transition
5probability p, if π is a probability measure on (S,B(S)) satisfying
π(B) =
Z
p(x,B)π(dx) ∀BǫB(S). (2.4)
In turn, (2.4) implies that π(B) =
R
p(n)(x,B)π(dx) ∀BǫB(S), and ∀n ≥ 1. If one
denotes the distribution of Xn as T ∗n , where   is the distribution of X0, then T ∗n
is the n-fold composition of T ∗ : T ∗n = T ∗T ∗(n−1)(n ≥ 2), T ∗1 = T ∗. Note that T ∗
(as well as T ∗n) is a map on the space ℘(S) of all probability measures on (S,B(S)) :
(T
∗n )(B) =
Z
p
(n)(x,B) (dx) ( ǫ℘(S),BǫB(S)). (2.5)
Clearly, an invariant probability π is just a ﬁxed point of T ∗ ≡ T ∗1, in which case it
is a ﬁxed point of T ∗n for every n.
On the space ℘(S), deﬁne, for each a > 0, the metric
da( ,ν) = sup
gǫGa
￿
￿ ￿
￿
Z
gd  −
Z
gdν
￿
￿ ￿
￿, ( ,ν ǫ℘(S)), (2.6)
where Ga is the class of all Borel measurable monotone (increasing or decreasing)
functions g on S into [0,a]. It is simple to check that (i) da = ad1, and (ii)
the distance (2.6) remains the same if Ga is restricted to monotone increasing Borel
measurable functions on S into [0,a]. The following result is due to Chakraborty
and Rao (1998), who derived a number of interesting results on the metric space
(℘(S),da). One can show that convergence in the metric da implies weak convergence
if (A1) holds (see Bhattacharya and Majumdar (2007), pp. 287-288).
6Lemma 1 Under the hypothesis (A.1), (℘(S),da) is a complete metric space.
The following splitting condition generalizes that in Dubins and Freedman (1966).
To state it, let ˜ γ be as in (2.3), but with n = N : ˜ γ = γNγN−1...γ1 for γ =
(γ1,γ2,...,γN) ǫ ΓN.
(A.2) There exist Fi ǫC⊗N(i = 1,2) for some N ≥ 1, such that
(i) δi ≡ QN(Fi) > 0 (i = 1,2), and
(ii) for some x0ǫ S, one has
˜ γ(x) ≤ x0 ∀xǫS, ∀γ ǫF1,
˜ γ(x) ≥ x0 ∀xǫS, ∀γ ǫF2,
Also, assume that the set H+ = {γǫΓN : ˜ γ is monotone increasing}ǫC⊗N.
Readers interested in the veriﬁcation of the splitting condition in dynamic models
in economics may turn to Bhattacharya and Majumdar (2007).
Our main result is the following:
Theorem 2 Let {αn : n ≥ 1} be a sequence of i.i.d. measurable monotone maps
with a common distribution Q. Assume (A.1), (A.2) hold. Then there exists a
unique invariant probability for the Markov process (2.1) and
sup
xǫS
d1(p
(n)(x,.),π) ≤ (1 − δ)[
n
N](n ≥ 1), (2.7)
where δ = min{δ1,δ2}, and
￿
n
N
￿
is the integer part of n
N.
7Proof. The proof uses Lemma (2.1) and two steps. The ﬁrst involves detailed
calculations.
Step 1. T ∗N is a uniformly strict contraction on (℘(S),d1)
Let Fi+ = Fi ∩ H+, Fi− = Fi ∩ H−, where H+ is deﬁned in (A.2), and H− =
ΓN\H+(i = 1,2). Deﬁne, for any given g ǫG1, the functions
hi+(x) =
Z
Fi+\(Fi+∩Fj)
g(˜ γx)Q
N(dγ),
hi−(x) =
Z
Fi−\(Fi−∩Fj)
g(˜ γx)Q
N(dγ), (i = 1, 2; j  = i);
h3+(x) =
Z
H+∩(F1∪F2)c
g(˜ γx)Q
N(dγ),
h3−(x) =
Z
H−∩(F1∪F2)c
g(˜ γx)Q
N(dγ),
h4(x) =
Z
F1∩F2
g(˜ γx)Q
N(dγ). (2.8)
Then the functions hi±(i = 1,2,3), are monotone. To see this, let g be monotone
increasing, then hi+(i = 1,2,3) are monotone increasing while hi−(i = 1,2,3) are
monotone decreasing. If g is monotone decreasing, then the reverse holds. Now,
8for g monotone increasing (g ǫG1),
h1+(x) ≤ g(x0)(Q
N(F1+) − Q
N(F1+ ∩ F2)) ≡ a1+,
h1−(x) ≤ g(x0)(Q
N(F1−) − Q
N(F1− ∩ F2)) ≡ a1−,
h2+(x) ≥ g(x0)(Q
N(F2+) − Q
N(F2+ ∩ F1)),
h2−(x) ≥ g(x0)(Q
N(F2−) − Q
N(F2− ∩ F1)),
h3+(x) ≤ Q
N(H+ ∩ (F1 ∪ F2)
c) ≡ a3+,
h3−(x) ≤ Q
N(H− ∩ (F1 ∪ F2)
c) ≡ a3−. (2.9)
Also, write
h
′
2+(x) =
Z
F2+\(F2+∩F1)
(1 − g(˜ γx))Q
N(dγ),
h
′
2−(x) =
Z
F2−\(F2−∩F1)
(1 − g(˜ γx))Q
N(dγ). (2.10)
Then h
′
2±(x) are monotone and satisfy
h
′
2+(x) ≤ (1 − g(x0))(Q
N(F2+) − Q
N(F2+ ∩ F1)) ≡ a2+,
h
′
2−(x) ≤ (1 − g(x0)(Q
N(F2−) − Q
N(F2− ∩ F1)) ≡ a2−. (2.11)
Thus h1±ǫGa1±, h
′
2±ǫGa2±, h3±ǫGa3±. Also,
￿
￿
￿ ￿
Z
h2±(x) (dx) −
Z
h2±(x)ν(dx)
￿
￿
￿ ￿ =
￿
￿
￿ ￿
Z
h
′
2±(x) (dx) −
Z
h
′
2±(x)ν(dx)
￿
￿
￿ ￿, (2.12)
9and
Z
h4(x) (dx) −
Z
h4(x)ν(dx) = 0, ( ,νǫ℘(S)). (2.13)
The last relation follows from the fact that h4(x) = g(x0)QN(F1 ∩ F2), a constant
function on S. For, on F1 ∩ F2, ˜ γ(x) = x0∀xǫS. Hence
￿
￿
￿
￿
Z
gd(T
∗N ) −
Z
gd(T
∗Nν)
￿
￿
￿
￿
=
￿ ￿
￿
￿
￿
3 X
i=1
[
Z
hi+(x) (dx) −
Z
i
hi+(x)ν(dx) +
Z
hi−(x) (dx) −
Z
hi−(x)ν(d(x)]
￿ ￿
￿
￿
￿
(2.14)
≤
X
i=1,3
￿
￿
￿
￿
Z
hi+(x) (dx) −
Z
hi+(x)ν(dx)
￿
￿
￿
￿ +
X
i=1,3
￿
￿
￿
￿
Z
hi−(x) (dx) −
Z
hi−(x)ν(dx)
￿
￿
￿
￿
+
￿
￿
￿
￿
Z
h
′
2+(x) (dx) −
Z
h
′
2+(x)ν(dx)
￿
￿
￿
￿ +
￿
￿
￿
￿
Z
h
′
2−(x) (dx) −
Z
h
′
2−(x)ν(dx)
￿
￿
￿
￿
≤ (a1+ + a1− + a2+ + a2− + a3+ + a3−)d1( ,ν) ≡ ¯ bd1( ,ν), say.
Note that
a1 + a1− = g(x0)(Q
N(F1) − Q
N(F1 ∩ F2)),
a2 + a2− = (1 − g(x0))(Q
N(F2) − Q
N(F1 ∩ F2)),
a3+ + a3− = Q
N((F1 ∪ F2)
c) = 1 − Q
N(F1) − Q
N(F2) + Q
N(F1 ∩ F2),
so that, adding these terms, one gets
¯ b = 1 − [(1 − g(x0))Q
N(F1) + g(x0)Q
N(F2)]
≤ 1 − min{Q
N(F1), Q
N(F2)} = 1 − δ. (2.15)
10Taking the supremum over all monotone increasing g ǫG1 on the left in (2.14), one
arrives at the inequality
d1(T
∗N , T
∗Nν) ≤ (1 − δ)d1( ,ν), ∀ ,νǫ℘(S). (2.16)
Note that, the supremum in (2.6) over all of Ga is the same as the supremum over
the subset of all monotone increasing functions in Ga, since a − g ǫGa and is mono-
tone increasing if g is monotone decreasing, g ǫGa. Thus T ∗N is a uniformly strict
contraction on (℘(S),d1).
Step 2. Application of the Contraction Mapping Theorem.
From (2.16) and Lemma 2.1, it follows by the contraction mapping theorem that
T ∗N has a unique ﬁxed point π in ℘(S) and that, writing n =
￿
n
N
￿
N + r, one has
d1(T
∗n , π) = d1(T
∗[
n
N]NT
∗r , T
∗[
n
N]Nπ)
≤ (1 − δ)[
n
N]d1(T
∗r ,π) ≤ (1 − δ)[
n
N] ∀ ,νǫ℘(S). (2.17)
In particular, (2.7) follows by letting   = δ{x}− the Dirac measure at x in (2.17).
Note that T ∗N(T ∗π) = T ∗(T ∗Nπ) = T ∗π, so that T ∗π is also a ﬁxed point of T ∗N.
By uniqueness of the ﬁxed point T ∗π = π.
Remark 2.1. In order to derive conﬁdence regions of (or tests for) useful func-
tionals of π (e.g., the mean or dispersion), based on a ﬁnite set of observations
Xj(1 ≤ j ≤ n), one needs to derive asymptotic distributions of the corresponding
functionals of the empirical distribution 1
n
Pn
j=1δXj. As in Bhattacharya and Ma-
jumdar [(2007), Sections 5.3, 5.4], one can show that, under the assumptions (A.1),
11(A.2), for every bounded function g on S which may be expressed as the diﬀerence
g1 − g2 of two bounded measurable monotone functions (or, equivalenty, for every
ﬁnite linear combination of monotone functions), the central limit theorem (CLT)
holds for its empirical mean 1
n
Pn
j=1g(Xj), whatever the initial state. That is,
√
n(
1
n
n X
j=1
g(Xj) −
Z
gdπ)
L →
n→∞ N(0,σ
2), (2.18)
where
L → denotes convergence in law, or distribution, and N(0, σ2) is the Normal
distribution with mean 0 and variance σ2. The variance parameter may be expressed
as
σ
2 =
Z
f
2(y)π(dy) −
Z
(Tf)
2(y)π(dy), (2.19)
where T is the transition operator: Th(x) =
R
h(y)p(x,dy) and f solves the Poisson
equation in L2(S,π)
(I − T)f = g −
Z
gdπ. (2.20)
Here L2(S,π) is the Hilbert space of functions on S which are square integrable
(with respect to π). See Bhattacharya and Majumdar [(2007), Chapter 5] for more
details on this general theme. In the case S is non-compact and g is unbounded
(e.g., g(x) = xj for x = (x1,...,xk)), one requires that there exist a solution f to the
Poisson equation (2.20). Certain broad conditions for this solvability may be found
in Bhattacharya and Lee (1988), for the case of i.i.d. monotone increasing maps.
Remark 2.2. Instead of the metric d1, one may use a somewhat weaker metric
12dA deﬁned by
dA( ,ν) = sup
AǫA
| (A) − ν(A)| ( ,ν ǫ℘(S)), (2.21)
where A comprises all sets of the form
A = {yǫS : ϕ(y) ≤ x}, xǫS, ϕ monotone measurable. (2.22)
One may prove the completeness of (℘(S),dA) more or less following the steps of the
proof of Lemma C5.1, p. 287, in Bhattacharya and Majumdar (2007), where A is
restricted to the class of sets A in (2.22) with ϕ continuous and monotone increasing.
The analog of Theorem 2.2, with dA in place of d1, may then be proved roughly along
the lines of the proof of Corollary 5.1, pp. 257-258, in Bhattacharya and Majumdar
(2007).
13References
[1] Arrow, K.J. and Hurwicz, L. (1977): Studies in Resource Allocation Processes,
Cambridge University Press, Cambridge.
[2] Bhattacharya, R.N. and Lee, O. (1988): “Asymptotics of a Class of Markov
Processes which Are Not in General Irreducible”, Annals of Probability, 16, 1333-
47 (correction (1997): Annals of Probability, 25, 1541-43).
[3] Bhattacharya, R. and Majumdar, M. (2007) Random Dynamical Systems: Theory
and Applications, Cambridge University Press, Cambridge.
[4] Bhattacharya, R. and Majumdar, M. (1999): “On a Theorem of Dubins and
Freedman”, Journal of Theoretical Probability, 12, 1067-1087.
[5] Chakraborty, S. and Rao, B.V. (1998): “Completeness of Bhattacharya Metric
on the Space of Probabilities”, Statistics and Probability Letters, 36, 321-326.
[6] Dubins, L.E. and Freedman, D.A. (1966): “Invariant Probabilities for Certain
Markov Processes”, Annals of Mathematical Statistics, 37, 837-868.
[7] Hurwicz, L. (1944): “Stochastic Models of Economic Fluctuations”, Economet-
rica 12, 114-124.
[8] Majumdar, M. and Mitra, T. (1983): “Dynamic Optimization with Non-Convex
Technology: The Case of a Linear Objective Function”, Review of Economic
Studies, 50, 143-151.
14[9] Majumdar, M., Mitra, T. and Nyarko, Y. (1989): “Dynamic Optimization Under
Uncertainty: Non-Convex Feasible Set”, Joan Robinson and Modern Economic
Theory, (ed. G.R. Feiwel), MacMillan, 545-590.
15

Random Iterates of Monotone Maps

http://www.arts.cornell.edu/econ/CAE/08-08.pdf

Random Iterates of Monotone Maps

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics