A technique is developed for proving existence and obtaining bounds for the concentration of a stationary distribution for a given Markov process on the basis of comparisons, via stochastic dominance, with a different Markov process, having a known stationary distribution

Green, Edward J.

McKelvey, Richard D.

Packel, Edward W.

Caltech Authors - Main

DIVISION OF THE HUMANITIES AND SOCIAL SCIENCES 
CALIFORNIA INSTITUTE OF TECHNOLOGY 
PASADENA, CALIFORNIA 91125
METHODS FOR COMPARISON OF MARKOV PROCESSES 
BY STOCHASTIC DOMINANCE 
Edward J. Green 
California Institute of Technology 
Richard D. McKelvey 
California Institute of Technology 
Edward W. Packel 
Lake Forest College 
w_c,'t\lUTE OF '\""' }'� �,.. � � � � ('.'.'._, 0 � Q 
� �
� � 
.-'! cc: 
�G- �� 
7'/y SHALL tA"'�\.
SOCIAL SCIENCE WORKING PAPER 395 
July 1981
ABSTRACT 
A technique is developed for proving existence and obtaining 
bounds for the con centration of a stationary distribution for a given 
Markov process on the basis of comparisons, via stochastic dominance, 
with a different Markov process, having a known stationary 
distribution. 
METllODS FOR COMPARISON OF MARKOV PROCESSES 
BY STOCHASTIC DOMINANCE 
Edward J, Green, Richard D. McKelvey, and Edward W. Packel 
1. Introduction
This paper provides a technique for proving existence and 
obtaining bounds for the concentration of a stationary distribution 
for a given Markov process on the basis of comparisons, via stochastic 
dominance, with a different Markov process, having a known stationary 
distribution. Thus, given a particular Markov process p on the
measurable sets of X, we look for a nonnegative real-valued function L 
on X, and another Markov process q on X, such that for every x e X,
the transition measure px generated by p always concentrates at least 
as much of the distribution in each lower contour set of L as does the 
transition measure qx' This by itself is not enough to guarantee a 
limiting distribution for p. However if p is sufficiently smooth, and 
if q satisfies an additional condition which we call ''stochastically 
increasing'' with respect to L, then if q has a stationary 
distribution, so does p, and further, the limiting distribution for p 
is more concentrated (with respect to L) than is that of q. 
2, The Main Results 
Let X be a measurable topological space and ! be a a - Algebra 
on X. Probability measures on X are denoted p, q, etc., i.e. 
p : ! -> �. while Markov processes are denoted p, q, etc., i. e. 
p ! x X -) •· Let L : X -) I+ be a measurable function on X. We
use the following notation. 
For any c & I,
2 
s c {x e X I L(x) i c}. ( 1.1) 
We assume Sc is compact for all c. Also, if p : ! x X -> R is a
Markov process on X, we use the following notation. 
For any ye X, p is the measure p('jy) y ! -> �. For any A s A• PA
is the function p(AI") : X -> R
Now, given any two probability measures, p and q on X, we say 
g dominates p with respect to L, written q >> p, iff, for all c B I, 
p(Sc) 2. q(Sc) (1.2) 
Second, given two Markov processes on X, p and q, we say that q 
dominates p with respect to L, written q >> p, iff, for all y & X,
qy » Py ( 1.3) 
Thirdly, we say that the Markov process q is stochastically increasing 
with respect to L, iff, for all x,y e X,
L(y) 2. L(x) =) qy )) qx
Lemma 1, Let p,q be probability measures on X. If q >> p with 
(1.4) 
respect to L, then 
fLdp i fLdq 
Proof: For any integer. n, define Ln : X -> I
+ by
n2 1 
L = n 
2
2n i� ..1. x zn E. 1 
where N = 2 , c = �. and E. = X - S 
2n 1 _! 
2n 
� c x 
ill Ei
{x 8 XI L(x)) -1}. Then
2n 
{Ln}:=l is a sequence of simple functions converging monotonically to 
L, Further, since q >> P• p(Ei) i q(Ei) for all i, hence
f Lndp = fit c XEi dp 2 cp(E.)i=O 1 
i � cq(E.) = JLndq.ilo 1 
Taking limits, as n -> �. the result follows. 
3 
Q.E.D. 
Lemma 2. Let p and q be two Markov processes on X, and let p and q be 
two probability measures on X such that 
(a) q is stochastically increasing 
(b) q » p 
(c) q » p 
Then, defining p1 : X -> I and q1 : X -> I by, ¥ A s X 
pl(A) 
q1(A) 
= f pAdp 
= f qAdp 
it follows that q1 >> p1.
1 1 Proof: We must show, for all c s I, that q (Sc) i p (Sc) (where
Sc = {x e X I L(x) i c)). But, since q >> p, we have Pg (y) l q8 (y)
for all y. Hence 
1 
f- f-p (S ) = Pg dp l qS dpc c c 
c c 
Further, by assumption (a),  q8 
is monotone decreasing with L. So
c 
p >> q with respect to q
8 
, Hence, applying Lemma 1,
c 
f qs dp l f -qs dq = l<sc> • c c 
4 
Q.E.D. 
Theorem 1. Let p and q be two Markov processes on X such that 
(a) q is stochastically increasing 
(b) q » p 
(c) For all A e !, PA is a continuous function on X. 
Then if q has a stationary distribution, q*, then p has a stationary 
distribution, p*, and q* >  p*. 
Proof: A set Tf, of probability measures is defined to be tight if, 
for every e 0, there exists a compact set, F8, such that 
p(F8) L 1 - e for all p s TT. Here we define 
lT = {p : ! -) R I p is a probability measure and q* )) p} 
It is easily verified that TT is tight since {q*} is tight. 
We now define T : TT -> TT, for all A e M and p e TT, by 
Tp(A) = JpAdp
It follows, from Lemma 2, that T maps 1T into lT.
Let C(X) denote the set of bounded, continuous real valued 
5 
functions on X and let il denote the set of regular, countably additive 
set functions on the measurable subsets ! of X. For each f e C(X)
define the linear functional 1 : D -> � by, for all A & 0, 
1(A) = JtdA, 
Endow 0 with the weak topology generated by {f I f e C(X)] and let lT
inherit the relative weak topology from ii. Then under this topology, 
6 
( i) IT is a convex subset of the locally convex topological
1 inear space !l. 
( ii) 1T is (weakly) closed in a since any A t TT can be 
''separated'' from TI by an appropriately chosen f (using 
the fact that p B 1T -> q* >> p). 
(iii) TT is (weakly) compact by Lemma 2.8 of Futia [1980, p. 
27] since the closure of any tight set of probability
measures is compact in the weak topology we are using, 
The Tychonoff fixed point theorem (see Smart [1980, p. 15]) 
can be invoked if we can show that T : Tf -> 1T is (weakly) 
continuous, A fixed point p* s TT for T will then satisfy the 
requirements of the theorem, We now show that T is indeed continuous. 
Let V be the subbasic open neighborhood of � = Tp0 s IT 
generated by e> 0 and f a C(X), Thus 
v £q e Tfl lf<q - �l l < sl = £q s ITI lfrd(q - �>I < sl. 
For this f e C(X) define h : X -> 2 by 
h(y) = Jrdpy
It follows that h e  C(X). To see this consider two cases: 
(a) If f is a simple function, say f 2 c. XE , then
i=l l. i 
h( y) = J [ 2 c. XE ] dp = 2 c i PE ( y) i=l 1 i y i=l i 
So h is continuous since each PE. is continuous. 
1 
(b) For the general case, since f is continuous and bounded, 
we can construct a sequence of simple functions {fn} for 
which 
sup If - fl -> 0 as n -> m n 
Then, define hn by 
hn(y) = Jfndpy
It follows that for any y e X, 
lhn<Y> - hCy> I lf<fn - f)dpyl i Jlfn - fldpy
i sup lfn - fl
So {hn} is a sequence of continuous functions converging 
uniformly to h. Hence h is continuous. 
Further, ¥ p s Tr, we prove that 
7 
h(p) = 1(T ) • 
p (*) 
Again, it suffices to show that (*) is true for simple functions. 
Thus, let f = � c. XE • Then
i�l 1 i 
h(p) = Jhdp = J [ Jfdpy ] dp
J [ J l c i XE. dpy ] dp1 
l Ci [ J [ J XE_dpy] dp ]1 
l ci [ JPE.dp ] = }ciTp(Ei) 
1 
= J 2 ci xE.dTp = JfdTp = 1<Tp).
1 
8 
Let U = {p e TTI lhCp - p0)1 < e}, a subbasic open neighborhood of p. 
Then 
P s u -> lhCp - Po>I < s -> li<P> - hCp0>1 < s 
-> 11CTp) - f(Tpo>I < 8 -> lr(Tp - �>I < 8 
-> Tp e V. 
Thus T : rr -> TT is continuous and the proof is complete. 
Q.E.D. 
9 
1.._ Comparison of Processes in Different Spaces, 
The above theorem can be extended to the case where the Markov 
processes p and q reside in different spaces, Of particular interest 
is the case when p is a Markov process over an arbitrary measurable 
topological space Y, while q is a Markov process over some measurable 
subspace X of I. Let X and .! denote the Borel sets on Y and X
respectively. We assume there is a X measurable function K : Y -> X
such that for all c e I, {y a Y I K(y) � c} is compact, Let M(X) and 
M(Y) denote the sets of probability measures on X and Y, respectively. 
Also set I : X -> X be the identity function. Now, given any 
p e M{Y), p induces a measure p o K-l e M(X) in the natural way: i.e. 
(p o K-1)(A) = p (K-1( A)), We can now define stochastic dominance, 
For p e M(Y), and q e N(X), we say that q stochastically 
dominates p with respect to K, written q )) p iff 
q >> p o K-l with respect to I. 
( Note that both q and p o K-l reside in N(X), so this is the same use 
of stochastic dominance as before,) Next, given two Markov processes, 
p and q over Y and X respectively, we say that q stochastically 
dominates p with respect to K, written q )) p iff, for all y e Y, 
qK(y) )) Py
The definition of stochastic increasing Markov processes remains 
unchanged since all the action takes place within a single space, 
However, since X '::_ I, we can use the identity function, I, for the 
mapping L. Then we say, for a Markov process q over X, that q is 
stochastically increasing if it is stochastically increasing with 
respect to I. 
We can now prove a generalized version of Lemma 2. 
Lemma 3, Let p and q be Markov processes on Y and X respectively
where X c I, and let p e M(Y) and q e M(X). Further, assume 
( a) q is stochastically increasing 
(b) 
(c) 
Then if P1 
pl(A) 
q » p 
q » p 
1X -> I, and q 
I ;Adp 
ql(B) = f qBdp
it follows that q1 >> p1• 
! -> I are defined by, ¥ A s I, B e !
Proof: Setting L I, we have for any c B I, the sets Sc are of the
the form 
s c (x e X I x i c}. 
We must show, for all c, that 
10 
Pl o K-1 ( s ) l. q1 ( s ) c c 
or, setting R0 {y s Y I K(y) i c}, we must show 
pl(R l l. i<s > c c 
for all c, But now q >> p => qK(y) >> Py (¥ ye Y). 
-
- -1 
-> qK(y) ) )  Py o K (¥ y & Y) .
-1 => qK(y)(Sc) i Py o K (Sc) = py(Rc)(¥ ye Y). 
Or, written differently, for all c, and all y, 
Hence 
PR (y) l. q8 (K(y))c c 
1 s- f-p < Rcl •= PR dp l. qs o K dp c c 
2. f -qs dq c 1 q (S )c 
qS o K(y)c 
f qS d(p o K-1) c 
The next to last step follows from Lemma 1 using the fact that 
q ) )  p ° K-l with respect to I, and the fact (from assumption (a)} , 
that qS is monotonic decreasing with I. Hence p ° K
-l << q with 
c 
respect to qS • 
c 
11 
Q.E.D. 
This gives us the following corollary to Theorem 1. 
Corollary l, Let p be a Markov process on Y, and q be a Markov 
process on X c I such that 
(a) q is stochastically increasing 
(b) q » p 
(c) For all A & I, PA is a continuous function on X,
12 
Then if q has a stationary distribution, q*, then p has a stationary 
distribution, p*, and q* >> p*, 
ProQf: The argument is exactly the same as that of Theorem 1. Lemma 3 
establishes that the mapping T maps Tf into 11.
4. An Example and an Application. 
Our initial intuition led us to believe that Theorem 1 would 
be true without condition (a). We close with an example illustrating 
that the result is not true without this condition. 
Let X = ,N, the natural numbers. Write p(i,j) for p({i}, j), 
and similarly for q(i, j). Then define p and q as follows. For j odd, 
set 
-;:-(i,j) = { � 
and for j even, set 
if i = 0 
otherwise 
13 
{ ai if i = j + 2 
�(i.j) = (1 -ai) if i = j + 3 
0 otherwise 
Then let q be defined to be the process r generated when 
ai = 1/2 for all i, and let p be any process generated when ai ) 1/2
for all i and when 1T ai ) O. Clearly q >> p with respect to thei=O 
identity function, and q has a stationary distribution q 
[specifically, writing q(i) for q({i}) and so forth, q(2i) = 2-i 3-l • 
q(l} = 0, and q(2i+3) = 2-(i+l) 3-11, but it will be shown that p has 
no stationary distribution. 
Suppose that p has a stationary distribution p, i.e., that 
Tp = p where T: 1T -> 1T is defined by Tr(A) = } p(Afi)r(i). Denote 
�o 
the even numbers by 2N and the odd numbers by 2,N+l. It is evident 
that if r(2,N+1) = 1, then Tr(2,N+l) O. Thus p(2,N) > 0, so p(2j) > 0 
m 
for some j. For some k > j, p(2k) < 1T aip(2j). However,i=O 
2k-l m 
�(k�) - -p(2k) =I- p ( 2k) = II aip(2j) l II a.p(2j), a contradiction.i=2j i=O 1 
Since p and q satisfy hypotheses (b) and (c) of Theorem 1 (continuity 
of PA is trivial because N is a discrete topological space), the 
indispensability of assumption (a) for the theorem is established. 
The motivation for the ideas developed in this paper emerged 
from investigation of a stochastic model of committee voting behavior 
(Ferejohn, McKelvey, and Packel 1981). We conclude with a brief 
description of how Theorem 1 applies in this setting, 
14 
The underlying space X is Euclidean and represents the set of 
alternatives, one of which must be selected by a finite set of voters 
operating under majority rule. Each voter has an ''ideal point'' in X, 
with voter preference for any given alternative decreasing 
monotonically with its Euclidean distance from the voter's ideal 
point. A stepwise selection procedure is modelled as a discrete time 
Markov process p over X, whose stationary distribution p* provides the 
desired solution concept. When X is a compact subset of �m 
convergence follows readily, but the noncompact case is theoretically 
important and leads to a number of difficulties. By studying a more 
tractable process q , a Markov chain over the positive integers, which 
satisfies the conditions of Corollary 1, we are able to establish the 
existence of p* in a fairly general setting. Corollary 1 also ensures 
that the stationary distribution q* for q satisfies q* >> p*, 
providing useful bounds on the concentration of p* in terms of the 
voter's ideal points, 
It seems reasonable to expect the Theorem and Corollary of 
this paper to have application in a variety of practical and 
theoretical situations of this sort, In this role, the result serves 
as a ''dominated convergence theorem'' for Markov processes. 
15 
REFERENCES 
Ferej ohn, J., R. McKelvey, and E. Packel. ''Limiting Distributions for 
Continuous State lllarkov Voting Models.'' Social Science 
Working Paper #394. 
Futia, Carl A. ''A Stochastic Approach to Economic Dynamics.'' 
Unpublished manuscript, Bell Telephone Laboratories, 1980. 
Smart, D. R. Fixed Point Theorems. Cambridge: Cambridge University 
Press, 1980. 


Methods for Comparison of Markov Processes by Stochastic Dominance

https://authors.library.caltech.edu/82099/1/sswp395.pdf

Methods for Comparison of Markov Processes by Stochastic Dominance

Abstract

Similar works

Full text

Available Versions

Caltech Authors - Main