D. Blackwell

L. Dubins

English

Research Papers in Economics

Merging of Opinions with Increasing Information
Author(s): David Blackwell and Lester Dubins
Source: The Annals of Mathematical Statistics, Vol. 33, No. 3 (Sep., 1962), pp. 882-886
Published by: Institute of Mathematical Statistics
Stable URL: http://www.jstor.org/stable/2237864
Accessed: 08/12/2010 11:08
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at
http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless
you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you
may use content in the JSTOR archive only for your personal, non-commercial use.
Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at
http://www.jstor.org/action/showPublisher?publisherCode=ims.
Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed
page of such transmission.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.
Institute of Mathematical Statistics is collaborating with JSTOR to digitize, preserve and extend access to The
Annals of Mathematical Statistics.
http://www.jstor.orgMERGING  OF  OPINIONS  WITH  INCREASING  INFORMATION' 
BY DAVID  BLACKWELL  AND LESTER  DUBINS 
University of California, Berkeley 
1.  History.  One of us [1] has shown that  if Zn,  n  1, 2,  *  is a stochastic- 
process with  D  states,  0,  1, *  ,  D-  1 such that  X  En=  Zn/D'  has an 
absolutely  continuous distribution with  respect to  Lebesgue measure, then the 
conditional  distribution  of  Rk  =  En=1  Zk+n/Dn  given  Z1,  *  ,  Zk  converges, 
with probability one as k  -*  o to the uniform distribution on the unit interval,, 
in the sense that for each X, 0 <  X <  1, P(Rk  <  X I Zl X  ***  , Zk)  -  Xwith- 
probability  I  as  k  -*  oo. It  follows  that  the  unconditional  distribution  of  Rk 
converges  to the uniform  distribution  as k -?  co. If {Znj  is stationary,  the distri- 
bution  of  Rk  is independent  of k, and hence uniform, a result obtained  earlier 
by  Harris [3]. Earlier work relevant to  convergence of opinion can be found in 
[4, Chap. 3, Sect.  6]. 
Here we generalize these results and also show that  the  conditional distribu-- 
tion of Rk given Z1,  *  *  *,  Zk  converges in a much stronger sense. All probabilities. 
in this paper are countably additive. 
2.  Statement  of  the  theorem.  Let  65i be  a  a-field of  subsets  of  a  set  Xi,. 
i  =  1, 2, *  *  *; and let (X, 63)  =  (X1 X X2 X  * * *,  6(1  X  6(2 X  *.. ). Suppose 
(X,  63, P)  is  a  probability  space  and  let  P.  be  the  marginal distribution  of 
(X1  X  ...  X  Xn,  681  X  ...  X  Bn); that  is, Pn(A)  =  P(A  X  Xn+1 X  ... 
for all A  e  6h X  ...  X  6.n  . The probability P  is predictive  if for every n  >  1, 
there exists a conditional distribution Pn for the future Xn+i X  *..  given the past 
X1  X *  *  ,  Xn ; that  is,  if  there  exists  a  function  Pn(xi, X *  *  ,  x.)  (C)  where 
(xi,  *  *  *, x.)  ranges over X1 X  ...  X X.  and C ranges over  6,n+i  X  ...  with 
the  usual three properties: Pn (xi,  *  *  *,  x.) (C)  is  6?1  X  *  *  * X  63X-measurable 
for fixed C; a probability distribution on  (Xn+l  X  ...  ; 6n+1 X  ...  for fixed 
(xi,  ,  Xn); and for bounded (9-measurable q5 
(  fdcdP =  f[((xi,  x.,  x+,  )dPn  (xn+l,  xl,  *.. 
*dP  (xi,  x.n) 
holds. 
The assumption that  P  is predictive is mild and applies to  all natural prob- 
abilities known to us. It is easy to verify that any probability which is absolutely 
continuous with respect to a predictive probability is also predictive. 
Received December 12, 1961. 
1 This  paper was prepared with  the  partial  support of the  Office of Naval  Research 
(Nonr-222-43) for Mr. Blackwell;  and with  the  partial  support of the  National  Science 
Foundation, Grant G-14648  for Mr. Dubins. This paper in whole or in part may be repro- 
duced for any purpose of the United States Government. 
882 MERGING  OF  OPINIONS  883 
For  any  two  probabilities  Al and  ,u2 on the  same  a-field i:, the  well  known 
distance p(rn  ,  g2)  between gi  and g2 is  the  least  upper  bound  over  D E CF  of 
Itl (D)  -2(D)  |. Of course gi  is absolutely  continuous with  respect to  (gil + 
A2)/2=  m  and  has  a  density  qi,  so  that  P(gl  ,  82)  =  fA(01  -  q52)dm  = 
(1/2)f  q'  -  14)21  dm where A  is the  set where q5i -  42  >  0. 
MAIN  THEOREM.  Suppose that P  is a predictive  probability on (X,  6() and that 
Q is absolutely  continuous with respect to P.  Then for each conditional distribution 
p'  of the  future given the past with respect  to P, there  exists a conditional distribution 
Qn  of the future given the past with respect to Q such that, with the exception of a 
set of histories (xi ,  ,  * * X Xn  Xn  +1  ,  *)  of Q-probability  0,  the distance between 
pn(Xl,  ...  ,  xn)  and Q'(x1,  X  .  Xn) converges  to 0 as n converges  to oo. 
3.  Martingale  preliminaries.- The  proof  of  the  theorem  requires a  slightly 
generalized martingale convergence theorem. Say that a sequence {yn} of random 
variables is dominated  in the sense of Lebesgue  if supn lYnl has a finite expectation. 
THEOREM  2.  Suppose that {yn},  n  =  1, 2,  ***,  a sequence of random variables 
dominated in the sense of Lebesgue,  converges  almost everywhere  to a random varia- 
ble y. Then  for every  monotone  increasing or monotone  decreasing  sequence  of a-fields 
"uj  j  =  1, 2,  *** converging  to a a-field ct, 
(2)  lim E[yn  I 'ltj]  =  E[y  I 'it], 
n--3.oo 
almost everywhere  and in L1 . 
In  this  note  we  are  primarily  interested  in  the  weaker  conclusion  that 
limn coo  E[Yn  I 'an] =  E[y I  St]. The  two  important  special  cases in which either 
yn or 'an is independent of n are in [2]. 
PROOF  OF THEOREM  2.  Let  gk  =  sup  yn for n  ?  k. Equalities  and  inequalities 
below are asserted to hold with probability 1. Fix k for a moment and let n  ?  k. 
Then  yn  <  ~gk  and 
(3)  E[yn I  'il]  _  E[gk  c  il 
Letting 
z=  lim  sup  E[y,  cti] 
j  i ~!j 
(4)  n>.? 
x  =  lim  inf  E[yn  I  cttlI, 
j  i ~!j 
n?j 
you  conclude from  (3)  and  a  usual form of  martingale  convergence  theorem 
[For example, see 2, Theorem 4.3, Chap. VII] that 
(5)  z  _  limj  supi?j E[gk  I qILi] =  limi  E[gk  I St,]  =  E[gk  I 'a]. 
Therefore z _  lim E[gk  I 't]  =  E[y I  Sit] by Lebesgue's theorem  suitably general- 
ized  so as to  apply  to  conditional expectations.  [See, for example, 2,  CE6 Sec- 
tion  8,  Chap.  1]. Similarly, x  >  E[y I 'L], and the  proof of almost everywhere 
convergence is complete. The proof of Li convergence is routine and omitted. 884  DAVID  BLACKWELL  AND  LESTER  DUBINS 
COROLLARY 1. Suppose that with probability 1, only a finite number of the events 
E1,  E2,  -  occur. Then for any monotone sequence of a-fields 9Tl,  912,  **. 
(6)  P[ UEk  I'xj]  and  P[E  |'uj]-  O. 
k_n 
almost  surely  as n and  j  ->  oo. 
COROLLARY 2.  If fn is  any sequence of random variables that converges  almost 
everywhere  to 0 and 'tj  is  a monotone sequence  of a-fields, then with probability 1, 
for all  e  >  0, 
(7)  P[supk>n  Jfkl >  6 |  ci.],  and  P[If.l  >  e I ctj] 
converge  to 0 as n and j  converge  to  oo. 
COROLLARY 3. Let q _  0 be a density function for which Q(B)  =  fE qdP for all 
B c 6(; let 
(8)  q. (xi,  *  x)  =  fq(xi,  .  '  x1,Xn+1,  ) dP(xn+l,  xl,  X*  ) 
and let 
(9)  dn(x,  ,  ,  X  )  =  q(X  *  Xn  X  xn+i  * 
qn(xl  ,  *  , xn)  or  1, 
according  as qn(xi,  *  , xn)  0  0  or not. Then, with P-probability 1, for all e  >  0, 
(10)  P[dn-  1  > e  IX1, 
...  Xn]  O  as  n  oo, 
and with Q-probability  1, for all  e  >  0, 
(11)  Q[ldn-  11 >  E Ixl , *  *xn  Xl  O  as  n  oo. 
PROOF  OF COROLLARY 3. With respect  to P measure, 
(12)  E[q lxl,  .,  x]  -  qn\(Xi,  *  *  *, Xn) 
so  that  according to  Doob's  martingale  convergence theorem,  qn(Xl  .  *  xn) 
converges to q(xi,  *  *,  x, Xn+1,  * *.*  )  almost surely with respect to P.  Conse- 
quently, lim dn  <  1 a.s. P and dn -*  1 a.s. Q since q >  0  a.s. Q.  An  application 
of Corollary 2 completes the proof. 
4.  Proof of main theorem. Define 
Qn(Xl  .  Xn)  (C) 
(13)  =|dn,  (Xi, *  Xn  Xn+1,  *  *  dpn(X,+1,  ..  I X1,*i  X-n), 
for  all  C  c  63n+iX MERGING  OF  OPINIONS  885 
It is routine to verify that  Qf is a conditional distribution for the future given 
the past. Let u  =  (xl,  - - *, xz)  and v =  (x.?1,  ***),  and compute thus: 
pP  (P'xl  Xn) Qn(xl,  . 
z  X n) ) 
p(pn(U),  Qn(U)) 
=  f  (dn(u,  v)  -  1)dPn(v  U)  over  v:dn(u,  v)  -  1  >  0 
(14)  _  - +  f dn(u, v)dP'(v  I u) over v:  dn(u, v) -  1 >  E 
E  +  Q n(u)(v:dn(u,  v)  -  1  >  E) 
=  E +  Q[dn -  1  >  e  I Xl,  * * *  Xn 
=  E  +  E 
for all but  a finite number of n with  Q-probability 1, according to  (11).  This 
completes the proof. 
5.  Interpretation. Usually, there is essentially only one conditional distribution 
Qf  of the future given the past. Therefore, our theorem may  be interpreted to 
imply that if the opinions of two individuals, as summarized by P  and Q, agree 
only  in that  P(D)  >  0 O-* Q(D) >  0,  then they  are certain  that  after a suffi- 
ciently  large finite number of observations  xi,  ,  x-,--  their opinions will be- 
come and remain close to each other, where close means that for every event  E 
the probability that  one man assigns to E  differs by at most  e from the proba- 
bility  that  the other man assigns to it, where  e does not depend on E.  Leonard 
J. Savage observed that  our theorem applies to the particularly interesting case 
in which  P  and Q are symmetric  (or exchangeable).  That  is,  if the  measures 
P  and  Q on the  sequences xi  are those  that  arise when the  xi  are, for a fixed 
parameter  value,  independent  and  identically  distributed  observations,  with 
prior distributions  p  and  q on  the  parameter,  then  the  relations  of  absolute 
continuity  between P  and Q are precisely those between p and q. 
6.  Caution.  Though  the  conditional  distributions  of  the  future  Pn and Qn 
merge as n becomes large, this need not happen to the unconditional distributions 
of  the  future.  That  is,  let  P(n)  (D)  =  P(X1  X  ...  X Xn X D)  for  all 
D c 63n+l X  - *  ,  and let  Q(n)  be similarly defined. The  following is a simple 
example of two probabilities P and Q absolutely continuous with respect to each 
other for which P(n)  and Q(n)  do not  merge with  increasing n.  Let R be the 
probability  on infinite sequences xl,  X2,  *** of O's and l's  determined by inde- 
pendent  tosses  of  a  coin which  has probability  r of  success,  and let  S  be the 
probability determined if the coin has probability s for success, with 0  ?  r <  1, 
0  <  s  ?  1, and r #  s. Now let 0  <  p  <  q <  1 and let P and Q be mixtures of 
R  and  S:  P  =  pR  +  (1  -  p)S,  Q  =  qR  +  (1  -  q)S.  Since  P(n)  =  P  and 
Q(n)  =  Q for all n, there is no tendency  for P(n)  and Q(n)  to merge. 886  DAVID  BLACKWELL  AND  LESTER  DUBINS 
7.  An application. By viewing the unit interval as a product of two point spaces, 
the  interested  reader will see that  the  main theorem yields  information about 
the local behavior of positive integrable functions q(x)  defined for 0  <  x  <  1. 
REFERENCES 
[1] BLACKWELL,  DAVID  (1957). On discrete variables whose sum is absolutely continuous. 
Ann. Math. Statist. 28 520-521. 
[2] DooB, J. L. (1953). Stochastic  Processes. Wiley, New York. 
[3] HARRIS,  T. E.  (1955). On chains of infinite order. Pacific J. Math. 5 707-724. 
[4] SAVAGE,  LEONARD J. (1954). The Foundations of Statistics. Wiley, New York. 

Merging of Opinions with Increasing Information

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics