In these notes ( which supplement my SINAPE lecture) I will survey some early and some current results and applications of the dominating point construction in large deviation theory

Ney, Peter

English

Cadernos Espinosanos (E-Journal)

Resenhas IME-USP 1999, Vol. 4, No.1, 79 - 91. Notes on Dominating Points and Large Deviations Peter Ney Abstract: In these notes ( which supplement my SINA-PE lecture) I will survey some early and some current results and applications of the dominating point construction in large deviation theory. 1. Dominating points and large deviation asymptotics. Let A(a) : JRd -t JR be a convex function, and let A*(x) = sup [< a,x > -A (a)), x E JRd, aERd be its convex conjugate. ( < .,. > is inner product). Let B C JRd, BO = interior of B, B =closure of B, BB = boundary of B. Let D(A) = {a E JRd : A(a) < oo}, assume tha.t A is differentiable on D , and let R('\1 A) = the range of the map '\1 A. Definition (N, 1983,84). A point VB is called a dominating point of (A, B) if (i) VB EBB and inf[A*(v) : V E Bj = A*(VB)' (ii) '\1 A(a) = VB has a solution aB E D()"), and (iii) Be {x:< x,aB > ~ < VB,aB >}d~fHaB' The situation that concerns us here is when J-t is a probability measure on JRd , p,(a) = J e<a,x> J-L(dx), a E JRd, and A(a) = logp,(a). Let S = the closure of the convex hull of the support of J-t. Then we have Theorem 1.1 ([N2j, 1984). Assume that D()") contains a neighborhood of the origin, A is essentially smooth, B is convex with [BnSjO =j:. cP and m = f xJ-t(dx) i. B. Then a unique dominating point VB for (A, B) exists. The construction is illustrated in Figure la. 80 Peter Ney "" (x:J\<x)aJ\<uS> J Level sets of ,; Figure 1a. Remarks (i) Note that (ii) Also note that for v E 1JO(A*) :::: SO and thus CXB :::: VA-I(VB) :::: VA*(VB), as ilustrated in Figure 1. See e.g. Rockafellar [R] (1970) for such prop-erties of convex functions. (iii) A sufficient condition for essential smoothness is that 1J(A) be an open set. Extensions of the above construction to infinite dimensional spaces have been treated by Bolthausen [Bo] (1984), Dinwoodie [Di] (1992), Einmahl and Kuelbs [E,K] (1996), and Kuelbs [K] (1998). A related concept, called an exposed point is discussed in Dembo and Zeitouni [D,Z] (1993). We start with a sequence X o, Xl, ... of independent, identically distributed random variables (i.i.d.r.v.'s), taking values in IRd , with probability law J..l. Let A Notes on Dominating Points and Large Deviations 81 be as above, and define the probability measure (1.3) Rd = Borel sets. Let Sn = Xl + ... + X n. Take any a, v E !Rd, BE Rd. Then a straight forward manipulation and change of variable yields where In(B, v, a) = J e-<a,s> J1~n(ds + nv). n(B-v) ( J1*n = the n fold convolution of J1 ). (1.4) (1.5) Assume B is as in Theorem 1.1 . We are free to choose a and v as we like. We take v = VB = the dominating point of B, and take a = aB = the solution of V'A(a) = VB, which exists by theorem. Now we note two important properties. First and hence *n ( ) def - *n ( ) J1a{3 . + nVB = J1B . is the p.m. of a sum of i.i.d.r. 's with mean zero. Secondly We can thus write where in the notation of (1.5) In(B) = In(B,vB,aB) = J e-<aB ,X > jJ,jp (dx). n(B-vB) We call (1.9) the representation formula for P{ ~ E .}. Now from property (iii) in the definition of dominating point < aB, x > ~ 0 for x E n(B - VB), (1.6) (1.7) (1.8) (1.9) (1.10) (1.11) 82 Peter Ney and hence we immediately see that In(B) :::; 1 ( for convex B ), and we have the upper bound p{: E B} :::; e-nA*(B), (1.12) (where we write A*(B) = inf{A*(v) : v E B} ). By a standard covering argument one can then extend (1.12) to get the clas-sical Cramer large deviation upper bound, namely: for all compact F lim sup ~ IOgp{: E F} :::; -A*(F). (1.13) If also 0 E VO(A), then (1.13) also holds for all closed F. By applying elementary ( central limit ) estimates of {LBn one gets lower bound estimates on In(B), yielding the large deviation lower bound lim inf ~ log P {~ E G} ;::: - A * (G) for open G. ( See e.g. [Nl] ). (1.14) But the real value of the representation formula is that sharper central limit estimates ( as can be found in Bhatacharya and Rao [B,R] (1976) ) can now be used in (1.9) to obtain asymptotic expansions for P{ ~ E f}, r E JRd. These finer results depend on smoothness properties of r in the neighborhood of the dominating point Vr, provided the later exists. (Do.minating points, or even dominating sets, may also exits for non-convex sets). These ideas were developed in [Nl] (1983) and more throughly by Ilts [II] (1995). Up to first order one finds that (1.15) where -00 < , :::; d;2, and, depends on the geometry of r. ( 0 < 8, C < 00 are constants). With the author's permission I reproduce some of the illustrations (for various ,'s) from [II] in Figures 1-6. Figure 1. Case i : r = Va = Complement of the level set of the rate function. Here the dominating set Sa has dimension d - 1 and, = (d - 2)/2. Notes on Dominating Points and Large Deviations Figure 2. Case ii: Intersection of a convex set B with the com-plement of the level set of the rate function. Here the dominat-ing set Sa n B again has dimension (d -1) so that 'Y = (d - 2)/2 is maximal ( as in case i ) . Figure 3. Cases iii and v: Set r with unique dominating point v E Sa. Here (d - 2)/2> 'Y ::=:-: -~ Figure 4. Cases iii and iv: Set r with unique dominating point v E Sa· Here (-~ < 'Y::::; -~ and 1 > f3 > ~). 83 84 Peter Ney Figure 5. Case iv: Set r with unique dominating point v E Sa having a cusp at v ( here I < -~ and (3 > 1 ) Figure 6. Example of a dominating set of dimension d- 2 found by removing a convex set Bend from the set obtained by following the normals to the portion B n Sa of the level surface Sa. (Compare with Figure 2 above). Here the dominating set is BB n Sa where BB is the boundary of B. 1= (d - 4)/2. A similar situation prevails in the Markov case. Let X o, Xl, ... be an aperi-odic, irreducible Markov chain, taking values in a state space S, which we take to be finite for simplicity. Let {p(x,y);x,y E S} denote the transition matrix of {Xn}, and 7r(x) be its invariant measure. (Under suitable hypotheses on the MC, the following results also extend to general state spaces; see e. g. Iscoe, Ney and Nummelin [I,N,Nu] (1985) ). Let! : S -t JRd and Sn = E~=l !(Xi ). We want analogues of the represen-tation formula (1.9) and asymptotics like (1.15) for IPx{Xn = y,Sn E nr}. Define the matrix {Pa(x,y)} = {e<a,f(y»p(x,y), X,y E S, a E JRd}. (1.17) Then {Pal will have a maximal, real ( Perron-Frobenius ) eigenvalue 'x(a) with Notes on Dominating Points and Large Deviations 85 associated right and left eigenvectors Ta(X), la(x), xES. Let 1fa(x) = la(X)Ta(X),X E S, normalized so that LXEs1fa(x) = 1. We will see that >.(a) plays a role analogous to the generating function [l(a) in the ij.d. case. Let A = log >., A * = convex conjugate of A. Now define the matrix (1.18) and note that this is a stochastic matrix ( irreducible, aperiodic) with invariant probability measure 1f a. Define the "Markov-additive" kernels (1.19) whose multiplication is governed by q~'+m)(x,A x r) = L ! qin)(x,y x ds)q;;'(y,A x r - s), r c JRd. yESRd Now some algebra yields the analog of (1.4) and (1.5), namely for a, v E JRd where In(B,v,a) = ! e-<a,s>qin)(x,y x (ds+nv». nCB-v) (1.20) (1.21) The existence of a dominating point for (A, B) follows by specializing Theo-rem 5.2 (i) of [I,N,Nu]. Namely we have Lemma Let B be convex with [BnR(VA)]O =I- 4> and assume Err!(X1 ) f{. B. Then there exists a unique dominating point VB of (A, B). Define aB as the solution of VA(a) = VB, and take a = aB, V = VB in (1.20) and (1.21). Abbreviate Then we have (1.22) Now {q~~} is "centered" in the sense that L! 1faB(dx)sqi~(x,y X ds) = nVB x Rd and so central limit estimates for centered sums of functions of Markov chains can be applied to (1.22) to yield results like (1.15). Some details of such calculations 86 Peter Ney are in [1,N,Nu] and [12] (1998). Central limit estimates for Markov chains can be found e. g. in Nagaev [Nal], [Na2] (1957), (1961). 2. Dominating points and conditioned limit laws for independent ran-dom variables. Let XO,XI , ... be i.i.d.r.v.'s taking values in a measure space (8,S),P{Xo E . } = JL( . ), 9 : 8 -+ IRd, u : 8 -+ IRm , measurable functions, 0 < d, m < 00, Gn = I:~:OI g(Xi), Un = I:~:OI U(Xi). We are interested in the limiting behavior ( as n -+ 00 ) of conditioned measures of the form (2.1) and of (2.2) Example: Let Xl, X 2 , ... be i.i.d. Bernoulli r. v.'s with P{Xi = +1} = P{Xi = -1} = ~, and take 0 < a < 1. Then nl~~ P { X I = + 11 t Xi ~ an} = 1; a. i=l (2.3) There is an extensive literature on this subject under the heading "Gibbs Conditioning principle". A basic paper is Csiszar [Cs] (1984). A general reference with extensive bibliography in Dembo and Zeitouni [D,Zl]. We focus here on an approach based on the dominating point construction. We follow an approach of Lehtonen and Nummelin [Le,Nu1) for LLd. case, and then discuss some extensions involving Markov chains. First we need some definitions. For a sequence og measures v, VI, V2, . . . on (8, S), write vn~V on IRd if <PVn -+ <PV for all bounded measurable <p : 8 -+ IRd. ( Abbreviate Is <p(s)v(ds) = <pv ). For a sequence ofr.v .'s {Zn} c IRd , write zn'=fzo with respect to {lPn} as n -+ 00, if given any E > 0 there is an a > 0 such that (2.4) For a sequence of random measures {ji,n} on (8, S), write {ji,n} ~ JL w.r. to {lPn} if <pjj,n '=f<pJL w.r. to {lPn} for all bounded, measurable <po Later we will use Lemma 2.1 [Le,Nu1] Ifjj,n~JL w. T. to {lPn} then (2.5) Notes on Dominating Points and Large Deviations 87 Let S(j.l) = convex hull of support of j.l, and Ag(a) = log f e<<>,g(x» j.l(dx). We can now state some conditioned limit laws. Lemma 2.2. Let 9 be bounded and B C JRd be open and convex. Let Sg = S(j.lg-l) and assume that [B n Sg]O =I- 4;. Assume that JEg(Xo) ~ B. Let Vb be the dominating point of (B , Ag) and aB E JRd be the solution of \7 Ag(a) = Vb. Then (2.6) with respect to the measures (2.7) Idea of proof. VB is the unique point at which A * achieves its minimum. Hence by its convexity, A*(v) > A*(VB) + fJ for all V outside an to-ball around VB and some fJ > O. Now applying the definition of conditional probability, and the fact that lim ~ log P {-; E B} = - A * (VB) ( by Cramer's large deviation theorem ), it follows that (2.8) for some a> O. From Lemma 2.2 one can go to a more general conditioning of the form (2.1), where the conditioning functions u differ from g. The idea is to apply the lemma to f = (g , u) : S -+ JRd+m, and Sn = (Bn, Un), where u, g, Un and Gn are defined above (2.1); with B = JRd X C, C C JRm. Let Au(f3) = logfse<{3,u(x»j.l(dx), f3 E JRm and j.l{3 (A) = e- Au ({3) fA e<{3 ,u(x»j.l(dx). Then one obtains Lemma 2.3 [Le,Nu2] Let u and 9 be bounded, C C JRm be open and convex with [CnSu]O =I- 0, Eu(Xo) ~ C. Let Vc be the dominating point of (Au, C), and f3c E mm be the solution of \7 Au(f3) = Vc . Then (2.9) with respect to (2.10) 88 Peter Ney Now let Fn denote the empirical measure of {Xn} : _ 1 n-l Pn(A) = :;;: L lA(Xi ), A E S, i==O and note that Gn ! - --:;;:- = g(x)Pn(dx) = gPn · (2.11) s Then if u and C are as in Lemma 2.3, one can conclude that (2.12) for all bounded measurable g. Hence by Lemma 2.1 (2.13) or writing this out (2.14) But by simmetry, the terms in the above summand are all equal. Hence we have Theorem 2.1 [Le,Nu2j Let u, C and /3c be as Lemma 2.3. Then { I Un } b d 1P Xl E . -;:;: E C -+J-Lf3c on IR . (2.15) One can carry the above argument further to conclude Corollary 2.1. Under the above hypotheses on u, C and /3c (2.,16) where J-L~ is thek-foldproduct measure of J-Lf3. Thus in the sense of (2.16), Xl, X 2 , ... are "asymptotically independent". Csiszar [Cs] proved a stronger convergence for (2.7) under somewhat different conditioning. The case when k = kn -+ 00 has been studied in Dembo and Zeitoni [D,Z2] (1996) and Dembo and Kuelbs [DK] (1997). Notes on Dominating Points and Large Deviations 89 3. Conditioned limits for Markov chains. We briefly consider the case when X o, XI, ... is a ( time homogeneous) Markov chain. To avoid technicalities we describe the simplest situation namely when the state space S is finite, and {Xn} is irreducible and aperiodic. The general state space case has been studied in Schroeder [Sc] and Meda and Ney M,N1] and [M,N2]. Let P = {p(x, y), x, yES} be the transition matrix of {Xn}. Let 9 : S x S --+ JRd, u: S x S --+ JRm, Gn = L~=~l g(Xi ,Xi+l), Un = L~=~l u(Xi ,Xi+l). ( We use the same symbols u and 9 as before, but they are now functions on S x S ). Consider the matrix Pg(a) = {pg,a,{x,y)} = {e<a,g(x,y»p(x,y); x,y E S}, a E JRd. (3.1) Let >'g(a) = the maximal, real ( Perron-Frobenius) eigenvalue of Pg(a), with associated right ( left) eigenvector T g,a (x) (lg,a (x». Let Ag = log >.g. Let 7r g,a (x) = 19 ,a (x) Tg,a(X), normalized so that LkES 7rg,a(x) = 1, and let {qg ,a(x, y)} = {e-Ag(a) (T9.,,((y»)pg a(x,y)}. This is a stochastic matrix. Let Pu(f3), /3 E JRm, Tg,o. X ' and corresponding quantities be defined similarly for u. ( When S is finite and {Xn} is irreducible these quantities always exist, but for general state spaces fur-ther hypotheses are needed ). Let A * : JRd --+ JR = the convex conjugate of A and V(A*) = {x E JRd : A*(x) < oo}. Then we have the following analogues of Lemmas 2.2 and 2.3. Lemma 3.1 Let B C JRd be open and convex, with [B n V(A;)]O f:. 0. Let VB = the dominating point of(Ag,B) and aB = the solution of 'VAg (a) = VB. Assume E7rg ,aBg(XO, Xd ¢ B. Then (3.2) with respect to (3.3) The proof again follows from properties of dominating points. The analog of Lemma 2.3 becomes Lemma 3.2 Let C c JRm be open and convex with [C n V(A:)]O f:. 0. Let Vc be the dominating point of (Au, C) and /3c be the solution of 'V Au (/3) = VC. Assume IE7ru , iJc u(Xo, Xd ¢ C. Then ~n'=f 2.:: g(x, y)7ru,{3c (x)qu,{3c (x, y). x,yES (3.4) 90 Peter Ney with respect to From this we can argue as before that Lemma 3.3 Under the hypothesis of Lemma 3.2 1 n-l U lim - " P{Xi = X, X i+l = yl ~ E C} = 7ru {Jc (x)qu f>c (x, y). (3.5) n~oo n L..." n"fJ i=O This is the analog of (2.14). However, an analog of the symmetry argument used to get to (2.15) does not work here. In the finite state space case Csiszar, Cover and Choi [Cs,Co,Ch] (1987) use a careful counting argument to obtain a version of (2.15), but for general state spaces the question seems open. One can also extend (3.5) to 1 n U lim - LlP{Xi+l = Xl,··· ,Xi+k = Xkl~ E C} n--7OO n n i=l k-l = 7ru,{Jc(xd II qu,{Jc(Xi,Xi+1), i=l (3.6) and thus in this Cesaro convergence sense, {Xi, i = 1,2, . .. } conditioned on ~ E C is asymptotically Markov with transition matrix {qu,{Jc }. ( See [M,N2] for details of the above calculations and extensions to general state spaces ). References [B,R] Bhattacharya, R. N. and Rao, R. Ranga (1976). Normal Approxima-tion and Asymptotic Expansions. John Wiley & Sons. . [Bo] Bolthausen, E. (1986). Laplace approximations for sums of indepen-dent random vectors Part 1. Prob. Th. Rei. Fields (72) 305-318. [Cs] Csiszar, 1. (1984). Sanov property, generalized I-projection and a con-ditionallimit theorem. Ann. Prob. (12) 768-793. [D,K] Dembo, A. and Kuelbs, J. (1997). A Gibbs conditioning principle for certain infinite dimensional statistics. Univ. of Wisconsin technical report. [D,Zl] Dembo, A. and Zeitouni, O. (1993). Large Deviation Techniques. Jones and Bartlett, Boston. [D,Z2] Dembo, A. and Zeitouni, O. (1996). Refinements of the Gibbs condi-tioning principle. Prob. Th. Relat. Fields (104) 1-14. Notes on Dominating Points and Large Deviations 91 [Di] Dinwoodie, I. H. (1992). Measure dominantes et theoreme de Sanov. Ann. Inst. H. Poincare Prob. Stat. (28) 365-373. [E,K] Einmahl, U. and Kuelbs, J. (1996). Dominating points and large de-viations for random vectors. Pr40b. Th. Relat. Fields (105) 529-543. [11] Iltis, M. (1995). Sharp asymptotics of large deviations in JRd. J. of Th . Prob. (8) 501-522. [12] Iltis, M. (1998). Asymptotics of large deviations for Markov-additive chains in JRd . Univ . of Wisconsin technical report. (Also 1991 Ph. D. thesis) . [I,N,Nu] Iscoe, I., Ney, P. and Nummelin, E. (1985). Large deviations of uni-formly recurrent Markov additive processes. Adv. in Appl. Math. (6) 373-412. [K] Kuelbs , J. (1998). Large deviation probabilities and dominating points for open convex sets: non-logarithmic behavior. Univ. of Wisconsin technical report. [Le,Nul] Lehtonen, T . and Nummelin, E. (1988). On the convergence of empiri-cal distributions under partial observations. Ann. Acad. Sc. Fennicae. Ser. A. 1. (13) 219-223. [Le,Nu2] Lehtonen, T. and Nummelin, E. (1990). Level I theory of large devia-tions in the ideal gas. Int'l J. of Theoret. Phys. (29) 621-635. [M,Nl] Meda, A. and Ney, P. (1998). A conditioned law of large numbers for Markov-additive ch ains. Studia Sc. Math. Hungarica to appear. [M,N2] Meda, A. and Ney, P. (1998). The Gibbs conditioning principle for Markov chains. Univ. of Wisconsin technical report. [Nal,Na2] Nagaev, S. V. (1957,1961) . Some limit theorems for stationary Markov chains. Th. of Prob. and Appl's. (2) 378-406; (6) 62-8l. [Nl] Ney, P. (1983). Dominating points and the asymptotics of large devi-ations on JRd. Ann. Prob.(ll) 158-167. [N2] Ney, P. (1984). Convexity and Large Deviations. Annals of Prob . (12) 903-906. [Sc] Schroeder, C. (1993). I-projection and limit theorems for discrete pa-rameter Markov chains. Ann. Prob. (21) 721-758. Peter Ney University of Wisconsin, Madison ney@math.wisc.edu U.S.A. 

Notes on Dominating Points and Large Deviations

https://www.revistas.usp.br/resenhasimeusp/article/download/74894/78460

Notes on Dominating Points and Large Deviations

Abstract

Similar works

Full text

Available Versions

Cadernos Espinosanos (E-Journal)