We provide a coupling proof of Doob's theorem which says that the transition
probabilities of a regular Markov process which has an invariant probability
measure $\mu$ converge to $\mu$ in the total variation distance. In addition we
show that non-singularity (rather than equivalence) of the transition
probabilities suffices to ensure convergence of the transition probabilities
for $\mu$-almost all initial conditions

Kulik, Alexei

Scheutzow, Michael

Rendiconti Lincei - Matematica e Applicazioni

English

arXiv

Dieser Beitrag ist mit Zustimmung des Rechteinhabers aufgrund einer (DFG geförderten) Allianz- bzw. Nationallizenz frei zugänglich.This publication is with permission of the rights owner freely accessible due to an Alliance licence and a national licence (funded by the DFG, German Research Foundation) respectively.We provide a coupling proof of Doob’s theorem which says that the transition probabilities of a regular Markov process which has an invariant probability measure μ converge to μ in the total variation distance. In addition we show that non-singularity (rather than equivalence) of the transition probabilities suffices to ensure convergence of the transition probabilities for μ-almost all initial conditions

DepositOnce (Techn. Univ. Berlin)

Rend. Lincei Mat. Appl. 26 (2015), 83–92DOI 10.4171/RLM/694Probability Theory — A coupling approach to Doob’s theorem, by Alexei Kulikand Michael Scheutzow, communicated on 14 November 2014.Abstract. — We provide a coupling proof of Doob’s theorem which says that the transitionprobabilities of a regular Markov process which has an invariant probability measure m convergeto m in the total variation distance. In addition we show that non-singularity (rather than equiva-lence) of the transition probabilities su‰ces to ensure convergence of the transition probabilities form-almost all initial conditions.Key words: Markov process, invariant measure, coupling, convergence of transition probabil-ities, total variation distance.Mathematics Subject Classification: 60J05, 60J25, 37L40.1. IntroductionDoob’s theorem, as formulated in [1], p. 43, states that if the stochastically con-tinuous Markov semigroup Pt, tb 0 with Polish state space ðE; dÞ has an invari-ant probability measure (ipm) m and is t0-regular for some t0 > 0, then m is uniqueand all transition probabilities converge to m in the total variation distance. Here,t0-regular means that all transition probabilities Pt0ðx; :Þ are mutually equivalent.One common way to check t0-regularity is to show that the Markov semigroup isirreducible and strong Feller, see [1], Proposition 4.1.1 (known as Khas’minskii’stheorem).In fact, Da Prato and Zabczyk formulate and prove Doob’s theorem withrespect to strong convergence (which is weaker than total variation convergence)and refer the reader to [4] and [3] for di¤erent proofs of total variation conver-gence. Neither of the proofs is short and elementary. In particular, neither of theproofs uses coupling which has been a powerful tool to prove convergence oftransition probabilities in the past decades. The aim of this article is to providea coupling proof of Doob’s theorem. At the same time we generalize the resultin various directions: instead of a Polish state space, we just require a mild condi-tion on the measurable space ðE;EÞ. Further, we allow the inﬁmum of the timest > 0 for which Ptðx; :Þ and Ptðy; :Þ are equivalent to depend on the pair ðx; yÞwithout being uniformly bounded from above. We also show that we can replaceequivalance of the transition probabilities by the much weaker property of non-singularity but in this case convergence of the transition probabilities only holdsfor m-almost all initial conditions x. In the next section, we formulate the mainresults in the discrete time setting (in Remark 2 we say why the continuous timeclaim follows) and provide an example showing that the weaker assumption inCorollary 1 does not guarantee the conclusion of Theorem 1.2. Main resultsLet Xn, n a Zþ be a Markov chain with the state space ðE;EÞ. The measurablespace ðE;EÞ is assumed to be countably generated. We also assume that thediagonal D ¼ fðx; xÞ; x a Eg belongs to EnE. A typical example of such aspace is a Borel measurable space, e.g. a Polish space E endowed with the Borels-algebra E.Transition probabilities and n-step transition probabilities for X are de-noted respectively by Pðx; dyÞ and Pnðx; dyÞ. The law of the sequence fXng inðEl;EnlÞ with initial distribution LawðX0Þ ¼ m is denoted by Pm, the respectiveexpectation is denoted by Em; in case m ¼ dx we write simply Px, Ex.Recall that an invariant probability measure for X is a probability measure mon ðE;EÞ such thatmðdyÞ ¼ZEPðx; dyÞmðdxÞ:ð1ÞEquivalently, a probability measure m is invariant if the sequence fXn; n a Zþg isstrictly stationary under Pm.We use the usual relations for probability measures m, n on ðE;EÞ: m and nare equivalent (notation mP n) if each of them is absolutely continuous w.r.t.the other; m and n are singular (notation m ? n) if there exists A a E such thatmðAÞ ¼ 1, nðAÞ ¼ 0; otherwise m and n are non-singular (notation m 6? n). Thetotal variation distance between probability measures m, n on ðE;EÞ is the totalvariation of the signed measure m n (notation km nk).Theorem 1. Assume that for each x; y a E there exists n ¼ nx;y such thatPnðx; ÞPPnðy; Þ:ð2ÞThen there exists at most one ipm for the chain X. If an ipm m exists, then forevery x a EkPnðx; Þ  mk ! 0; n !l:ð3ÞThe proof of Theorem 1 is given in Section 3 below. The following theoremshows to what extent the basic assumption (2) can be relaxed.Theorem 2. Let X be a Markov chain which has an ipm m. Assume further thatfor mn m-almost all ðx; yÞ a E  E there exists n ¼ nx;y such thatPnðx; Þ 6? Pnðy; Þ:ð4ÞThen (3) holds true for m-almost all x a E.84 a. kulik and m. scheutzowThe following corollary is simple and straightforward.Corollary 1. Assume that for each x; y a E there exists n ¼ nx;y such that (4)holds true.Then there exists at most one ipm for the chain X. If an ipm m exists, then (3)holds true for m-a.a. x a E.To prove uniqueness of an ipm (which is the only addition to Theorem 2), letus assume that there exist two di¤erent ipm’s m1, m2, and consider the averagedipm m ¼ ð1=2Þðm1 þ m2Þ. Applying Theorem 2 ﬁrst to m1 and then to m, we get acontradiction: because m1 is absolutely continuous w.r.t. m, we get that for m1-a.a.x a E the transition probabilities Pnðx; Þ converge both to m1 and to m, butm1Am. rRemark 1. Note that the condition of Theorem 2 alone does not yield unique-ness of the ipm for X : a simple counter-example is given by a chain with a ﬁnitestate space with at least two mutually disconnected classes of states.Remark 2. Note that the results of Theorem 1, Theorem 2, and Corollary 1 arealso true in the continuous time case when Pt, tb 0 is a Markov semigroup(no regularity in t is required). To see this, note that uniqueness of an ipm m forthe discretized chain Pn, n a N0 implies uniqueness of an ipm for Pt, tb 0 andthat for any (discrete or continuous time) Markov semigroup ðPrÞ with ipm m,the function r 7! kPrðx; :Þ  mk is non-increasing.Example 1. The following example shows that the assumptions of Corollary 1do not imply the conclusion of Theorem 1. Equip E ¼ Zþ with the discrete s-algebra E and deﬁne the transition probabilities by p0;0 ¼ 1, pi; i1 ¼ 1=3 andpi; iþ1 ¼ 2=3 for ib 1. Then m ¼ d0 is the unique invariant probability measure,the assumptions of Corollary 1 hold but Pnði; :Þ does not converge to m for anyiA 0.Remark 3. Note that in the discrete case (i.e. E is ﬁnite or countably inﬁnite)the assumptions in Theorem 2 and in Corollary 1 are also necessary for therespective conclusion to hold. This is not true for Theorem 1 however as theexample E ¼ Zþ with p0;0 ¼ p0;1 ¼ 1=2, pi; i1 ¼ 2=3 and pi; iþ1 ¼ 1=3 for ib 1shows.3. Proofs of Theorem 1 and Theorem 2An auxiliary construction. Denote for N a N, p a ð0; 1ÞCN;p ¼ fðx; yÞ : kPNðx; Þ  PNðy; Þka 2ð1 pÞg:Note that CN;p a EnE by Lemma 1(i) below. Sincen 6? n , kn nk < 2;85a coupling approach to doob’s theoremthe assumption of Theorem 2 (and therefore the stronger assumption of Theorem1, as well) yield that there exist N a N, p a ð0; 1Þ such thatðmn mÞðCN;pÞ > 0:In the sequel we ﬁx these values N, p and write simply C instead of CN;p. In ad-dition we assume that N ¼ 1. Remark 2 shows that this is no loss of generality.An outline of the method. Our aim, in fact, is to prove the convergencekPnðx1; Þ  Pnðx2; Þk ! 0; n !lð5Þeither for all ðx1; x2Þ a E  E in the case considered in Theorem 1, or for mn m-a.a. ðx1; x2Þ a E  E in the case considered in Theorem 2. Once (5) is proved, therequired convergence (3) follows using the representation (1) and the triangleinequality. The following fact is well-known ([5], p. 14): for any two randomelements x1, x2, deﬁned on a same probability space ðW;F;PÞ, valued in ðE;EÞ,and such that LawðxiÞ ¼ ni, i ¼ 1; 2, one haskn1  n2ka 2Pðx1A x2Þ:ð6ÞHence for any sequence fZn ¼ ðZ1n ;Z2n Þ; n a Zþg such that the laws offZin; n a Zþg, i ¼ 1; 2 equal respectively Pxi , i ¼ 1; 2, one has a boundkPnðx1; Þ  Pnðx2; Þka 2PðZ1nAZ2n Þ:This is the essence of the famous coupling approach which dates back to W.Do¨blin [2]: to prove (5) one should construct a sequence Z which veriﬁes theabove assumption (such a sequence is usually called a coupling for X ) in sucha way thatPðZ1nAZ2n Þ ! 0; n !l:ð7ÞConstruction of the coupling. The sequence Z will be taken as a Markov chain onE  E with suitably constructed transition probability. The ﬁrst part of this con-struction is based on the fact that a proper choice of the pair ðx1; x2Þ may turninequality (6) into an identity. This fact, sometimes called the Coupling Lemma,is well known; the law of any such pair ðx1; x2Þ is called a maximal coupling. Werefer to [5] Section 1.4, where the construction of a maximal coupling based onthe splitting representation of a random variable is given. In our framework weuse essentially the same construction, but with modiﬁcations which are causedby the necessity (a) to deal with transition probabilities instead of measures, andtherefore to take care of measurability issues; (b) to manage properly the lawof the pair ‘‘outside of the diagonal’’. Namely, we have the following statement(the proof is given in the Appendix).Lemma 1 (The Coupling Lemma for transition probabilities). Let ðE;EÞ be acountably generated measurable space.86 a. kulik and m. scheutzowThen for any Markov kernel Pðx; dyÞ on ðE;EÞ there exists a Markov kernelQððx1; x2Þ; dy1dy2Þ on ðE  E;EnEÞ such that for every ðx1; x2Þ a E  E:(i) Qððx1; x2Þ;DÞ ¼ 1 ð1=2ÞkPðx1; Þ  Pðx2; Þk;(ii) the measure Qððx1; x2Þ; dy1dy2Þ restricted to ðE  EÞnD is absolutely continu-ous w.r.t. Pðx1; dy1ÞnPðx2; dy2Þ.Remark 4. We refer a reader to [6] for another version of the Coupling Lemmawhich takes into account measurability issues; the measurable space ðE;EÞtherein is assumed to be a Borel one.DenoteRððx1; x2Þ; dy1dy2Þ ¼ Pðx1; dy1ÞnPðx2; dy2Þ;which is just the transition probability of the Markov chain in E  E whosecomponents are independent and each of the components is a Markov chainwith the transition probability Pðx; dyÞ. Such a chain is usually called an indepen-dent coupling, and below we denote it by W ¼ fWn; n a Zþg:Finally, we deﬁne the transition probability for Z bySððx1; x2Þ; :Þ :¼ Qððx1; x2Þ; :Þ if ðx1; x2Þ a CRððx1; x2Þ; :Þ otherwise;with the set C deﬁned at the beginning of the proof. By construction, Z is a cou-pling for X , and our aim is to prove (7) with Z0 ¼ ðx1; x2Þ either for all ðx1; x2Þ inthe case of Theorem 1, or for mn m-a.a. ðx1; x2Þ in the case of Theorem 2.Proof of (7). Note that DHC, and for any point ðx; xÞ a DQððx; xÞ;DÞ ¼ 1:Hence, by the construction the following property holds: once Z hits D, all thesubsequent values of Z a.s. stay in D (‘‘once the components are coupled theystay coupled’’). ThereforePðZ1nAZ2n Þ ¼ PðT > nÞ; T ¼ inffm : Z1m ¼ Z2mg;with the usual convention inf j ¼l.Consider the sequence of stopping timest0 ¼ 0; tk ¼ inffn > tk1 : Zn a Cg; kb 1;and assume for a moment that we know thattk <l; kb 1ð8Þ87a coupling approach to doob’s theoremwith probability 1. Clearly, for any kb 1fT > tkg ¼[ln¼1ftk ¼ n;Z1 B D; . . . ;Zn B Dg a Ftk ;here fFng denotes the natural ﬁltration of the sequence Z. By the constructionof Z,PðZtkþ1 a D jFtkÞb p; T > tk;¼ 1; otherwise:Hence for the sequence pk ¼ PðTa tkÞ, kb 1 one haspkþ1b pð1 pkÞ þ pk; kb 1;and therefore pk ! 1, k !l, which together with (8) yields (7).Proof of (8): the Recurrence Lemma. We have reached the last and thecrucial step in the proof: we need to prove that the set C, which in a sense is‘‘favorable for the subsequent coupling attempt’’, is a.s. visited by Z inﬁnitelyoften. We use the following lemma, whose proof is given in the Appendix.Lemma 2 (The Recurrence Lemma). Assume that the Markov chain X satisﬁesthe condition of Theorem 2.Then for any B a E with mðBÞ > 0, for m-a.a. x a EPxðXn a B inﬁnitely oftenÞ ¼ 1:ð9ÞIf, in addition, the condition of Theorem 1 holds true, then (9) holds true forevery x a E.Now we can ﬁnish the whole proof; consider ﬁrst the case of Theorem 1. Theindependent couplingW veriﬁes the assumptions of Lemma 2 with E  E insteadof E and mn m instead of m. Because ðmn mÞðCÞ > 0, this yieldsPðWn a C inﬁnitely oftenÞ ¼ 1ð10Þfor all initial values W0 ¼ ðx1; x2Þ a E  E.Observe that up to t1 the law of Z coincides with the law of the independentcoupling W up to its ﬁrst visit to C, hence by (10)Pðt1 <lÞ ¼ 1for all initial values Z0 ¼ ðx1; x2Þ a E  E. Hence Z a.s. performs at least one‘‘coupling attempt’’. If this attempt is successful, i.e. Ta t1 þ 1, then a.s.t2 ¼ t1 þ 1; t3 ¼ t2 þ 1; . . . because DHC and, once the components of Zare coupled, they stay coupled. In that case (8) holds true. If ‘‘the ﬁrst coupling88 a. kulik and m. scheutzowattempt is not successful’’, the chain Z afterwards again performs as the inde-pendent coupling W up to the time moment t2. Applying Lemma 2 once againand the strong Markov property of Z, we getPðt2 <lÞ ¼ 1:Iterating this argument, we get (8) for all initial values Z0 ¼ ðx1; x2Þ a E  E,which completes the proof of Theorem 1.Let us proceed with Theorem 2; in that case it is convenient to prove (8) fora version of Z with LawðZ0Þ ¼ mn m. The argument, completely the same asabove, proves that t1 <l a.s., and if ‘‘the ﬁrst coupling attempt is successful’’then (8) holds true. Consider the law of Zt1þ1 conditioned by the event that‘‘the ﬁrst coupling attempt is not successful’’; that is, fZt1þ1 B Dg. By the choiceof the law of Z0 and the construction of the kernel Q, this law is absolutelycontinuous w.r.t. mn m. Using this and the strong Markov property of Z at thestopping time t1 þ 1, we apply Lemma 2 once again and getPðt2 <lÞ ¼ 1:Iterating this argument, we get (8) for Z with LawðZ0Þ ¼ mn m, which completesthe proof of Theorem 2. rA. Proofs of the auxiliary lemmasA.1. Proof of Lemma 1DenoteLðx1; x2; dyÞ ¼ ð1=2ÞðPðx1; dyÞ þ Pðx2; dyÞÞ:Then for every x1, x2 Piðxi; dyÞfLðx1; x2; dyÞ, i ¼ 1; 2. Let us show that respec-tive Radon-Nikodym derivatives can be chosen in a jointly measurable way; thatis, there exist measurable functions fi : E  E  E ! Rþ, i ¼ 1; 2 such thatPðxi;AÞ ¼ZAfiðx1; x2; yÞLðx1; x2; dyÞ; i ¼ 1; 2; x1; x2 a E; A a E:Let E0 be a countable algebra which generates E, then the countable classH, which consists of all functions representable in the form of a ﬁnite sumPk ck1Ak , fckgHQ, fAkgHE0, is dense in L1ðE; lÞ for any probability measurel on ðE;EÞ.Consider a Radon-Nikodym derivativer1x1;x2ðyÞ ¼Pðx1; dyÞLðx1; x2; dyÞ ;89a coupling approach to doob’s theoremthen for every e > 0 there exists h a H such thatsupA AE0ZAðr1x1;x2ðyÞ  hðyÞÞLðx1; x2; dyÞ <e2:Observe that this relation is equivalent tosupA AE0Pðx1;AÞ ZAhðyÞLðx1; x2; dyÞ<e2;ð11Þand yields ZEjrx1;x2ðyÞ  hðyÞjLðx1; x2; dyÞ < e:Fix some enumeration of the class H ¼ fhm;m a Ng, and denote for nb 1 bymðx1; x2; nÞ the minimal mb 1 such that (11) holds true for h ¼ hm withe ¼ 2n1. Then mð; nÞ : E  E ! N is measurable, and thereforef n1 ðx1; x2; yÞ ¼ hmðx1;x2;nÞðyÞis measurable as a function E  E  E ! R. When x1, x2 are ﬁxed, the sequencef f n1 ðx1; x2; yÞ; nb 1g converges to r1x1;x2ðyÞ for Lðx1; x2; Þ-a.a. y: this followsfrom the Borel-Cantelli lemma because, by the construction,k f n1 ðx1; x2; Þ  r1x1;x2kL1ðE;Lðx1;x2; ÞÞa 2n:Therefore the functionf1ðx1; x2; yÞ ¼ limn!l fn1 ðx1; x2; yÞ; if the limit exists0; otherwisegives the required measurable version of the Radon-Nikodym derivative forPðx1; dyÞ (the construction for Pðx2; dyÞ is the same).We ﬁnish the proof by repeating essentially the construction from [5] Section1.4, based on the splitting representation for probability laws. Writegðx1; x2; yÞ ¼ minð f1ðx1; x2; yÞ; f2ðx1; x2; yÞÞ;pðx1; x2Þ ¼ZEgðx1; x2; yÞLðx1; x2; dyÞ;Yðx1; x2; dyÞ ¼ gðx1; x2; yÞpðx1; x2Þ Lðx1; x2; dyÞwith the convention that (anything)=0 ¼ 1. Then we have representationsPðxi; dyÞ ¼ pðx1; x2ÞYðx1; x2; dyÞ þ ð1 pðx1; x2ÞÞSiðx1; x2; dyÞ; i ¼ 1; 2with probability kernels Y, S1, S2 and measurable p : E  E ! ½0; 1.90 a. kulik and m. scheutzowNote that the mapping E C x 7! ðx; xÞ a E  E is E EnE measurable, anddenote by Q1ððx1; x2Þ; dy1dy2Þ the image of Yðx1; x2; dyÞ under this mapping.Denote also by Q2ððx1; x2Þ; dy1dy2Þ the product of the measures Siðx1; x2; dyiÞ,i ¼ 1; 2. ThenQððx1; x2Þ; Þ ¼ pðx1; x2ÞQ1ððx1; x2Þ; Þ þ ð1 pðx1; x2ÞÞQ2ððx1; x2Þ; Þis the required kernel; observe that the assertion (ii) now holds true becauseQ2ððx1; x2Þ; Þ is chosen to be a product measure with the componentsSiðx1; x2; ÞfPðxi; Þ, i ¼ 1; 2. rA.2. Proof of Lemma 2DenotecðxÞ ¼ PxðXn a B inﬁnitely oftenÞ;and consider a stationary version of X with LawðX0Þ ¼ m. Then the sequencefcðXnÞg is stationary. But, in addition, this sequence is a Le´vy martingale: bythe Markov property of X , we have with probability 1cðXnÞ ¼ EXn1Xk AB i:o: ¼ E½1Xk AB i:o:;kbn jFn ¼ E½1Xk AB i:o:;kb0 jFn:Then with probability 1cðXnÞ ! 1Xk AB i:o:;kb0; n !l;and hence by stationarity of fcðXnÞg we have cðxÞ ¼ 0 or 1 for m-a.a. x a E.DenoteC0 ¼ fx : cðxÞ ¼ 0g; C1 ¼ fx : cðxÞ ¼ 1g;and observe that because cðxÞ a ½0; 1 in any case, by the martingale property offcðXnÞg one has for any nb 1Pnðx;C0Þ ¼ 1 for m-a:a: x a C0 and Pnðx;C1Þ ¼ 1 for m-a:a: x a C1:Because of the assumption of the lemma (or Theorem 2), it is impossible thatboth C0 and C1 have positive measure m. The identity mðC1Þ ¼ 0 contradictsBirkho¤ ’s ergodic theorem: with probability 1 we have1NXNn¼11BðXnÞ ! h; N !lwhere Eh ¼ mðBÞ > 0. Therefore mðC0Þ ¼ 0; which implies the required identitymðC1Þ ¼ 1.91a coupling approach to doob’s theoremIf, in addition, the condition of Theorem 1 holds true, then it follows fromwhat we have just proved that for any x a E there exists n ¼ nx such thatPnðx;C1Þ ¼ 1. By the Markov property of X and the deﬁnition of c, this impliescðxÞ ¼ 1. rReferences[1] G. Da Prato - J. Zabczyk, Ergodicity for Inﬁnite Dimensional Systems, CambridgeUniversity Press, Cambridge (1996).[2] W. Doeblin, Ele´ments d’une the´orie ge´ne´rale des chaıˆnes simples constantes de Marko¤,Ann. Sci. E.N.S., 57, 61–111, 1940.[3] J. Seidler, Ergodic behaviour of stochastic parabolic equations, Czechoslovak Math. J.,47(122), 277–316, 1997.[4] L. Stettner, Remarks on ergodic conditions for Markov processes on Polish spaces,Bull. Polish Acad. Sci. Math., 42, 103–114, 1994.[5] H. Thorisson, Coupling, Stationarity, and Regeneration, Springer, NY (2000).[6] A. Yu. Veretennikov, Coupling method for Markov chains under integral Doeblintype condition, Theory Stoch. Processes, 8 (3–4), 383–391, 2002.Received 23 August 2014,and in revised form 1 September 2014.A. KulikInstitute of MathematicsNAS of Ukraine, 3Tereshchenkivska str., 01601 Kyiv, Ukrainekulik.alex.m@gmail.comM. ScheutzowInstitut fu¨r MathematikMA 7-5, Technische Universita¨t BerlinStr. des 17. Juni 136, 10623 Berlin, Germanyms@math.tu-berlin.de92 a. kulik and m. scheutzow

A coupling approach to Doob's theorem

Abstract

Similar works

Full text

Available Versions

DepositOnce (Techn. Univ. Berlin)

CiteSeerX