Zanten, J.H. (Harry) van

English

CWI's Institutional Repository

©Board of the Foundation of the Scandinavian Journal of Statistics 2001. Published by Blackwell Publishers Ltd, 108 Cowley Road, Oxford OX4 lJF, UK and 350 Main Street, Malden, MA 02148, USA Vo! 28: 617-623, 2001 A Note on Consistent Estimation of Multivariate Parameters in Ergodic Diffusion Models J. H. VAN ZANTEN Centre for Mathematics and Computer Science (CW!) ABSTRACT. Certain aspects of maximum likelihood estimation for ergodic diffusions are studied via recently developed empirical process theory for martingales. This approach enables us to remove some undesirable regularity conditions that usually appear in the statistical literature on ergodic diffusions. In particular, dimension dependent conditions for the existence of a continuous likelihood and for consistency of the maximum likelihood estimator turn out to be unnecessary. Key words: consistency, continuous likelihood, empirical process methods, ergodic diffusions, maximum likelihood estimation, multivariate parameter. 1. Introduction We consider the family of stochastic differential equations (I) where f) runs through a compact parameter space 8 ~ !Rd. For every e E (9 we suppose that ( 1) has a unique stationary, weak solution taking values in a (possibly unbounded) interval (l, r) ~ IR. In particular, the state space(/, r) is assumed to be independent of the parameter e. We suppose that a> 0 on (l, r). For every e Ee we denote by Pe the law that is generated by the weak solution on the space C[O, oo) of continuous functions on [O, oo ). We write µe for the marginal law of X 1 under Pe. By the stationary assumption the distribution µ 6 is independent of t. In particular, the law of the initial random variable Xo is equal to µ0 . We suppose furthermore that under Pe, the process X has the ergodic property with invariant measure µ 0, i.e. for every f E L1(µe) ~ Lf(Xs) ds ---+ J f dµe Pe a.s. (2) as t---+ oo. See e.g. Gihman & Skorohod (1972) or Skorokhod (1989) for conditions on the functions be and a under which the stochastic differential (1) admits such a stationary, ergodic solution. Throughout this note, the parameter fJo E 8 will play the role of the "true parameter". Denote by Pe,r the restriction of the measure Po to the space C[O, t] of continuous functions on [O, t]. It is well-known that under certain conditions, the measures Pe,r are absolutely continuous with respect to a dominating measure that is independent of e. Up to a factor involving the initial random variable X 0 , the likelihood process is given by (J1 be(X,) 1 J' b~(Xs) ) L1(8) =exp o a2(X,) d.Xs - 2 o a2(Xs) ds (3) (see for instance Liptser & Shiryayev ( 1977) ). The maximum likelihood estimator {j 1 is the maximizer of the random map er--> L1(8), provided it exists. For the definition of the estimator 618 J H van Zanten Scand J Statist 28 {j 1 it is not relevant that L1( B) is in fact a likelihood ratio. To ensure that L1( 8) is well-defined under Pe0 it is only needed to assume that J' b~(Xs) 0 a 2(Xs) ds < oo Pe0 a.s. (4) In this paper, we will never need the fact that L1(8) is a likelihood. Instead of conditions for absolute continuity, we shall therefore use condition (4) in the statement of the theorems below. Expression (3) can be used to calculate the estimator 81 from the observations (Xs)s,.; 1• To study the (asymptotic) properties of the estimator, another expression is more useful. Take logarithms in (3) and use (1) to see that under Pea, maximizing e ,__. L1(8) is equivalent to maximizing 8 f-+ !1(8), where li(B) = J1 be(Xs) - be0 (Xs) dWs _ ~1 1 (be(Xs); be0 (Xs))2 ds 0 a(Xs) 2 o a (Xs) (5) and W is a standard Brownian motion under Pea· So the maximum likelihood estimator fJ 1 is also the maximizer of the log-likelihood e f-+ !1(8), provided it exists. In this paper we consider two aspects of maximum likelihod estimation in the model ( l ). First we question the existence of a continuous version of the log-likelihood (5), viewed as a function in e. If such a continuous version exists, the maximizer 81 of that version also exists, by compactness of the parameter space. We then investigate the consistency of the maximum likelihood estimator 81• The problem of maximum likelihood estimation in non-linear ergodic diffusion models has been studied by several authors, see Kutoyants ( 1977), Umska (1979), Prakasa Rao & Rubin (1981 ), Basu (1983), Kutoyants (1984 ), Yoshida (1990) and Prakasa Rao (1999), to mention just a few. We reconsider this classical subject in order to apply new empirical process results that were developed recently by entropy methods. This will allow us to analyse the model (1) more effectively than in the past and to improve several known results. We use empirical process theory in a similar fashion as for instance Van der Vaart & Wellner (1996) in the case of i.i.d. observations, Van de Geer (1995) for point processes and Nishiyama (1999) for small noise diffusion models. In section 2 we consider the continuity of the log-likelihood (5). The stochastic integral in (5) is only defined up to indistinguishability. Therefore, since the parameter space <9 is uncountable, different versions of the log-likelihood may not even be almost surely equal. In particular, we see that a statement like "the log-likelihood is continuous" does not make sense. The correct way to formulate the problem is to ask whether or not there exists a continuous version of the random map 8 ,__. !,(&), i.e. a random map !1 that is continuous and such that for every 8 E f9 the relation (5) holds almost surely. In the literature, two main approaches to this problem are pursued. In L:inska (1979) for instance, Ito's formula is used to see that with fi =Ix be(y) - beo(Y) d e a2(y) Y and b f' I 21 ,, 1 (be - be0)2 ge= eo e+2a e+2 a2 (a prime denotes differentiation with respect to x) the random map e >-+ fe(X1) - fe(Xo) - t ge(Xs) ds is a version of the log-likelihood. It can then simply be assumed that this version is continuous © Board of the Foundation of the Scandinavian Journal of Statistics 2001. Scand J Statist 28 Estimation in ergodic diffusion models 619 in e. A drawback here is that one needs to assume that the functions be and a are differentiable with respect to x in order to have the continuity in e. Other authors use Kolmogorov's continuity theorem (see for instance Ibragimov & Has'minskii (1981), app. 1) to conclude that there exists a continuous log-likelihood. The drawback of this method is that the resulting conditions on the log-likelihood depend on the dimension d of the parameter space (see e.g. Kutoyants (1984)). In connection with smoothness in e, both assumptions of smoothness in x and dimension dependent conditions are undesirable and it turns out that we can do without them. Instead, we prove in section 2 that l1 admits a continuous version if the pointwise Holder condition (6) is satisfied. In section 3 we discuss the consistency of the maximum likelihood estimator, which is the (or rather some) point where the continuous version of the likelihood attains its maximum. In the case of a multivariate parameter, consistency is usually derived under certain smoothness conditions in 8 that depend on the dimension d of the parameter space (see Basu, 1983; Kutoyants, 1984; Prakasa Rao, 1999). This is typically done by using conventional weak convergence theory as can be found for instance in Ibragimov & Has 'minskii (1981, app. 1 ). This procedure turns out to be too crude however and the dependence on the dimension that it causes turns out to be superfluous. Using finer methods, we show in section 3 that it suffices to require the Holder condition ( 6) again, as well as a natural indentifiability assumption. 2. Existence of a continuous likelihood In this section we prove that there exists a continuous version of the random map (5) if the following Holder condition is satisfied: There exist a number a> 0 and a function b E L2(Jte0 ) such that I be(x) - bl/!(x) I ~ 18 - 'lj>lab(x) a(x) for all 8, 'lj> E <9 and x E iR. Theorem 1 (6) Suppose that ( 4) holds for all 8 E <9 and t ;;: 0. If the Holder condition (6) is satisfied, then the random map /1 given by (5) admits a continuous version. Proof Let t ;;: O be fixed. It follows readily from the Holder condition that the Lebesgue integral in (5) is continuous in 8, so we only have to show that there exists a continous version of the random map e >-+ Z(8) = Lee ~ beo) (Xs) dWs. Note that by Fubini 's theorem and the stationarity of X it holds that It (be_ b ·) 2 II be - biµ 11 2 Ee0(Z(8) - Z('!j>))2 = Ee0 --11 (Xs) ds = t -a - ? • 0 O' L-(µ!Jo) Using assumption (6) we thus find that Ee0(Z(8) - Z('!j>))2 ~ ti8 - 'lj>l 2allbll~2(µ80 ) (7) for every 8, 1j> E <9. In particular, the random map Z is continuous in probability, i.e. if 8n -+ 8, then Z( 8 n) -+ Z( 8) in probability. © Board of the Foundation of the Scandinavian Journal of Statistics 2001. 620 J H. van Zanten Scand J Statist 28 Now let <EJ* be a countable, dense subset of e. We want to show that with probability I, the random map z is uniformly continuous on e*. To that end, we view Z as a collection of endpoints of continuous local martingales that are indexed by a parameter (cf. (7)). For such random maps, Nishiyama (1998) gives sufficient conditions for uniform continuity. In the present case th. 2.4.3 of Nishiyama (1998) states that if dis a metric one such that II Zll := ~~~ d(O~ 1/J) J~ ee: b90 ) \x,) ds < oo, Pea a.s. (8) and J~ Jlog N(s, e, d) de< oo, (9) then with probability 1 the random map Z is uniformly d-continuous on the countable subset e* (here N(e, e, d) denotes the minimal number of balls of d-radius E needed to cover B).We apply this result with the metric d given by d(O, 1/J) = IO -1/Jl"A1. Then by the boundedness of e and condition ( 6) there exists a constant C > 0 such that the quantity II ZI I in (8) satisfies II Zll .;;;; C J>2(X,) ds. (10) Since b E L2{µ1Jo) it holds in particular that II Zll < oo, P90-almost surely. As for (9), use the well-known fact that this condition is always satisfied if e is a bounded subset of Euclidean space and d is the Euclidean distance to some power 0 < f3 .;;;; 1. So by the cited theorem of Nishiyama (1998) the random map Z is indeed uniformly continuous on E>* (with respect to the metric d and therefore also with respect to the Euclidean metric). To conclude the proof, define the new random map Z' on e by taking it equal to Z on E>* and then extending it continuously to all of e. Then Z' is continuous by construction and the continuity in probability of Z implies that Z' is a version of Z. 3. Consistency of the MLE Let us assume condition (6) and denote by 11 the continuous version of the log-likelihood (5) (see theorem 1). Since the parameter space e is compact and 11 is continuous, the maximizer of 11 (the maximum likelihood estimator) exists. In fact, it is already enough for the consistency result that we are going to prove in this section to require the estimator {} 1 satisfies l ' 1 -11(01) ;;. sup -11(0) - op(l). t !lee t ( 11) Of course, we cannot expect to estimate the true parameter 00 consistently if we cannot identify it properly. We will assure the identifiability of the parameter by demanding that the function 0 II be - beo II i- a Ll(µei,) (12) only has a zero at 00. Theorem 2 Suppose that (4) holds for all 0 E 61 and t;;. 0. If the Holder condition (6) is satisfied, the estimator Bi satisfies (11) and the function (12) only has a zero at 00, then © Board of the Foundation of the Scandinavian Journal of Statistics 2001. Scand J Statist 28 Estimation in ergodic diffusion models 621 as t-+ oo. Proof The law of large numbers for martingales and the ergodic property (2) tell us that with increasing t, the random map Ii/ t approaches the limit map 1(8) = - ! II be - beo 112 . 2 a [2(µyo) We might therefore expect that for large t the estimator 01 is close to the maximizer of I, which is the true parameter 80. To make this reasoning precise we apply corol. 3 .2.3 of Van der Vaart & Wellner (1996). Using the Holder condition ( 6) and the identifiability assumption we see that I is a continuous function that has a unique maximum at 80 • The cited result of Van der Vaart & Wellner ( 1996) therefore implies that to prove the consistency, it suffices to show that sup I! /1(8) - /(8)1-+ O 8EfJ f (13) in (outer) probability as t-+ oo. Both 11 and l are continuous in(} so we have established (13) as soon as we have proved that for the countable, dense subset e* i;;; e it holds that sup I !JI be(X.,.) - beo(Xs) dWsl ~ 0 eEe* t o a(Xs) (14) and sup I! J1 (be(Xs); beo(Xs))2 ds - II be - beo 112 I~ 0. eEe* t o a (Xs) a L2(µeo) (15) To prove (14) we define for every 8 E $and t > 0 the process M 8•1 by Me,1 = J_ Js1 be(Xu) - beo(Xu) dW s ~ 0. s Vt o a(X,,) u, Observe that M 8•1 is a martingale with quadratic variation (M8•1) given by (Me. 1ls = ! Jst (be(Xu); be0(Xu)) 2 du. t 0 a (X,,) Our aim is first to show that as t -+ oo, the random maps (} >--+ Mf· 1 (16) have a weak limit in the space /00(8*) of bounded functions on e* (see sect. 1.5 of Van der Vaart & Wellner (1996) for basic facts about weak convergence in such spaces). By the ergodic property (2) and the central limit theorem for martingales (see e.g. Kutoyants, 1984, p. 78) or more recently Van Zanten (2000a)) we have finite dimensional convergence of the random maps (16) to a zero-mean Gaussian random map. To see that this may be strengthened to weak convergence in/""($*) we apply th. 3.4.2 of Nishiyama (1998). That result implies that if dis a metric on e such that j(M8•1 - Af'i',1)1 llM1ll :=sup = Op(l) (17) e-tip d(8, 1/J) and (9) holds, then the weak convergence takes place in /00 ( e* ). As in the proof of theorem I ,-g Board of the Foundation of the Scandinavian Journal of Statistics 200 L 622 J H van Zanten Scand J Statist 28 we take d(&, 1/J) = I& -1/JlaAl. The entropy condition (9) is then satisfied again. By the boundedness of e and condition (6) there exists a constant C > 0 such that So the ergodic property (2) implies that (17) holds. We may thus conclude that the random maps (16) have a weak limit in loc(e*). By Slutsky's lemma (see Van der Vaart & Wellner (1996, p. 32) this implies that the random maps e H Mf·1 I Jt converge weakly to 0 in 100 (@*), which is equivalent to (14). It remains to show that (15) holds. Observe that (15) can be viewed as a uniform version of the ergodic property (2). With .'F = {fe: e Ee*}, it reads as ( )2 be - be where f e = --0 - 0 sup I~ J1f(Xs) ds - Ji dµe0 I ~ 0. fE.V t 0 (18) As was observed in the paper Van Zanten (2000b), such a uniform ergodic theorem can be proved by a straightforward modification of the arguments that were used by Dudley (1984) to prove the "classical" uniform law of large numbers for i.i.d. random variables. Sufficient condition for (18) are then formulated in terms of the bracketing numbers of the class Y. Recall that given two functions land u, the bracket[/, u] is the set of all functions /with l ~ f ~ u. If II ·II is a norm on !7, a bracket [l, u] is called an i:-bracket if llu - lll <E. The bracketing number N[ 1(e, Y, 11·11) is the minimum number of i:-brackets needed to cover Y. Lem. 3.4 of Van Zanten (2000b) implies that (18) holds if N[ 1(E, Y, ll·llr1cµe0 )) < oo (19) for every e > 0. To verify ( 19) we note that Y is a class of functions that is Lipschitz in a parameter. Indeed, by ( 6) and the boundedness of e, there exists a constant C > 0 such that lfe(x) - f'i'(x)I ~ Cd(&, 1/J)b2(x) for all 8, 1/J E 8 and x E llt Therefore th. 2.7.11 of Van der Vaart & Wellner (1996) may be applied to conclude that N[ 1(2C, llblli2cµe0),..97, II ·llr1(µe,) ;;;::; N(e, B, d) < oo for all e > 0. Thus (15) holds true and the proof is completed. References Basu, A. (1983). Asymptotic theory of estimation in nonlinear stochastic differential equations for the multiparameter case. Sankhyti Ser. A 45, 56-65. Dudley, R. M. (1984). A coutse on empirical processes. In Ecole d'ete de probabilites de Saint-Flour XII-1982, 1-142. Springer-Verlag, Berlin. Gihrnan, I. I. & Skorohod, A. V (1972). Stochastic differential equations. Springer-Verlag, New York. Ibragimov, I. A. & Has'minskii, R. Z. (1981). Statistical estimation, asymptotic theory. Springer-Verlag, New York. Kutoyants, Y. A. (1977). Estimation of the trend parameter of a diffusion process in the smooth case. Theor. Probab. Appl. 22, 399-405. Kutoyants, Y. A. ( 1984 ). Parameter estimation for stochastic processes. Helderrnann Ver lag, Berlin. © Board of the Foundation of the Scandinavian Journal of Statistics 2001. Scand J Statist 28 Estimation in ergodic diffusion models 623 Lanska, V. ( 1979). Minimum contrast estimation in diffusion processes. J. Appl. Probab. 16, 65-75. Liptser, R. S. & Shiryayev, A N. ( 1977). Statistics of random processes [ Springer-Verlag, New York. Nishiyama, Y. (1998). Entropy methods for martingales. PhD thesis, Universiteit Utrecht. Published in 2000 as CWI Tract 128, CWI Amsterdam. Nishiyama, Y. (1999). A maximal inequality for continuous martingales and M-estimation in a Gaussian white noise model. Ann. Statist. 27, 675-696. Prakasa Rao, B. L. S. (1999). Statistical inference for diffusion type processes. Edward Arnold, London. Prakasa Rao, B. L. S. & Rubin, H. (1981). Asymptotic theory of estimation in nonlinear stochastic differential equations. Sankhyii Ser. A 43, 170-189. Skorokhod, A. V. (1989). Asymptotic methods in the theory of stochastic differential equations. American Mathematical Society, Providence, RL Van de Geer, S. (1995). Exponential inequality for martingales, with application to maximum likelihood estimation for counting processes. Ann. Statist. 23, 1779-1801. Van der Vaart, A. W. & Wellner, J. A. (1996). Weak convergence and empirical processes with applications to statistics. Springer-Verlag, New York. Van Zanten, J. H. (2000a). A multivariate central limit theorem for continuous local martingales. Statist. Probab. Lett. 50, 229-235. Van Zanten, J. H. (2000b ). On the uniform convergence of the empirical density of an ergodic diffusion. Statist. Inference Stochastic Process. to appear. Yoshida, N. (1990). Asymptotic behavior of M-estimator and related random field for diffusion process. Ann. Inst. Statist. Math. 42, 221-251. Received March 2000, in final form October 2000 J. H. van Zanten, CWI, P.O. Box 94079, 1090 GB Amsterdam, The Netherlands. © Board of the Foundation of the Scandinavian Journal of Statistics 2001. 

A note on consistent estimation of multivariate parameters in ergodic diffusion models

Abstract

Similar works

Full text

Available Versions

CWI's Institutional Repository