We propose an estimator for parameters of nonlinear mixed effects model, obtained by maximization of a simulated pseudo likelihood. This simulated criterion is constructed from the likelihood of a Gaussian model whose means and variances are given by Monte Carlo approximations of means and variances of the true model. If the number of experimental units and the sample size of Monte Carlo simulations are respectively denoted by N and K, we obtained the strong consistency and asymptotic normality of the estimator when the ratio NJ/2 /K tends to zero

Núñez, Olivier

Name not available

Asymptotic properties for a simulated pseudo maximum likelihood estimator

Universidad Carlos III de Madrid e-Archivo

Working Paper 98-73 Statistics and Econometrics Series 32 September 1998 Departamento de Estadistica y Econometria Universidad Carlos III de Madrid Calle Madrid, 126 28903 Getafe (Spain) Fax 34 - 91- 624.9849 ASYMPTOTIC PROPERTIES FOR A SIMULATED PSEUDO MAXIMUM LIKELIHOOD ESTIMATOR Olivier Nufiez* Abstract We propose an estimator for parameters of nonlinear mixed effects model, obtained by maximization of a simulated pseudo likelihood. This simulated criterion is constructed from the likelihood of a Gaussian model whose means and variances are given by Monte Carlo approximations of means and variances of the true model. If the number of experimental units and the sample size of Monte Carlo simulations are respectively denoted by N and K, we obtained the strong consistency and asymptotic normality of the estimator when the ratio NJ/2 /K tends to zero. KeyWords Nonlinear mixed-effects models; Simulation estimators; Asymptotic normality; Consistency. "Department of Statistics y Econometrics, Universidad Carlos III de Madrid, e-mail: nuneZ@est-econ.uc3m.es. Asymptotic Properties for a Simulated Pseudo Maximum Likelihood Estimator Olivier Nunez SUMMARY We propose an estimator for parameters of nonlinear mixed effects model, obtained by maximization of a simulated pseudo likelihood. This simulated criterion is constructed from the likelihood of a Gaussian model whose means and variances are given by Monte Carlo approximations of means and variances of the true model. If the number of experi-mental units and the sample size of Monte Carlo simulations are respectively denoted by Nand K, we obtained the strong consistency and asymptotic normality of the estimator when the ratio N~ / K tends to zero. Some key words: Nonlinear mixed-effects models; Simulation estimators; Asymptotic normality; Consistency. 1. INTRODUCTION We consider the following mixed effects model (1) and where i = 1...N, N is the sample size. The vector Yi is a vector of ni observations made on the ith experimental unit. The random vectors (ci)i=1..N and (Ei )i=1..N are unobserved and are assumed mutually independent. The matrices A and Bi are full rank matrices of deterministic explanatory variables. The vectorial functions 1Ji(.) and the matrices Ai(.) depend non-linearly on <Pi and are assumed to be sufficiently regular. The vector a is a parameter vector of size r and D is a q x q covariance matrix. We are interested in the estimation of the parameter vector () = (a, vec(D) , a2), where vec(D) denotes the set of parameters of the matrix D. 1 Several different methods have been proposed for this estimation in the homoscedastic framework (i.e.: Ai depends on i only through its dimension). The methods suggested by Sheiner-Beal (1992) or Linstrom-Bates (1990) are based on a Laplacian approximation of the marginal log-likelihood of observations. These methods involve a linearization of the conditional model (1) with respect to the vector Ei about 0 or about a posterior mode, before integration. The main problem which occurs when using these methods is related to asymptotic prop-erties of estimators (when N -+ 00 and SUPi2:1 ni < 00) which are either not known or poor (for the non consistency see Ramos and Pantula (1995)). These properties depend on the control, with respect to (), of the remainder of the mar-ginallog-likelihood expansion. Some authors (Vonesh 1996) imply that choosing "a good order" of expansion may improve these properties. But it seems difficult to provide suitable assumptions about the model curvature to control the remaining terms. Nev-ertheless, when the number of observations per experimental unit tends uniformly to infinity (infi > 1 ni -+ 00), Vonesh (1996) gives a consistency result of estimators. But in this case, the usual assumption of independence between observations made on the same experimental unit (the Ai's assumed diagonal) is no more tenable. As a consequence, it is difficult to make an inference (which is often the primary aim of the experiment) due to the lack of at least the asymptotic distribution of the estimator. In this paper, we propose a simulated method of pseudo maximum likelihood, for which we give asymptotic properties. In section 2, the Simulated Pseudo Maximum Likelihood (SPML) method is presented. The main results are given in section 3, in which the strong consistency and asymptotic normality of the SPML estimator are demonstrated. 2. SIMULATED PSEUDO MAXIMUM LIKELIHOOD (SPML) ApPROACH The estimation of the parameter vector () = (a, vec(D), 0-2 ) with the SPML method, consists of constructing an objective function derived from a family of probability densities parametrized by the first moments of the observations. To this end, let us define for all i ~ 1, J-Li(()) - lEe (ri) - lEe [7Ji(q>i)] , where J-Li(()) depends on () only through (a, vec(D)), and Vi(()) - vare (ri) - vare [7Ji(~i)] + 0-2lEe [N(q>i)] . These two first moments have generally no explicit form, and are respectively approxi-mated using the following Monte-Carlo sums : 1 ~ [ k _ ] [ k _ ]' (]"2 ~ 2 k Vi,K(B) = K -1 ~ 7Ji(q,i) - J-Li,K(B) 7Ji(q,i) - J-Li,K(B) + K ~Ai(q,i)' k=l k=l where q,f = Aa + BiD! Ef, and Ef are drawn independently from N(O, Iq). The objective function is constructed by considering that the Yi's are respectively drawn from a gaussian distribution Nni (J-Li (0), Vi (B)). This approach leads to consideration of the following objective function: where The two first simulated moments Pi,K{B) and Vi,K{O) are used to evaluate each Ci{B). Thus, the proposed simulated objective function is : where A similar idea was used by Gourieroux and Monfort (1991). These authors gave con-sistency results in the independent and identically distributed framework. This result cannot be extended readily to the non identically distributed case. The estimator of B which minimizes the criterion C~(B) will be denoted by O~. In this paper, the word asymptotic refers to the sample size N tending to infinity, and the numbers of observations per experimental unit being uniformly bounded (Le. SUPi~l ni < 00). 3. ASYMPTOTIC PROPERTIES Let us consider the following assumptions in order to ensure the strong consistency of the AK estimator B N' (Al) (A2) (A3) (A4) (A5) The true value of the parameter vector denoted by ()* = (a*, vec(D*), (72*) is an interior point of a compact metric space e c IRP (where p is the dimension of ()*, p = r + q( q + 1) /2 + 1 ). The size of observed vectors Yi (i 2: 1) are uniformly bounded : supni < 00. i~l For all i 2: 1, and for all () in e, the four first moments of er «()) are finite uniformly in K: supIEo(e{«())4) < 00. K~l For all i 2: 1, the functions rJi(.) and Ai(.) are almost surely twice continuously differentiable and square integrable. the vector ()* is second order identifiable : Vi 2: 1 Assumption (AI) is not restrictive because we can generally include the parameter vector in a compact set. This assumption is one of the main argument in our framework to show results about the uniform convergence in e of the simulated criterion, this kind of ~K convergence being very useful for establishing the strong consistency of () N (see Andrews (1989)). On the other hand, assumption (A3) can be quite restrictive because it requires the existence of the eight first moments of observations. We start by giving a technical lemma which ensures that when Nand K are sufficiently large, the simulated criterion CfS«()) is close to the expectation of CN «()). LEMMA. Under assumptions (Al-A4), we have almost surely clS«()) - EO*(CN «())) I 0, uniformly on e N-+oo K-+oo This lemma shows the ability of C/S (0) to locate ( at least for large Nand K) the parameter vector 0*, as can be seen by the study of IEO*(CN(O)). N IEo* (CN(O)) = ~ LIEo* (IIYi - JLi(O)II~-l(O) + In IVi(O)I) i=l is equal (up to an additive constant) to where Pt! is the probability measure of the vector Y = (Y1 , Y2 , .. , YN) and CPN(Y; 0) is the probability density of gaussian distribution N (g)Nni (JLi(O), Vi(O)) . i=l But, this last expression is also equal to because In CPN(Y; 0) is a quadratic function of the vector Y, and the two first moments of Pt! and CPN(Y; 0*) coincide. Therefore, we deduce from Jensen's inequality that (2) and in view of assumption (A5), the equality holds if and only if 0 = 0*. We conclude this discussion with the following proposition related to the strong consis-~K tency of ON. PROPOSITION 1. Under assumptions (A1-AS) , the sequence (O~) converges almost K,N surely, when Nand K tend to infinity, to the parameter vector 0* . Proof. Let us consider the quantity and B an open ball of e centered in 0*. Then, from (2) there exist c > 0 such that the function 0 ~ QN(O,O*) is undervalued by c on e \ B. The demonstration is based on the following inclusion {o~ tt B} C {~% c/S (0) ~ cf,S (0*) } . (3) (A6) On the other hand, in view of the uniform convergence showed in the lemma, we have almost surely and [ inf cf,5 ((I) - cf,5 (0*)]- inf QN(O, 0*) e\B e\B inf QN(O, 0*) > c. e\B Therefore, from (3) and (4), we have P (limsup {e~ !=.B}) 5: P (limsup {inf cf,5 (0) - cf,5 (0*) ::; o}) = O. N-+oo N-+oo e\B K-+oo K-+oo o AK We can now study the asymptotic normality of the estimator ON' (4) The convergence in distribution of this estimator requires the additional following as-sumption. Let us consider the vector "V oM{ (0*) = "V oC{ (0*) -JEoo ("V oC{ (0*)). There exists 8 > 0 such that, for N sufficiently large uniformly in K. The operator "V 0 denotes the gradient with respect to O. This condition is the so called Ljapunov's condition, which is generally easier to check than Lindeberg's in order to establish a central limit theorem. It is only required in the heteroskedastic case. This condition guarantees that the variances of individual terms "VoCr(O) are small as compared to their sum (see for details Feller (1971)). PROPOSITION 2. Let us denote 1(0*) = lim lEoo (HOCN(O*)) , where HOCN(O*) is the N-+oo N hessianofthefunctionCN(O*), with respect to 0, andr(O*) = lim NI ~varoo ("VoCr(O*). N-+oo ~ K-+oo i=l If the assumptions (AI-A6) hold, then when Nand K tend to infinity, with N~/K --+ 0, we have Remark. By construction the hessian matrix lEo• (H/JN ((}*)) is (up to a constant) the Fisher's information matrix of the gaussian pseudo-model N (i!)Nni (Jli((}) , Vi((})) . i=l After further calculus, we derive that the (l, m )th element of the (p + 1) x (p + 1) matrix lEo• (HOCN ((}*)) is lEo• (aB~;Bm CN ((}*)) - ~ t. [2 { a~, fJ., (e') r -.;-1 (e') a:m fJ.,( e') + tr ( -.;-1 (e') :0, V; (e') -.;-1 (e') a:m V; (e')) ]. Finally, the expression of the variances varo· (\7 oCi (B*)) (i = l..N), obtained after tedious calculus, is presented in the appendix. Notice that ApPENDIX Proof of the Lemma sup IC~(fJ) - IEo' (CN(O)) I o < sup IC~ (0) - IEO' (C~ (tJ)) I + sup IIEo· (C~ (0) - CN(O)) I o 0 then, it remains to demonstrate that the two terms of the right hand side converge to zero when N and K tend to infinity. Let us start by showing that For convenience, we adopt the following notations, the components of Pi K(O) and the elements of , matrix Vi,K(O) are stored in the vector ~i,K(O). Then, one can write - 1 ~ k ~i,K(O) = K ~~i (0), k=l where the vectors (~f(O))k=1..K are uncorrelated and identically distributed. The expectation IE~~ (0) and the variance var( ~~ (0)) are respectively denoted by mi (0) and Wi (0). Finally, let us introduce the function fi(~) = IlYi - J..l1I~-1 + In lVI, where ~ denotes the components of I-L and V. According to the previous notations, we have for a given 7" > 0, IEo'IC{(O) - Ci(O)1 = IEo'lfi(~i,K(O)) - fi(mi(O))1 - IEo' (lfi(~i,K(O)) - fi(mi(O))lll{lI~i.K(O)-mi(O)II2:T}) +IEo' (lfi(~i'K(O)) - fi(mi(O))lll{lI~i.K(O)-mi(O)II<T}: By continuity of fi' the second term of the last expression is arbitrary small for 7" sufficiently small, uniformly on e. Using the Cauchy-Schwarz's inequality, we see that the first term is less than and by Chebishev's inequality, the previous expression is bounded above by IEJ. (Ji(~i,K(O)) - fi(mi(O)))2 ~IEJ. (lI~i,K(O) - mi(O)1I2) - IEt. (C{(O) - Ci(O))2 ~ trt (Hti(O)). yK7" Now, tr (Wi(O)) is uniformly bounded on e and this quantity is obviously an increasing function of ni, but in view of (A2) we obtain that supsuptr(Wi(O)) < 00. o i Finally, uniformly in i the expectation is from the same arguments used for tr (Wi (0)), uniformly bounded on e. In addition, this expectation is from (A3) uniformly bounded for K 2: 1. Thus, we can conclude that uniformly on e. It remains to show that almost surely, Let's denote, for all 0 E 8, with M{(O) = C[«O) -1Eo· (C[«O)). So, Mff(O) is defined as a sum of independent and centered variables. In order to establish that if (A1-A4) hold then almost surely -K sup sup MN (0) N ..... d:, 0, K;:::l 0 we proceed by contradiction . If the previous convergence failed, there would exist a strictly increasing sequence (Nr )r;:::l which would tend to infinity, and two sequences (Or)r and (Kr)r both associated with (Nr)r, such that (5) From Borel-Canteli's lemma and Chebyshev's inequality, (5) cannot hold if there exist 8 > 0 such that 00 L1Eo./Mff;(Or)/o < 00, (6) r=l Now, notice that 00 r=l and, and the expressions in brackets are bounded in view of assumptions (AI-A4). So, we have shown that (6) holds for 8 = 4, which contradicts (5). This accomplishes the proof of the lemma. o Proof of the Proposition 2 This demonstration proceeds in three steps . Step 1. We demonstrate that the normalized sum of variances N ~ 2: varo· (\7 oCf (B*)) i=l converge when N tends to infinity and uniformly in K to a limit fK(B*). Thus, ~fK(B*) is an asymptotic equivalent of varo· (\7oC~(B*)), when N ---+ 00. Furthermore, we show that for K sufficiently large Step 2. We establish that for Nand K sufficiently large Nhl,C'j,(O') ~ N" (0 (~) , rK(O'») . Step 3. We deduce from the uniform convergence obtained in lemma and in view of differentiability assumption (A4) that uniformly on 8, -K -HOCN(B) - Eo·HoCN(B) N 10, almost surely. ~oo (7) K~oo AK Finally, we establish the asymptotic normality of ()N' by considering the following Taylor expansion with Lagrange remainder. We have, for all 1 :S l :S p, ° = N~~CfS(B~) a()l 1 a -K 1 ~ AK t a -K ( AK ) N'2 a()l CN (()*) + N'2 ~(()N,rn - ():n) lo a()la()rn CN ()* + S(()N - ()*) ds. Now from (7) and proposition 1, we have for alII :S l, m :S p, ------>. 0, almost surely. N-oo K-oo Therefore from the previous steps we conclude that when Nand K tend to infinity, with N~ / K - 0, N~ (B~ - ()*) rv Np (0, I- 1(()*)r(()*)I- 1(()*)) . Proof of step 1. Using (A3) and (A4), we note that SUPi varo· (V'oCr(()*)) is bounded uniformly in K. N Since L varo* (V' oCr (()*)) = O(N) , the normalized sum i=l N ~ Lvaro* (V'oCr(()*)) i=l converge when N tends to infinity, uniformly in K, and let us denote its limit by N r K(()*) = lim NI "'varo* (V'oCf(()*)). N--+oo ~ i=l In an other way, using notations from the proof of the lemma, we have (when K tends to infinity) IEo' (V' oCfS (()*) - V' oCN ( ()*)) N - ~ LIEo· [\70fi(~i,K(()*)) - V'ofi(mi(()*))] i=l N - ~ L (\7{;\7ofi(mi(()*))) IE (~i,K(()*) - mi(()*)) + 0 (lI~i,K(()*) - mi(()*) 112) i=l N - ~ t:; ~o (tr(W;(O'))) ~ ~O (tr(W;(O'))) ~ 0 G) . In view of Jensen's inequality (2) given in section 3, IEo' (V' oCN ( ()*)) = 0. Therefore, for K sufficiently large, Proof of step 2. Let us define the normalized sum Thus from (A3) and (A4), the vector \l M~(O) is a sum of independent and centered vectors, which are square integrable uniformly in K. Thus we can now, under assumption (A6), establish asymptotic normality (when N tends to infinity) of \loM~(O). We obtain uniformly in K. Finally, we deduce from step 1, that for Nand K sufficiently large o Expression of the variances varo> (\loCi(O*)) The (l, m)th element of this p x p matrix is equal to lEo> [!l Ci(O*) a~m Ci(e*)] - 4 [:0,1',(0') r V;-'(9') a~m 1',(0') + lEo> ( tr (D~ (O*)a2 A~ (<Pi)) 1177i (<Pi) - J.li (£1*) 1I~f'(O») + IEo> ( tr (D~ (e*)a2 A; (<Pi)) 1177i (<Pi) - J.li (£1*) 1I~!(9*») (8) + IEo> (1I77i(<Pi) - J.li(O*)II~!(9*)1I77i(<Pi) - J.li(e*)II~f'(9*») - tr (D!(e*)Vi(e*)) tr (D~(e*)Vi(e*)) + 2IEo> [ tr (D!(e*)a2A;(<pi)D~(e*)a2A;(<Pi))] + IEo> [ tr (D!(e*)a2A;(<Pi)) tr (D~(e*)a2A;(<Pi))] + 2 [a~/i( 0*) r v:-1 (e*)IEo> [( 77i( <Pi) - J.li( £1*)) {2 tr (D~( e*)a2 A;( <Pi)) + 11 77i ( <Pi) - J.li( £1*) lI~r(9*) } ] + 2 [a~m J.li( £1*) r v:-1 ( e*)lEo> [(77i( <Pi) - J.li( £1*)) {2 tr (D!( e*)a2 A;( <Pi)) + 1177i( <Pi) - J.li( £1*) II~i(9*) }] , where REFERENCES AMARI, S.l.. (1982). Differential geometry of curved exponential families. Curvature and information loss. Annals of Probability 10, 357-85. ANDREWS, D. (1987). Consistency in nonlinear econometric models: a generic uniform law of large numbers. Econometrica 55, 1465-71. BEAL, S.L. & SHEINER, L.B. (1992). Nonmem user's guide. Nonlinear mixed effects models for repeated measures data. University of California, San Francisco. DRAPER, N.R. & SMITH, H. (1981). Applied regression analysis (2nd ed.), WHey, New York. FELLER, W. (1971). An introduction to probability theory and its applications, vo1.2. Wiley, New York. GOURIEROUX, C. & MONFORT, A. (1991). Simulation based inference in models with heterogeneity. Annales d'Economie et de Statistique 20/21, 69-107. LINDSTOM, M.J. & BATES, D.M. (1990). Nonlinear mixed effects models for repeated measures data. Biometrics 46, 673-87. PINHEIRO, J.C. & BATES, D.M. (1995). Approximation of the log-likelihood function in the nonlinear mixed-effects model. Journal and Computational and Graphical Statistics. 4, 12-35. RAMOS, R.Q. & PANTULA, S.G. (1995). Estimation of nonlinear random coefficient models. Statistics and Probability Letters 24,49-56. VONESH, E.F. (1996). A note on the use of Laplace's approximation for nonlinear mixed-effects models. Biometrika 83, 447-52. WOLFINGER, R. (1993). Laplace's approximation for nonlinear mixed models. Biometrika 80, 791-95. 

http://creativecommons.org/licenses/by-nc-nd/3.0/es/

Asymptotic properties for a simulated pseudo maximum likelihood estimator

Abstract

Similar works

Full text

Available Versions

Name not available

Name not available

Universidad Carlos III de Madrid e-Archivo