Barbu, V. S.

Beltaief, S.

Pergamenshchikov, Serguei M.

English

Tomsk State University Repository

МИНИСТЕРСТВО ОБРАЗОВАНИЯ И НАУКИРОССИЙСКОЙ ФЕДЕРАЦИИНАЦИОНАЛЬНЫЙ ИССЛЕДОВАТЕЛЬСКИЙТОМСКИЙ ГОСУДАРСТВЕННЫЙ УНИВЕРСИТЕТМеждународная лаборатория статистики случайныхпроцессов и количественного финансового анализаМеждународная научнаяконференция«Робастная статистика ифинансовая математика – 2017»(03–05 июля 2017 г.)Сборник статейПод редакциейд-ра физ.-мат. наук, профессора С.М. Пергаменщикова,канд. физ.-мат. наук, доцента Е.А. ПчелинцеваТомскИздательский Дом Томского государственного университета2017Model selection for a semi - Markovcontinuous time regression observed in thediscrete time moments *Barbu V.S., Beltaief S., Pergamenshchikov S.Université de Rouen Normandie, Rouen,Tomsk State University, Tomske-mail: Serge.Pergamenshchikov@univ-rouen.frAbstractIn this article we consider the nonparametric robust estima-tion problem for regression models in continuous time with semi-Markov noises observed in discrete time moments. An adaptivemodel selection procedure is proposed. A sharp non-asymptoticoracle inequality for the robust risks is obtained. We obtain suf-ficient conditions on the frequency observations under which therobust efficiency is shown. It turns out that for the semi-Markovmodels the robust minimax convergence rate may be faster orslower than the classical one.Keywords: Non-asymptotic estimation; Robust risk; Modelselection; Sharp oracle inequality; Asymptotic efficiencyIn this paper we consider the semi-Markov regression model in con-tinuous time introduced in [1], i.e.𝑑 𝑦𝑡 = 𝑆(𝑡)𝑑 𝑡 + 𝑑 𝜉𝑡 , 0 ≤ 𝑡 ≤ 𝑛 , (1)where 𝑆(·) is an unknown 1-periodic function defined on R with valueson R, (𝜉𝑡)𝑡≥0 is the unobserved noise process defined through a certainsemi-Markov process in [1].Our problem in the present paper is to estimate the unknown func-tion 𝑆 in the model (1) on the basis of observations(𝑦𝑡𝑗 )0≤𝑗≤𝑛𝑝, 𝑡𝑗 = 𝑗∆, ∆ =1𝑝, (2)where the integer 𝑝 ≥ 1 is the observation frequency. Firstly, this prob-lem was considered in the framework “signal+white noise” (see, for ex-ample, [3] or [11]). Later, to introduce a dependence in the continuoustime regression model in [10], [6], [4], [5] [7], the Ornstein-Uhlenbeckprocesses has been used to model the “color noise”. Moreover, in orderto introduce the dependence and the jumps in the regression model*The work was supported by the RSF grant 17-11-01049.5(1), the papers [8] and [9] use the non Gaussian Ornstein-Uhlenbeckprocesses defined in [2]. The problem in all these papers is that theintroduced Ornstein-Uhlenbeck type of dependence decreases with ageometric rate. So, asymptotically when the duration of observationsgoes to infinity, we obtain the same “signal+white noise” model veryquick. To keep the dependence for sufficiently large duration of ob-servations, in [1] it was proposed the model (1) with a semi-Markovcomponent in the jumps of the noise process (𝜉𝑡)𝑡≥0.The main goal of this paper is to develop adaptive robust methodfrom [1], that was based on continuous observations, to the estimationproblem based on discrete observations given in (2). In this paper weuse quadratic risk defined asℛ𝑄(𝑆𝑛, 𝑆) = E𝑄,𝑆 ‖𝑆𝑛 − 𝑆‖2 , (3)where 𝑆𝑛(·) is some estimate (i.e. any periodical function measurablewith respect to the observations 𝜎{𝑦𝑡0 , . . . 𝑦𝑡𝑝𝑛}), ‖𝑓‖2 =∫︀ 10𝑓2(𝑠)𝑑𝑠and E𝑄,𝑆 is the expectation with respect to the distribution P𝑄,𝑆 ofthe process (1) corresponding to the unknown noise distribution 𝑄 inthe Skorokhod space 𝒟[0, 𝑛]. We assume that this distribution belongsto some distribution family 𝒬𝑛 specified in [1].To study the properties of the estimators uniformly over the noisedistribution (what is really needed in practice), we use the robust riskdefined asℛ*𝑛(𝑆𝑛, 𝑆) = sup𝑄∈𝒬𝑛ℛ𝑄(𝑆𝑛, 𝑆) . (4)Thus the goal of this paper is to develop a robust efficient modelselection method based on the observations (2) for the model (1) withthe semi-Markov components in the jumps of the noise (𝜉𝑡)𝑡≥0. Weuse the approach proposed by Konev and Pergamenshchikov in [9] forcontinuous-time regression models observed in the discrete time mo-ments. Unfortunately, we cannot use directly this method for semi-Markov regression models, since their tool essentially uses the fact thatthe Ornstein-Uhlenbeck dependence decreases with geometrical rateand obtain sufficiently quickly the “white noise” case. In the presentpaper, in order to obtain the sharp non-asymptotic oracle inequalities,we use the renewal methods from [1] developed for the model (1). As aconsequence, we can obtain the constructive sufficient conditions thatprovide the robust efficiency for proposed model selection procedures.In this paper we construct some special family of the weight leastsquare estimators (𝑆𝜆)𝜆∈Λ, where Λ is some finite set from [0, 1]𝑛. Inthe sequel we assume that the number of the vector of the 𝜈 = cardΛ6is a function of 𝑛, i.e. 𝜈 = 𝜈𝑛, such that for any 𝛾 > 0lim𝑛→∞𝜈𝑛𝛾= 0 . (5)The model select procedure for this family is defined as𝑆* = 𝑆?^? and ?ˆ? = argmin𝜆∈Λ𝐽𝑛,𝛿(𝜆) , (6)where 𝐽𝑛,𝛿(𝜆) is the cost function defined for any in [1] and 0 < 𝛿 < 1is the penalty parameter.We assume that the renewal distribution in the semi-markov com-ponent of the noise process in (1) has a density 𝑔 that satisfies thefollowing conditions.H1) Assume that, for any 𝑥 ∈ R, there exist the finite limits𝑔(𝑥−) = lim𝑧→𝑥− 𝑔(𝑧) and 𝑔(𝑥+) = lim𝑧→𝑥+ 𝑔(𝑧)and, for any 𝐾 > 0, there exists 𝛿 = 𝛿(𝐾) > 0 for whichsup|𝑥|≤𝐾∫︁ 𝛿0|𝑔(𝑥 + 𝑡) + 𝑔(𝑥− 𝑡)− 𝑔(𝑥+)− 𝑔(𝑥−)|𝑡𝑑𝑡 < ∞. (7)H2) For any 𝛾 > 0,sup𝑧≥0𝑧𝛾 |2𝑔(𝑧)− 𝑔(𝑧−)− 𝑔(𝑧+)| < ∞ .H3) There exists 𝛽 > 0 such that∫︀R𝑒𝛽𝑥 𝑔(𝑥) 𝑑𝑥 <∞.H4) There exists 𝑡* > 0 such that the function 𝑔(𝜃 − 𝑖𝑡) belongs toL1(R) for any 0 ≤ 𝑡 ≤ 𝑡*, where𝑔(𝜃) =12𝜋∫︁R𝑒𝑖𝜃𝑥 𝑔(𝑥) 𝑑𝑥 .Theorem 1. Assume that the function 𝑆 is continuously differentiableand that Conditions H1)–H4) hold true. Then there exists some con-stant 𝑙* > 0 such that for any noise distribution 𝑄, the weight vec-tors set Λ, for any periodic function 𝑆 for any 𝑛 ≥ 1, 𝑝 ≥ 3 and0 < 𝛿 ≤ 1/6, the procedure (6) satisfies the following oracle inequalityℛ𝑄(𝑆*, 𝑆) ≤ 1 + 3𝛿1− 3𝛿 min𝜆∈Λℛ𝑄(𝑆𝜆, 𝑆) + 𝑙* 𝜈𝛿𝑛(8)andℛ*(𝑆*, 𝑆) ≤ 1 + 3𝛿1− 3𝛿 min𝜆∈Λℛ*(𝑆𝜆, 𝑆) +𝑈*𝑛𝛿𝑛, (9)where the the rest term 𝑈*𝑛 is such that for any 𝛾 > 0lim𝑛→∞𝑈*𝑛𝑛𝛾= 0 .7Remark 1. The oracle inequalities (8) and (9) are called sharp or-acle inequalities, since the coefficients in the main terms in the rightside closed to one. As we will see below this property allows to showasymptotic efficiency in the adaptive setting.In order to obtain the efficiency property, we specify the weightcoefficients (𝜆(𝑗))1≤𝑗≤𝑛 in the procedure (6). Consider, for some fixed0 < 𝜀 < 1, a numerical grid of the form𝒜 = {1, . . . , 𝑘*} × {𝜀, . . . ,𝑚𝜀}, (10)where 𝑚 = [1/𝜀2]. We assume that both parameters 𝑘* ≥ 1 and 𝜀 arefunctions of 𝑛, i.e. 𝑘* = 𝑘*(𝑛) and 𝜀 = 𝜀(𝑛), such that⎧⎪⎪⎨⎪⎪⎩lim𝑛→∞ 𝑘*(𝑛) = +∞ , lim𝑛→∞ 𝑘*(𝑛)ln𝑛= 0 ,lim𝑛→∞ 𝜀(𝑛) = 0 and lim𝑛→∞ 𝑛𝛾𝜀(𝑛) = +∞(11)for any 𝛾 > 0. One can take, for example, for 𝑛 ≥ 2𝜀(𝑛) =1ln𝑛and 𝑘*(𝑛) = 𝑘*0 +√ln𝑛 , (12)where 𝑘*0 ≥ 0 is some fixed constant. For each 𝛼 = (𝛽, 𝜏) ∈ 𝒜, weintroduce the weight sequence𝜆𝛼 = (𝜆𝛼(𝑗))1≤𝑗≤𝑝with the elements𝜆𝛼(𝑗) = 1{1≤𝑗<𝑗*} +(︀1− (𝑗/𝜔𝛼)𝛽)︀1{𝑗*≤𝑗≤𝜔𝛼}, (13)where 𝑗* = 1 + [ln 𝜐𝑛], 𝜔𝛼 = (d𝛽 𝜏𝜐𝑛)1/(2𝛽+1),d𝛽 =(𝛽 + 1)(2𝛽 + 1)𝜋2𝛽𝛽and 𝜐𝑛 = 𝑛/𝜍* .Threshold 𝜍* is a function of 𝑛, i.e. 𝜍* = 𝜍*(𝑛) such that for any 𝛾 > 0lim inf𝑛→∞ 𝑛𝛾 𝜍*(𝑛) = +∞ and lim sup𝑛→∞𝑛−𝛾 𝜍*(𝑛) = 0 . (14)Now we define the set Λ asΛ = {𝜆𝛼 , 𝛼 ∈ 𝒜} . (15)These weight coefficients are used in [8, 9] for continuous time re-gression models to show the asymptotic efficiency. Note also that inthis case the cardinal of the set Λ is 𝜈 = 𝑘*𝑚. Therefore, the condi-tions in (11) yield the property (5). Moreover, to obtain the efficiencyfor the model selection procedure we assume the following condition onthe frequency of the observations.H5) Assume that for any 𝑛 ≥ 3 the observation frequence 𝑝 ≥ 𝑛5/6.To this end, we assume that the unknown function 𝑆 in the model8(1) belongs to the Sobolev ball𝑊 𝑘𝑟 = {𝑓 ∈ 𝒞𝑘𝑝𝑒𝑟[0, 1] ,𝑘∑︁𝑗=0‖𝑓 (𝑗)‖2 ≤ 𝑟} , (16)where 𝑟 > 0 , 𝑘 ≥ 1 are some parameters, 𝒞𝑘𝑝𝑒𝑟[0, 1] is the set of 𝑘times continuously differentiable functions 𝑓 : [0, 1] → R such that𝑓 (𝑖)(0) = 𝑓 (𝑖)(1) for all 0 ≤ 𝑖 ≤ 𝑘. The function class 𝑊 𝑘𝑟 can bewritten as an ellipsoid in 𝑙2, i.e.𝑊 𝑘𝑟 = {𝑓 ∈ 𝒞𝑘𝑝𝑒𝑟[0, 1] :∞∑︁𝑗=1𝑎𝑗 𝜃2𝑗 ≤ 𝑟} (17)where 𝑎𝑗 =∑︀𝑘𝑖=0 (2𝜋[𝑗/2])2𝑖.Similarly to [8,9] we will show here that the asymptotic sharp lowerbound for the robust risk (4) is given by𝑟*𝑘 = 𝑙 ((2𝑘 + 1)𝑟)1/(2𝑘+1)(︂𝑘(𝑘 + 1)𝜋)︂2𝑘/(2𝑘+1). (18)Note that this is the well-known Pinsker constant obtained for thenonadaptive filtration problem in “signal + small white noise” model(see, for example, [11]).Let Π𝑛 be the set of all estimators 𝑆𝑛 measurable with respect tothe sigma-algebra 𝜎{𝑦𝑡 , 0 ≤ 𝑡 ≤ 𝑛} generated by the process (1).Theorem 2. Under Conditions (14)lim inf𝑛→∞ 𝜐2𝑘/(2𝑘+1)𝑛 inf𝑆𝑛∈Π𝑛sup𝑆∈𝑊𝑘𝑟ℛ*𝑛(𝑆𝑛, 𝑆) ≥ 𝑟*𝑘 , (19)where 𝜐𝑛 = 𝑛/𝜍*.Note that, if the parameters 𝑟 and 𝑘 are known, i.e. for the non-adaptive estimation case, in order to obtain the efficient estimation forthe “signal+white noise” model, Pinsker proposed in [11] to use theestimate 𝑆𝜆0 with the weights (13) in which𝜆0 = 𝜆𝛼0 and 𝛼0 = (𝑘, 𝜏0) , (20)where 𝜏0 = [𝑟/𝜀]𝜀. For the semi-markov regression model (1) we showthe same result.Proposition 3. The estimator 𝑆𝜆0 satisfies the following asymptoticupper boundlim𝑛→∞ 𝜐2𝑘/(2𝑘+1)𝑛 sup𝑆∈𝑊𝑘𝑟ℛ*𝑛(𝑆𝜆0 , 𝑆) ≤ 𝑟*𝑘 .For the adaptive estimation we user the model selection procedure(6) with the parameter 𝛿 defined as a function of 𝑛 satisfyinglim𝑛−→∞ 𝛿𝑛 = 0 and lim𝑛−→∞ 𝑛𝛾 𝛿𝑛 = 0 (21)9for any 𝛾 > 0. For example, we can take 𝛿𝑛 = (6 + ln𝑛)−1. Now theoracle inequality (9) for the family (15) and Proposition 3 imply thefollowing upper bound.Theorem 4. Assume that Conditions H1)–H5) hold true. Then therobust risk defined in (4) for the procedure (6) with the coefficients (13)and the parameter 𝛿 = 𝛿𝑛 satisfying (21) has the following asymptoticupper boundlim sup𝑛→∞𝜐2𝑘/(2𝑘+1)𝑛 sup𝑆∈𝑊𝑘𝑟ℛ*𝑛(𝑆*, 𝑆) ≤ 𝑟*𝑘 . (22)Theorem 2 and Theorem 4 imply the following result.Corollary 5. Under the conditions of Theorem 4, the model selectionprocedure 𝑆* is efficient, i.e.lim𝑛→∞inf𝑆𝑛∈Π𝑛 sup𝑆∈𝑊𝑘𝑟 ℛ*𝑛(𝑆𝑛, 𝑆)sup𝑆∈𝑊𝑘𝑟 ℛ*𝑛(𝑆*, 𝑆)= 1 . (23)Remark 2. It is well known that the optimal (minimax) risk conver-gence rate for the Sobolev ball 𝑊 𝑘𝑟 is 𝑛2𝑘/(2𝑘+1) (see, for example, [11]).We see here that the efficient robust rate is 𝜐2𝑘/(2𝑘+1)𝑛 , i.e. if the dis-tribution upper bound 𝜍* → 0 as 𝑛 → ∞ we obtain a faster rate withrespect to 𝑛2𝑘/(2𝑘+1), and if 𝜍* → ∞ as 𝑛 → ∞ we obtain a slowerrate. In the case when 𝜍* is constant the robuste rate is the same asthe classical non robuste convergence rate.Conclusion. In the conclusion we would like to emphasize that inthis paper :∙ we construct a selection model procedure based on the weight leastsquare estimators;∙ we find conditions for which we obtained an sharp non asymptoticoracle inequalities for the simple quadratic risks and for the robustrisks as well;∙ using the Pinsker method we obtain a lower bound for the robustquadratic risks, then, through the obtained sharp oracle inequal-ities we show that the risk upper bound for the constructed pro-cedure matters this lower bound, i.e. the procedure is efficient inthe adaptative setting.10References1. Barbu V.S. Robust adaptive efficient estimation for semi-Markovnonparametric regression models / Barbu V.S., Beltaif S., Perga-menchtchikov S.M. // Statistical inference for stochastic processes. 25March 2017.— (Preprint / Statistical inference for stochastic processes)– URL: https://arxiv.org/pdf/1604.04516.pdf2. Barndorff-Nielsen O. E. and Shephard N. Non-GaussianOrnstein-Uhlenbeck-based models and some of their uses in financialmathematics // J. Royal Stat. Soc. 2001. V. B 63. P. 167—241.3. Ibragimov I. A. Statistical Estimation: Asymptotic Theory /Ibragimov I. A., Khasminskii R. Z.—Springer, Berlin, New York. 1981.4. Höpfner R. and Kutoyants Yu. A. On LAN for parametrized con-tinuous periodic signals in a time inhomogeneous diffusion // Statist.Decisions. 2009. V. 27. No. 4. P. 309—326.5. Höpfner R., Kutoyants Yu. A. Estimating discontinuous periodicsignals in a time inhomogeneous diffusion // Statistical Inference forStochastic Processes. 2010. V.13. No. 3. P. 193—230.6. Konev V. V., Pergamenshchikov S. M. Sequential estimation ofthe parameters in a trigonometric regression model with the gaussiancoloured noise // Statistical Inference for Stochastic Processes. 2003.V. 6. P. 215—235.7. Konev V. V., Pergamenshchikov S. M. General model selectionestimation of a periodic regression with a Gaussian noise // Annals ofthe Institute of Statistical Mathematics. 2010. V. 62. P. 1083—1111.8. Konev V. V., Pergamenshchikov S.M. Efficient robust nonpara-metric estimation in a semimartingale regression model // Ann. Inst.Henri Poincaré Probab. Stat. 2012. V. 48. No. 4. P. 1217—1244.9. Konev V.V. and Pergamenshchikov S.M. Robust model selec-tion for a semimartingale continuous time regression from discrete data//Stochastic processes and their applications. 2015. V. 294—326.10. Kutoyants Yu. A. Estimation of signal parameter in Gaussiannoise // Problems of Information Transmission. 1977. V. 13. P. 266-271.11. Pinsker M. S. Optimal filtration of square integrable signals ingaussian white noise // Problems of Transimission information. 1981.V. 17. P. 120–133.11

Model selection for a semi - Markov continuous time regression observed in the discrete time moments

http://vital.lib.tsu.ru/vital/access/services/Download/vtls:000620491/SOURCE1

Model selection for a semi - Markov continuous time regression observed in the discrete time moments

Abstract

Similar works

Full text

Available Versions

Tomsk State University Repository