In this paper, we consider the robust adaptive non parametric estimation problem for the periodic function observed with the Levy noises in continuous time. An adaptive model selection procedure, based on the improved weighted least square estimates, is proposed. Sharp oracle inequalities for the robust risks have been obtained

Pchelintsev, Evgeny A.

Pchelintsev, Valeriy A.

Pergamenshchikov, Serguei M.

English

Tomsk State University Repository

C O M M U N I C A T I O N S73V O L U M E  2 0  C O M M U N I C A T I O N S    1 / 2 0 1 8   ●1. Introduction In this paper, we consider a signal statistical treatment problem in the framework of a nonparametric regression model in continuous time, i.e. ,dy S t dt d t n0t t # #p= +^ h  (1)where S (.) is an unknown 1-periodic signal, t t n0p # #^ h  is an unobserved noise and  n is the duration of observation. The problem is to estimate the function  S on the observations yt t n0# #^ h . Note that if t t n0p # #^ h  is a Brownian motion, then we obtain the well-known “signal + white noise” model which is very popular in statistical radio-physics (see, for example, [1-3] and etc.). In this paper, we assume that in addition to the intrinsic noise in the radio-electronic system, approximated usually by the Gaussian white or color noise, the useful signal  S is distorted by the impulse flow described by the Levy process, i.e. we assume that the noise process t t n0p # #^ h  is defined asw zt t t1 2p t t= +   and  z x*t tn n= - u^ h  (2)where ρ1 and ρ2 are some unknown constants, wt t 0$^ h  is a standard Brownian motion, ds dxn^ h  is a jump measure with deterministic compensator ,ds dx ds dx $n P P=u ^ ^ ^h h h  is a Levy measure, i.e. some positive measure on 0R R* = " , , such thatx 12P =^ h   and  x6 31P^ h  (3)Here we use the notation x y dym mR*P P=^ ^h h# . Note that the Levy measure R*P^ h  could be equal to  + ∞ . We use * for the stochastic integrals with respect to random measures (see [4], Chs. 2 and 3), i.e.,x y ds dytt0 R) n n n n- -=)u u^ ^ ^h h h##  (4)In the sequel we will denote by  Q the distribution of the process t t n0p # #^ h  and by Q*n  we denote all these distributions for which the parameters 1 $t  ς * and 12 22 #t t+  ς*   where  ς * and  ς   * are some fixed positive bounds. The cause of the appearance of a pulse stream in the radio-electronic systems can be, for example, either external unintended (atmospheric) or intentional impulse noise and the errors in the demodulation and the channel decoding for the binary information symbols. Note that, for the first time the impulse noises for detection signal problems have been introduced on the basis of compound Poisson processes was NON ASYMPTOTIC SHARP ORACLE INEQUALITIES  FOR THE IMPROVED MODEL SELECTION PROCEDURES  FOR THE ADAPTIVE NONPARAMETRIC SIGNAL  ESTIMATION PROBLEMEvgeny Pchelintsev - Valeriy Pchelintsev - Serguei Pergamenshchikov*In this paper, we consider the robust adaptive non parametric estimation problem for the periodic function observed with the Levy noises in continuous time. An adaptive model selection procedure, based on the improved weighted least square estimates, is proposed.  Sharp oracle inequalities for the robust risks have been obtained.Keywords: improved non-asymptotic estimation, weighted least squares estimates, robust quadratic risk, non-parametric regression, Levy process, model selection, sharp oracle inequality, asymptotic efficiency* 1Evgeny Pchelintsev, 2Valeriy Pchelintsev, 3Serguei Pergamenshchikov    1Department of Information Technologies and Business Analytics, Tomsk State University, Russia   2Department of Mathematics and Informatics, Tomsk Polytechnic University, Russia   3Laboratoire de Mathematiques Raphael Salem, Universite de Rouen, Saint Etienne du Rouvray, France and International Laboratory of Statistics   of Stochastic Processes and Quantitative Finance of Tomsk State University, Russia  E-mail: evgen-pch@yandex.ruC O M M U N I C A T I O N S74 ● C O M M U N I C A T I O N S    1 / 2 0 1 8  V O L U M E  2 0considerably improve the non-asymptotic estimation accuracy. Such idea was proposed, for the first time, in [9]. Our goal is to develop these methods for non-Gaussian regression models in continuous time and to obtain the sharp oracle inequalities. It should be noted that to apply the improved estimation methods to the non-Gaussian regression models in continuous time one needs to modify the well-known James-Stein procedure introduced in [10] in the way proposed in [11, 12]. So, by using these estimators we construct the improved model selection procedure and we show that the constructed estimation procedure is optimal in the sense of the sharp non-asymptotic oracle inequalities for the robust risks which are defined in the Equation (5).2.  Improved estimationLet j j 1z $^ h  be an orthonormal basis in  L 2 [0, 1] . We extend these functions by the periodic way on R , i.e. t t 1j jz z= +^ ^h h  for any t R! . For estimating the unknown function  S in the Equation (1) we consider it’s Fourier expansion (see, for example, [13])S t tj jj 1i z=3=^ ^h h/  and ,S S t t dtj j j01i z z= =^ ^ ^h h h#  (7)The corresponding Fourier coefficients can be estimated asn t dy1,j nnt0i z=t ^ h#  (8)We define a class of weighted least squares estimates for  S (t) as S j ,jnj n j1m i z=m=t t^ h/  (9)where the weights Rn!m  belong to some finite set  Λ from [0, 1]n.Now, for the first  d Fourier coefficients in Equations (7) we use the improved estimation method proposed for parametric models in [12]. To this end we set ,n j n j d1i i= # #u t^ h . In the sequel we will use the norm x xd jjd2 21==/  for any vector x x j j d1= # #^ h  from Rn . Now we define the shrinkage estimators asg j1,*,j n j ni i= - t^ ^ hh  (10)where ,c 1 1g jn dnj d A1i= # #u^ h " ,  is the indicator of the set A and c n is some known parameter such that c ndn .  as n " 3 . Now we introduce a class of shrinkage weighted least squares estimates for  S asS j* ,*jnj n j1m i z=m=^ h/  (11)by Kassam in [2]. However, the compound Poisson process can describe only the large impulses influence of fixed single frequency. There should be taking into account that in telecommunication systems, impulses are without limitations on frequencies. So, one needs to extend the framework of the observation model by making use of the Levy processes as in Equations (2). In this paper, we consider the estimation problem in the adaptive setting, i.e. when the regularity of  S is unknown. Since the distribution  Q of the noise process t t n0p # #^ h  is unknown we use the robust estimation approach developed for nonparametric problems in [5]. We define the robust risk as, ,supS S S SR R* nQQ nQ*n=!t t^ ^h h  (5)where snt  is an estimation of the unknown signal S, i.e. some function of , ,y Rt t n Q0 $ $# #^ ^h h  is the usual quadratic risk defined as, ES S S SR* ,n Q S n2|= -t t^ h  and S S t dt2 201= ^ h#  (6)Here EQ,S stands for the expectation with respect to the distribution PQ,S of the process in the Equation (1) with a fixed distribution Q of the noise t t n0p # #^ h  and a given function S.In this paper, we consider minimax optimization criteria which aim to minimize the robust risk which is defined in the Equation (5) (see, for example, in [6]). To do this we use the model selection methods. The interest to such statistical procedures is explained by the fact that they provide adaptive solutions for a nonparametric estimation through oracle inequalities which give a non-asymptotic upper bound for a quadratic risk including a minimal risk over chosen family of estimators. It should be noted that the model selection methods for parametric models were proposed, for the first time, by Akaike [7]. Then, these methods had been developed by Barron, Birge and Massart [8] and Fourdrinier and Pergamenshchikov [9] for the nonparametric estimation and oracle inequalities for the quadratic risks. Unfortunately, the oracle inequalities obtained in these papers cannot provide the efficient estimation in the adaptive setting, since the upper bounds in these inequalities have some fixed coefficients in the main terms which are more than one. In order to obtain the efficiency property for estimation procedures, one has to obtain the sharp oracle inequalities, i.e. in which the factor at the principal term on the right-hand side of the inequality is close to unity. For this reason, one needs to use the general semimartingale approach for the robust adaptive efficient estimation of the nonparametric signals in continuous time proposed by Konev and Pergamenshchikov [5]. The goal of this paper is to develop a new sharp model selection method for estimating the unknown signal  S using the improved estimation approach. Usually, the model selection procedures are based on the least square estimators. However, in this paper, we propose to use the improved least square estimators which enable us to C O M M U N I C A T I O N S75V O L U M E  2 0  C O M M U N I C A T I O N S    1 / 2 0 1 8   ●Here we denoted by Tr j j 1$^ h  the trigonometric basis in  L ₂ [0, 1] . For this change in the empirical squared error, one has to pay some penalty. Thus, one comes to the cost function of the formJ j j P2,*,njnj njnj n n2121m m i m i d m= - += =u t^ ^ ^ ^ ^h h h h h/ /  (17) where  δ is some positive constant, Pn mt ^ h  is the penalty term defined asP nnn n2mv m=tt^ h  (18)Substituting the weight coefficients, minimizing the cost functionagrmin J* nm m= !m K ^ h  (19) in the Equation (11) leads to the improved model selection procedureS S* * *= m  (20)It will be noted that *m  exists because  Λ is a finite set. If the minimizing sequence in the Equation (19) *m  is not unique, one can take any minimizer. In the case, when the value of Qv  is known, one can take n Qv v=t  and P nn Q n2 1m v m= -^ h .Theorem 2. For any  n ≥ 2 and  0 < δ < 1 / 3 , the robust risks defined in the Equation (5) of estimate in the Equation (20) for continuously differentiable function  S satisfies the oracle inequality, ,minS S S SnB1 31 3R R* * * **n* #ddd-+ +!mmmK^ ^h h  (21) where the rest term is such that B n 0*n "f-  as n " 3  for any ε > 0 .The inequality (21) means that the procedure in the Equation (20) is optimal in the oracle inequalities sense. This property enables to provide asymptotic efficiency in the adaptive setting, i.e. when information about the signal regularity is unknown.4.  Monte Carlo simulationsIn this section we report the results of a Monte Carlo experiment to assess the performance of the proposed model selection procedure in the Equation (20). In the Equation (1) we choose 1-periodic function S which, for  0 ≤ t ≤ 1 , is defined as S t t21= -^ h , if t41 43# #  and S t 41=^ h  elsewhere. We simulate the Equation (1) with the noise process defined as. .w z0 5 0 5t t tp = +  (22)We denote the difference of quadratic risks of the estimates in Equation (11) and Equation (9) as : ,S S SR *Q QD = -m^ ^h h  ,S SRQ- mt^ h . Now for this deviation we obtain the following result.Theorem 1. Assume that for any vector  λ ∈ Λ there exists some fixed integer  d = d (λ) such that their first  d components equal to one, i.e.  λ (j) = 1 for  1 ≤ j ≤ d for any  λ ∈ Λ . Then for any n ≥ 1 and r>0csupsup SrQ SQ n2Qn1D -! #^ h  (12)The inequality (12) means that non-asymptotically, i.e. for any  n ≥ 1 the estimate in the Equation (11) outperforms in mean square accuracy the estimate in the Equation (9). Moreover, as we will see below,  n  c n → ∞  as d → ∞ . This means that improvement is considerable may be better than for the parametric regression [11].3.  Model selectionThis Section gives the construction of a model selection procedure for estimating a function  S in the Equation (1) on the basis of improved weighted least square estimates and states the sharp oracle inequality for the robust risk of proposed procedure.The model selection procedure for the unknown function  S in the Equation (1) will be constructed on the basis of a family of estimates S* !m m K^ h .The performance of any estimate S*m  will be measured by the empirical squared errorErr S S*n2m = -m^ h  (13)In order to obtain a good estimate, we have to write a rule to choose a weight vector  λ ∈ Λ in the Equation (6). It is obvious, that the best way is to minimize the empirical squared error with respect to  λ . Making use the estimate definition in the Equation (6) and the Fourier transformation of  S impliesErr j j2,*,*njnj njnj n j jjn212121m m i m i i i= - += = =^ ^ ^ ^h h h h/ / /  (14)Since the Fourier coefficients j j 1i $^ h  are unknown, the weight coefficients j j 1m $^ h  cannot be found by minimizing this quantity. To circumvent this difficulty one needs to replace the terms ,*j n ji i  by their estimators ,j niu . We setn, ,*,j n j n j nni i i v= -u tt (15) where nvt  is the estimate for the noise variance of E ,Q Q j n2v p= =  1222t t= + which we choose in the following formt ,nj nnj n21v == +t t6 @/   and  Trt n t dy1,j n jnt0=t ^ h#  (16)C O M M U N I C A T I O N S76 ● C O M M U N I C A T I O N S    1 / 2 0 1 8  V O L U M E  2 0since the accuracy improvement is proportional to the parameter dimension. We remember that for the nonparametric estimation this dimension tends to infinity, but in the parametric case it is always fixed. Therefore, the gain in the non-asymptotic quadratic accuracy from the application of the improved estimation methods is much more significant in statistical treatment problems of nonparametric signals. Moreover, as shown by the results of numerical simulation, the improved estimate for non-asymptotic accuracy exceeds the usual least squares estimators approximately in 1.5-2 times for the chosen experimental model.AcknowledgementThe results of section 2 of this work are supported by the Ministry of Education and Science of the Russian Federation in the framework of the research project no. 2.3208.2017/4.6 and the RFBR Grant 16-01-00121 A. The results of Section 3 are supported by the RSF grant number 14-49-00079 (National Research University “MPEI”, Moscow, Russia). The last author is partially supported by the Russian Federal Professor program (project no. 1.472.2016/1.4, Ministry of Education and Science) and by the project XterM-Feder, University of Rouen.          where ,z Y Nt j tjN1t==/  is a Poisson process with the intensity λ=1 and Yj j 1$^ h  is i.i.d. Gaussian (0,1). We use the model selection procedure defined in the Equation (20) with the weights proposed in [5]: , /ln lnk n n100 1* f= + =  and /m 1 2f= 6 @. We used the cost function with lnn3 2d = + -^ h . We define the empirical risk as , ES S p S t S tR1n j jjp12= -=u t u^ ^ ^ ^h h hh/  and E S S N S S1n nllN2 21$ $ $ $- = -=t u u^ ^ ^ ^ ^ ^h hh h hh/  with the frequency of observations p=100001 and numbers of replications N=10000. Table 1 give the values for the sample risks for different numbers of observation period n.5.  Conclusion In conclusion, we would like to emphasize that in this paper we developed new model selection procedures based on the improved versions of the least square estimators. It turns out that the improvement effect in the nonparametric estimation is more important than for the parameter estimation problems References[1]  CHERNOYAROV, O. V., VACULIK, M., SHIRIKYAN, A., SALNIKOVA, A. V.: Statistical Analysis of Fast Fluctuating Random Signals with Arbitrary - Function Envelope and Unknown Parameters. Communications - Scientific Letters of the University of Zilina, 17(1a), 35-43, 2015. [2]  KASSAM, S. A.: Signal Detection in Non-Gaussian Noise. Springer, New York, 1988.[3]  IBRAGIMOV, I. A., KHASMINSKII, R. Z.: Statistical Estimation: Asymptotic Theory. Springer, New York, 1981. [4]  CONT, R., TANKOV, P.: Financial Modelling with Jump Processes. Chapman & Hall, London, 2004.[5]  KONEV, V. V., PERGAMENSHCHIKOV, S.: Robust Model Selection for a Semimartingale Continuous Time Regression from Discrete Data. Stochastic Processes and their Applications, 125, 294-326, 2015.[6]  JANACEK, J., KVET, M.: Min-Max Optimization of Emergency Service System by Exposing Constraints. Communications - Scientific Letters of the University of Zilina, 17(2), 15-22, 2015.[7]  AKAIKE, H.: A New Look at the Statistical Model Identification.  IEEE Transactions on Automatic Control, 19(7), 716 – 723, 1974.[8]  BARRON, A., BIRGE, L., MASSART, P.: Risk Bounds for Model Selection via Penalization. Probability Theory and Related Fields, 113, 301-415, 1999.[9]  FOURDRINIER, D., PERGAMENSHCHIKOV, S.: Improved Selection Model Method for the Regression with Dependent Noise. Annals of the Institute of Statistical Mathematics, 59(3), 435-464. 2007.Table 1 Empirical risks        n ,R S St^ h ,S SR *^ h ,R S St^ h / ,S SR *^ h20 0.0462 0.0331 1.40100 0.0262 0.0133 1.97200 0.0126 0.00824 1.531000 0.00129 0.00094 1.37C O M M U N I C A T I O N S77V O L U M E  2 0  C O M M U N I C A T I O N S    1 / 2 0 1 8   ●[10] JAMES, W., STEIN, C.: Estimation with Quadratic Loss. Proceedings of the Fourth Berkeley Symposium Mathematics, Statistics and Probability, California, USA, 1, 361-380, 1966.[11]  KONEV, V., PERGAMENSHCHIKOV, S., PCHELINTSEV, E.: Estimation of a Regression with the Pulse Type Noise from Discrete Data. Theory of Probability and Its Applications, 58(3), 442-457, 2014.[12]  PCHELINTSEV, E.: Improved Estimation in a Non-Gaussian Parametric Regression. Statistical Inference for Stochastic Processes, 16(1), 15-28, 2013.[13]  MARCOKOVA, M.: Equiconvergence of Two Fourier series. Journal of Approximation Theory, 80(2), 151-163, 1995.

Non asymptotic sharp oracle inequalities for the improved model selection procedures for the adaptive nonparametric signal estimation problem

http://vital.lib.tsu.ru/vital/access/services/Download/vtls:000658494/SOURCE1

Non asymptotic sharp oracle inequalities for the improved model selection procedures for the adaptive nonparametric signal estimation problem

Abstract

Similar works

Full text

Available Versions

Tomsk State University Repository