Burkatovskaya, Yulia B.

Vasiliev, Vyacheslav A.

English

Tomsk State University Repository

Applied Methods of Statistical AnalysisParameter estimation with guaranteed accuracyfor AR(1) by noised observationsYulia B. Burkatovskaya1,2 and Vyacheslav A. Vasiliev11 Tomsk State University, Tomsk, Russian Federation2 Tomsk Polytechnic University, Tomsk, Russian Federatione-mail: tracey@tpu.ru, vas@mail.tsu.ruAbstractThis paper presents a truncated estimator of the dynamic parameter of astable AR(1) process by observations with additive noise. The estimator isconstructed by a sample of a fixed size and it has a known upper bound of themean square deviation. Cases of known and unknown variance of observationnoise are considered.Keywords: autoregressive process, fixed sample size, guaranteed accuracy,observations with noise.1 Introduction and problem statementDevelopment of parameter estimation methods of dynamic systems by samples offinite or fixed size is very important in statistical problems such that model construc-tion and various adaptive problems (prediction, control, filtration etc.).One of the possibilities for finding estimators with the guaranteed quality of infer-ence using a sample of fixed size is provided by the approach of truncated estimation.Truncated estimators were constructed in [9] for ratio type multivariate functionalsby a fixed-size sample. They have guaranteed accuracy in the sense of the L2m-norm,m ≥ 1. This fact allows one to obtain desired non-asymptotic and asymptotic prop-erties of the estimators. The truncated estimation method was developed in [1] andothers for parameter estimation problems in discrete-time dynamic models. Solutionsof some non-asymptotic parametric and non-parametric problems can be found alsoin [4], [8], [5], [6], among others. In particular, [8] established the minimax optimalityof the least-squares estimator of the dynamic parameter in AR(1) model.In this paper, the truncated estimation method introduced in [9] is applied forthe parameter estimation of AR(1) by additively-noised observations with unknownnoise variance (another applications of this method can be found, e.g., in [2], [3]).Consider the estimation problem of the parameter λ of the scalar first-order au-toregressive process (xn)n≥0 satisfying the equationxn = λxn−1 + ξn, n ≥ 1 (1)by observationsyn = xn + ηn, n ≥ 0. (2)Process (1) is supposed to be stable, i.e. |λ| < 1. Introduce the notation ζ =(x0, ξ1, η0.) The processes (ξn), (ηn) and x0 are supposed to be mutually independent;219Novosibirsk, 18-20 September, 2019noises ξn and ηn form sequences of i.i.d. random variables such that Eζ = 0, E||ζ||4 <∞. Denote σ2 = Eη20. We assume that the variance of ξ1 is known. Then withoutloss of generality we put Eξ21 = 1.The main aim of the paper is to construct truncated estimators of λ ∈ (−1, 1)with guaranteed accuracy in the mean square sense by sample of fixed size. Cases ofboth known and unknown values of σ2 will be considered.A similar problem has been solved in, e.g., [10] on the basis of the sequentialapproach (when the sample size is a random value determined by a special stoppingrule) for λ ∈ (−1, 0) ∩ (0, 1)2 Parameter estimation of AR(1) with known noisevarianceTo estimate the parameter λ, we use the correlation method. To this end, we obtainfrom the system (1), (2) the recurrent equation for the observed process y = (yn)n≥0 :yn = λyn−1 + δn, n ≥ 1,δn = ξn + ηn − ληn−1. (3)Due to the dependence of noises δn, the least squares estimator (LSE) of λ obtainedfrom equation (3) is asymptotically biased, see, e.g., [7], [10]. Equation (3) impliesthe following formula for correlations of the process (yn):Eλynyn−1 = λEλ(y2n−1 − σ2), n ≥ 1.Hence, the consistent correlation estimator λˆn of λ has the following form (see [7])λˆn,σ =n∑k=1ykyk−1n∑k=1(y2k−1 − σ2), n ≥ 1. (4)It is easy to verify thatlimn→∞1nn∑k=1(y2k−1 − σ2) =11− λ2 > 1 Pλ − a.s. (5)Thus, according to the general procedure described in [9], it is reasonable toconstruct the truncated estimator λ˜n of λ as follows:λ˜n = λˆn · χ(n∑k=1(y2k−1 − σ2) ≥ hn), n ≥ 1, (6)where h ∈ (0, 1) and χ(A) is the indicator of the set A.The following theorem gives the first main result of this paper.220Applied Methods of Statistical AnalysisTheorem 1. Assume model (1), (2). Then for every |λ| < 1 and n ≥ 1, estimator(7) has the propertyEλ(λ˜n − λ)2 ≤ Cn. (7)The proofs of theorems and lemmas are given in Section 5.3 Parameter estimation of AR(1) with unknown noisevarianceTo estimate λ ∈ (−1, 1), we use an adaptive modification of estimator (5):λ∗n =1nn∑k=1ykyk−11nn∑k=1y2k−1 − σ2n, n > 1. (8)Taking into account (6), we construct the estimator σ2n of σ2 as followsσ2n =1nn∑k=1y2k−1 −11− λ2n, n > 1 (9)where λn is the pilot estimator of λλn = proj[−1,1]λ˘n, n > 1, (10)λ˘n =n∑k=2ykyk−2n∑k=2yk−1yk−2· χ(|n∑k=2yk−1yk−2| ≥ Hn), n > 1. (11)Here we put Hn = n(log n)−1. According to the general truncated estimation method[9], the multiplier (log n)−1 in the definition of Hn can be any other slowly-decreasingfunction.It should be noted that the estimator (10) is constructed on the bases of thecorrelation (Yule-Walker type) estimator which can not be used if λ = 0 (see Lemma1 below). Our main aim is to construct an estimator of λ without this restriction.Taking into account (10), estimator (9) can be written in the formλ∗n = (1− λ2n)1nn∑k=1ykyk−1, n > 1. (12)221Novosibirsk, 18-20 September, 2019Lemma 1. Assume that in model (1), (2), E||ζ||8 < ∞. Then estimator (10) forevery λ ∈ (−1, 0) ∪ (0, 1) and n > 1 has the following propertyEλ(λn − λ)2 ≤ C1n+ C2log4 nn2.This lemma makes possible to obtain the main result of the section.Theorem 2. Assume that in model (1), (2), E||ζ||8 < ∞. Then for every |λ| < 1and n > 1, estimator (12) satisfies the following conditionEλ(λ∗n − λ)2 ≤Cn+ Clog4 nn2.4 Simulation Results and DiscussionWe conducted numerical simulation of the proposed estimation algorithm. For everyset of the parameters, the experiment was performed 100 times, the number of ob-servations is equal to 100, the parameter of the procedure h = 0, 5. Table 1 presentsthe results of simulation. Here λ and σ are the parameters of model (1), λ˜n and λ∗nare the mean estimators of the parameter λ when the noise variance σ2 is supposedto be known and unknown, correspondingly; d˜n and d∗n are sample standard errors ofthe corresponding estimators.One can see that d˜n < d∗n in all experiments; thus, if the noise variance is unknownthen the standard error increases at least twice (if λ = 0, 5); but d∗n can be fully tentimes larger than d˜n, if λ = 0, 9. Both deviations increase with the grow of σ2, as oneshould expect; besides, d˜n decreases and d∗n increases with the increase of λ.5 Proofs5.1 Proof of Theorem 1To investigate the non-asymptotic properties of λ˜n we use the following representationof the deviationλ˜n − λ = fngn· χ(|gn| ≥ h)− λχ(|gn| < h), (13)wherefn =1nn∑k=1[yk−1(ξk + ηk)− λ(yk−1ηk−1 − σ2)],gn =1nn∑k=1(y2k−1 − σ2).It can be directly verified that for |λ| < 1Eλf2n ≤I−1(λ, σ)n. (14)222Applied Methods of Statistical AnalysisTable 1: Simulation resultsλ σ2 λ˜n d˜n λ∗n d∗n0,5 0,09 0,477 0,0092 0,452 0,02940,5 0,25 0,492 0,0111 0,490 0,03140,5 0,49 0,487 0,0150 0,470 0,04880,5 0,81 0,475 0,0229 0,418 0,07950,5 1 0,473 0,0419 0,424 0,09530,8 0,09 0,786 0,0046 0,796 0,04650,8 0,25 0,794 0,0054 0,854 0,07930,8 0,49 0,786 0,0054 0,789 0,07370,8 0,81 0,772 0,0120 0,765 0,14350,8 1 0,788 0,0122 0,797 0,15900,9 0,09 0,876 0,0038 0,865 0,07720,9 0,25 0,889 0,0018 0,913 0,05960,9 0,49 0,888 0,0030 0,910 0,10440,9 0,81 0,874 0,0044 0,886 0,18220,9 1 0,891 0,0028 0,891 0,1780Introduce the notation g = 1/(1− λ2). Then, using a representationgn − g = 1nn∑k=1(x2k−1 − σ2) +2nn∑k=1xk−1ηk−1 +1nn∑k=1(η2k−1 − σ2)and the following formula (see, e.g., the proof of Theorem 2 in [9])1nn∑k=1(x2k−1 − g) =gn· [x20 − x2n + 2λn∑k=1xk−1ξk +n∑k=1(ξ2k − 1)],it is easy to prove thatEλ(gn − g)2 ≤ C0n, n ≥ 1. (15)Further, similar to [9] using the Chebyshev inequality we estimatePλ(|gn| < h) ≤ Pλ(|gn − g| > g − h) ≤ Eλ(gn − g)2(g − h)2 ≤C0(1− h)2n, n ≥ 1. (16)Using (1316), we estimateEλ(λ˜− λ)2 ≤ 1h2Eλf2n + Pλ(|gn| < h) ≤I−1(λ, σ)h2n+Eλ(gn − g)2(g − h)2 ≤Cnand obtain assertion (7).223Novosibirsk, 18-20 September, 20195.2 Proof of Lemma 1The proof of Lemma 1 is similar to the proof of the second assertion of Theorem 1in [9].Definition (10) of λn impliesEλ(λn − λ)2 ≤ Eλ(λ˘n − λ)2.Introduce the following notationsfn =1nn∑k=2yk−2δk, gn =1nn∑k=1yk−1yk−2, g =λ1− λ2 , hn = (log n)−1.By the definition of λ˘ in (11), its deviation has the formλ˘n − λ = fngn· χ(|gn| ≥ hn)− λ · χ(|gn| < hn) = fng· χ(|gn| ≥ hn)+fn(g − gn)ggn· χ(|gn| ≥ hn)− λ · χ(|gn| < hn) = J1 + J2 + J3.Using the Cauchy-Schwarz-Bunyakovsky and Chebyshev's inequalities, estimatethe second moments of these summands:EλJ21 ≤ CEλf 2n, EλJ22 ≤1g2h2n√Eλf 4nEλ(gn − g)4, EλJ23 ≤ h−4n Eλ(gn − g)4.In view of the structure of the function fn it is easy to verify that Eλf 4n ≤ C/n2.By the definition of gn we havegn − g = 1nn∑k=1yk−1yk−2 − λ1− λ2 =1nn∑k=1xk−1xk−2 − λ1− λ2+1nn∑k=1xk−1ηk−2 +1nn∑k=1ηk−1xk−2 +1nn∑k=1ηk−1ηk−2 = λ(1nn∑k=1x2k−2 −11− λ2)+1nn∑k=1xk−1ηk−2 +1nn∑k=1(ηk−1 + ξk−1)xk−2 +1nn∑k=1ηk−1ηk−2.Using this representation, it is easy to verify similarly the proof of Theorem 1thatEλ(gn − g)4 ≤ Cn2.Thus we haveEλJ21 ≤ C1n, EλJ22 ≤ Clog2 nn2, EλJ23 ≤ Clog4 nn2.224Applied Methods of Statistical Analysis5.3 Proof of Theorem 2Introduce the following notations∆n = (λ2 − λ2n)λ1− λ2 + (1− λ2n)λ11− λ2{1n(x20 − λ2x2k−1) +2λnn∑k=2xk−2ξk−1+1nn∑k=2(ξ2k−1 − 1)−1n}+ (1− λ2n)1nn∑k=1[λxk−1ηk−1 + yk−1(ξk + ηk)].Definition (12) of the estimator λ∗n and equation (3) implyλ∗n = (1− λ2n)1nn∑k=1[λy2k−1 + yk−1(ξk + ηk)− λyk−1ηk−1]= (1− λ2n){λ1nn∑k=1x2k−1 +1nn∑k=1[λxk−1ηk−1 + yk−1(ξk + ηk)]}= (1− λ2n)λ1− λ2 + (1− λ2n)λ[1nn∑k=1x2k−1 −11− λ2]+(1− λ2n)1nn∑k=1[λxk−1ηk−1 + yk−1(ξk + ηk)] = λ+ (λ2 − λ2n)λ1− λ2+(1− λ2n)λ11− λ2{1n(x20 − λ2x2k−1) +2λnn∑k=2xk−2ξk−1 +1nn∑k=2(ξ2k−1 − 1)−1n}+(1− λ2n)1nn∑k=1[λxk−1ηk−1 + yk−1(ξk + ηk)] = λ+ ∆n.Thus the mean square deviation of the estimator λ∗n has the following formEλ(λ∗n − λ)2 = Eλ∆2n · χ(λ = 0) + Eλ∆2n · χ(λ 6= 0) =: I1 + I2,whereI1 = Eλ((1− λ2n)1nn∑k=1[yk−1(ξk + ηk)])2 · χ(λ = 0)= Eλ((1− λ2n)1nn∑k=1(ξk−1 + ηk−1)(ξk + ηk))2 · χ(λ = 0),I2 = Eλ∆2n · χ(λ 6= 0).From assumptions of Theorem 2 it follows I1 ≤ C/n. In view of Lemma 1 and theproperty |λn + λ| ≤ 2, we haveI2 ≤ Eλ(2|λn − λ|1− λ2 χ(λ 6= 0) +11− λ2{1n(x20 + x2k−1) +2n∣∣∣∣∣n∑k=2xk−2ξk−1∣∣∣∣∣+1n∣∣∣∣∣n∑k=2(ξ2k−1 − 1)∣∣∣∣∣+ 1n}+1n∣∣∣∣∣n∑k=1xk−1ηk−1∣∣∣∣∣+ 1n∣∣∣∣∣n∑k=1yk−1(ξk + ηk)∣∣∣∣∣)2≤ CEλ(λn − λ)2χ(λ 6= 0) + Cn+Cn2≤ Cn+ Clog4 nn2.225Novosibirsk, 18-20 September, 2019AcknowledgmentThis study is supported by the RSF (project 17-11-01049).References[1] Dogadova, T. V., and V. A. Vasiliev. 2014. Guaranteed parameter estimationof stochastic linear regression by sample of fixed size. Tomsk State UniversityJournal of Control and Computer Science. Vol. 26(1), pp. 3952.[2] Tatiana V. Dogadova, Marat I. Kusainov and Vyacheslav V. Vasiliev. 2017. Trun-cated estimation method and applications. Serdica. Mathematical Journal Bul-garian Academy of Sciences Institute of Mathematics and Informatics. Vol. 43,pp. 221266.[3] Kusainov, M. I., and V. A. Vasiliev. 2015. On optimal adaptive prediction of mul-tivariate autoregression. Sequential Analysis: Design Methods and Applications.Vol. 35(2), pp. 211234.[4] Mikulski, P. W., and M. J. Monsour. 1991. Optimality of the maximum likelihoodestimator in first-order autoregressive processes. Journal of Time Series Analysis.Vol. 12(3), pp. 237253.[5] Roll, J., A. Nazin, and L. Ljung. 2002. A Non-Asymptotic Ap-proach to Local Modelling. The 41st IEEE CDC, Las Vegas, Nevada,USA, 10-13 Dec. 2002 (Regular paper). Predocumentation is available athttp://www.control.isy.liu.se/research/reports/2003/2482.pdf[6] Roll J, Nazin A, Ljung L. 2005. Non-linear System IdentificationVia Direct Weight Optimization. Automatica, Journal of IFAC.Special Issue on "Data-Based Modelling and System Identifica-tion". Vol. 41(3), pp. 475490. Predocumentation is available athttp://www.control.isy.liu.se/research/reports/2005/2696.pdf[7] Shneeweiss, H. 1976. Consistent estimation of a regression with error in thevariables. Metrika. Physica-Verlag, Wien. Vol. 23, pp. 101115.[8] Shiryaev, A. N., and V. G. Spokoiny. 2000. Statistical Experiments and Deci-sions. Asymptotic Theory. Singapore, New Jersey, London, Hong Kong: WorldScientific.[9] Vasiliev, V. 2014. A Truncated Estimation Method with Guaranteed Accuracy.Annals of Institute of Statistical Mathematics. Vol. 66, pp. 141163.[10] Vasiliev V. A., and V. V. Konev 1982. The Sequential Parameters Estimationof Dynamic Systems with Noisy Observations. Izvestia Acad. USSR, TechnicalCybernetics. Vol. 6, pp. 145154.226

Parameter estimation with guaranteed accuracy for AR(1) by noised observations

http://vital.lib.tsu.ru/vital/access/services/Download/vtls:000721552/SOURCE1

Parameter estimation with guaranteed accuracy for AR(1) by noised observations

Abstract

Similar works

Full text

Available Versions

Tomsk State University Repository