The paper addresses the problem of estimating missing observations in linear, possibly nonstationary, stochastic processes when the model is known. The general case of any possible distribution of missing observations in the time series is considered, and analytical expressions for the optimal estimators and their associated mean squared errors are obtained. These expressions involve solely the elements of the inverse or dual autocorrelation function of the series.
This optimal estimator -the conditional expectation of the missing observations given the available ones-is equal oto the estimator that results from filling the missing values in the series with arbitrary numbers, treating these numbers as additive outliers, and removing the outlier effects from the invented numbers using intervention analysis

Maravall, Agustín

Peña, Daniel

English

Universidad Carlos III de Madrid e-Archivo

Working Paper 91-08 Departamento de Econom1a February 1990 Universidad Carlos III de Madrid Calle Madrid, 126 28903 Getafe (Madrid) e INTERPOLATION, OUTLIERS AND INVERSE AUTOCORRELATIONS Daniel Peña* and Agust1n Maravall** Abstract _ The paper addresses the problem of estimating missing observations in linear, possibly nonstationary, stochastic processes when the model is known. The general case of any possible distribution of missing observations in the time series is considered, and analytical expressions for the optimal estimators and their associated mean squared errors are obtained. These expressions involve solely the elements of the inverse or dual autocorrelation function of the series. This optimal estimator -the conditional expectation of the missing observations given the available ones- is equal oto the estimator that results from filling the missing values in the series with arbitrary numbers, treating these numbers as additive outliers, and removing the outlier effects from the invented numbers using intervention analysis. Key words Missing observations; Outliers, Intervention analysis; ARIMA models; Inverse autocorrelation function. * Departamento de Econom1a, Universidad Carlos III de Madrid and Laboratorio de Estad1stica, ETSII, Universidad Politécnica de Madrid. ** European University Institute, Firenze, Italy. 1. INTRODUCTION In this paper we consider the problem of estimating missing observations in linear, possibly nonstationary, stochastic processes, which we parametrize as parsimonious Autoregressive Integrated Moving Average (ARIMA) models. The model is assumed to be known, and hence we concern ourselves with obtaining analytical expressions for the conditional expectation of the missing observations given the available ones. In terms of the well-known EM algorithm, our aim is to obtain further insights into the E -the expectation- step. In section 2 we present the model, the basic assumptions and sorne background references. Section 3 contains the general result of the paper for the case of any possible distribution of missing observations in the series. Analytical expressions for the optimal estimator of the missing observations and the associated Mean Squared Error (MSE) are obtained, and the relationship with additive outlier models is discussed. section 4 presents sorne conclusions. 2. THE MODEL AND SOME BACKGROUND RESULTS Let the series Zt follow the general ARIMA model (2.1) where ~(B) and e(B) are finite polynomials in the lag operator B, and a t is a Gaussian white-noise process with variance (72 a . The autoregressive (AR) polynomial ~(B) contains the stationary as well as the nonstationary roots; the moving average (MA) polynomial e(B) is assumed to be invertible and hence, if n(B) (= 1-n1B -n2B2 - ••• ) denotes the convergent polynomial ~ (B) e (B) -1, model (2.1) can be expressed as the pure autoregression Def ine the inverse or dual roodel of (2.1) as the one that results froro interchanging the AR and MA polyno-roials; that is (2.2) or, equivalently, Since (2.1) is invertible, (2.2) will be stationary; its variance (vo) and autocovariance generating function (ro(B)) are given by (2.3) .. rD(B) = VD + Eri (Bi+F i ) = o: 1t (B)1t (F) I (2.4) i m l where F = B-1 is the forward operator. Then, the dual autocovariance at lag i will be given by (2.5) Following Cleveland (1972), the function (2.6) will be denoted the Inverse or Dual Autocorrelation Generating Function (DAGF). (Trivially, froro the ARlMA expression of the roodel, the DAGF is immediately availa-ble. ) 3 Assume the series Zt has a single missing value for t = To The minimum MSE estimator of zT (the conditional expectation of zT given the available observations) is a linear combination of the observed values, where the weights depend on the covariance structure of the processo Several authors have shown how to compute the estimator recursively using the Kalman filter (Jones, 1980; Harvey and Pierse, 1984; Kohn and Ansley, 1983) o Wincek and Reinsel (1986), and Ljung (1982, 1989) have concentrated on the explicit form of the likelihood function and the maximum likelihood estimates of the missing valueso The analytical expression for the conditional expectation of a missing value in a stationary stochastic process has been available for some time (Grenander and Rosenblatt, 1957); it can be expres-sed as 2T = - L pf (ZT-i+ZT+i) (207) i-l where PiD is the lag-i dual autocorrelation of Zto Moreo-ver, the MSE of the estimator is found to be ... MSE(2T) = a~/v;l = a:/E 1t~ I J-O where VD is given by (203) o Brubacher and Wilson (1976) obtained (2.7) by least squares for a seasonal nonstationary ARMA process and showed the relation between interpolation and the inverse autocorrelation functiono Kato (1984) showed that the inverse autocorrelation function at lag K is equal to the negative of the partial correlation between Zt and Zt+k after elimination of the influence of zh' (h"t, t+k) o Battaglia and Bhansali (1987) used the interpolation error variance to build an index of linear determinismo 4 .. -~---------------e e e e ," '-Pourahmadi (1989) has studied the estimation and interpolation of several missing values of a stationary time series. Finally, Bruce and Martin (1989) and Peña (1987, 1990) have shown the relationship between interpolation and additive outlier estimation. Focussing on this last extension, if the series has an additive outlier at period T, so that the observation for that period is (zT+w) where w is the (unknown) outlier effect, then the optimal estimator of w is given by (see Chang, Tiao and Chen, 1988) (2.8) where Z denotes the observed series such that ZT = zT+w, and Zt = Zt for t f T. Thus, back to the missing observa-tion case, if the "hole" in the series is filled with an arbitrary number ZT and this "invented" observation is treated as an outlier, the missing observation estimator can be obtained through (2.9) and it is easily seen that (2.8) and (2.9) yield express ion (2.7). Furthermore, since ZT-2T = Q-w, i t follows that M5E(Q) is also equal to Ga 4 /vD• Therefore, when the model is known, optimal estimation of an additive outlier is equivalent to the following procedure: First, assume the outlier is a missing observation and obtain its optimal estimate given the rest of the observations. Then, compute the outlier effect as the difference between the outlier and the interpolated value. Alternatively, estimation of a missing observation can be seen as the resul t of the following procedure: First, fill the "hole" in the series with an arbitrary number. Then treat this number as an additive outlier, and remove the outlier effect with intervention analysis. 5 3. TBE GENERAL CASE e Assume that, in all generality, the finite series Zt has k missing values at periods TI' T2, ••• , Tk , were Ti < Tj if i < j. We can always fill the k holes in the series with arbitrary numbers, ZT.' and construct an "observed"e .serJ.es Zt by ~ t = T1 , ••• , Tx otherwise. e where wt is an unknown parameter. In matrix notation, we can write Z = z+Hw (3.1) e where Z and Z are the series Zt and Zt expressed as vectors, H is a matrix with k columns such that HT.,j = 1, for j = 1, .•. , k, and Hij = 0, otherwise, and w\s a k-dimensional vector with elements Wj(= ZT. - zT.)' Let ~ be the covariance matrix of the series JZt ; t~en the generalized least squares estimator of w is given by (3.2) Treating the starting values as fixed constants, (Ansley, 1979), the Cholesky factorization of ~ leads to where n is a lower triangular matrix with the -nj's on the various lower diagonals. Then we can write (3.2) as ~ = (X/X) -1 X/Y, (3.3) where X = nH, Y = nZ. Expression (3.3) can alternatively be obtained as jfollows: Let dt , j = 1, ... , k, be a set of dummy varia-jbles such that dt = 1 for t = Tj and zero otherwise. Then, from (2.2), the following intervention model (Box and Tiao, 1975) is obtained 6 e (3.4) ( which can be rewritten as (3.5) e e where Yt = n(B)Zt, Xjt = n(B)dt j . Expression (3.5) is a regression equation with Xjt deterministic and a t white-noise; i t is immediately seen that the minimum MSE estimator of w is then given by (3.3). Expressions (3.2) and (3.3) apply to a finite series . Now, let us consider the case of a ser ies of infinite length. The results obtained in this case will be approximately exact when the sample size is large and the missing values are not near the end of the series. Using (2.3), (2.5) and (2.6), straightforward computation yields ( (3.6) ~ X;j = (vD/a:) =E 1t~ izo (3.7) ~ X jt X j +h • t = -1th + ..E 1t i 1t i +h ~zJ. =~/a:, (3.8) where r h D denotes the lag-h autocovariance of the dual process defined in (2.5). Using (3.7) and (3.8), XIX is a (kxk) symmetric matrix with the (i,j)-th element given by rD T._T.. The elements are thus autocovariances of the dual p~ocJss for lags reflecting the relative distances between the missing observations. Therefore, (3.9) 7 e where RD is the (kxk) symmetric matrix e D D D1 PT2 -T1 PT3-T1 PTk-T1 D D1 PT3-T2 PTK-T2 1 (3.10 )RD = e 1 ( with elements the dual autocorrelations of the process. If Zk denotes the vector of invented observations ZT , ... , ZT)' , from (3 . 3) , ( 3 . 6) and (3 . 9) , the 1 . kest1mator of w becomes ( (3.11) and the vector of missing observations estimators 2k = (2T , ... , 2T )' can then be obtained through 1 ke Zk = Zk "- w. (3.12) Equations (3.11) and (3.12) are the generalization of equations (2.8) and (2.9), derived for the single missing observation, to the case of any possible sequence of missing observations. The estimator 2k can be seen as the result of the following procedure: First, fill the holes in the series with arbitrary numbers, which then are treated as additive outliers. Removing from the arbitrary numbers the outlier effects, the missing observation estimator is obtained. It is straightforward to show that the estimator 2k does not depend on the vector Zk of arbitrary numbers. As for the MSE of the estimator, from (3.11) and noticing that ZT. - 2T . = ~j - Wj' it follows that J J 8 ( c: Therefore, both the estimator and its MSE can be expressed in terms of the dual autocorelations of the series. It is irnmediately seen that, for the scalar case, the matrix Ro of (3.10) becomes simply 1 and equation (2.8) is obtained. More relevantly, for the case of k consecutive missing observations, Ro becomes the symmetric Toeplitz matrixe D1 pf p~ Pk-l D1 pf Pk-2 ( RD = pf 1 ( that is, the (kxk) dual autocorrelation matrix. The MSE of the estimator is then the inverse of the (kxk) autocovariance matrix of the dual process, also a syrnmetric Toeplitz matrix with all the elements of the i-th diagonal equal to rio. Finally, from equation (3.11) another interesting express ion for ~k is obtained. Let Wj (1) denote the estimator of Wj obtained by assuming that, in the vector Zk, only the element ZT. is invented, and using the method of section 2 forJ the scalar case. Define the vector w(l) = (w1(1) , ... , wk(l) '; from (2.8), express ion (3.11) can be rewritten R-1 W(l).WA = D Therefore, for the case of a vector of missing observa-tions, the optimal estimator can be seen as a weighted average of the estimators obtained by treating each missing observation as if it were the only missing one (using (2.8) on the arbitrarily filled series). The 9 _._---------_._---------------_.  e e weights are the elements autocorrelation matrix. of the inverse dual e 4. CONCLUSION ( We have considered the general case of any distribution of missing observations in an infinite realization a possibly nonstationary linear stochastic process. Compact analytical expressions for the optimal estimator of the missing observations are derived, which explicitly show the dependence of the conditional expectation on the stochastic structure of the series, and involve only the elements of i ts inverse or dual autocorrelation function. Furthermore, analytical expressions for the Mean Squared Error matrix are immediately obtained, which are simply functions of the dual autocovariance matrix. The expression for the conditional expectation of the missing observations given the available ones is the same that results from filling the missing values with arbitrary numbers, treating t:hese numbers as additive outliers, and removing the outlier effects through intervention analysis from the invented observations. 10 ------------------------------------------_. e BIBLIOGRAPBYe Ansley, C.F. (1979). 'An algorithm for the exact likelihood of a mixed autoregressive-moving average process,' Biometrika, 66, 1, 59-65. Battaglia, F. and Bhansali, R.J. (1987). 'Estimation ofe the interpolation error variance and an index of linear determinism,' Biometrika, 74, 771-779. Box, G.E.P. and Tiao, G.C. (1975). 'Intervention Analysis with applications to economic and environmental pro-blems, ' Journal of the American Statistical Association, 70, 70-79. Brubacher, S.R. and Tunnicliffe-Wilson, G. (1976). 'Interpolating time series with application to the estimation of holiday effects on electricity demand, , Applied Statistics, 25, 2, 107-116.e Bruce, A.G. and Martin, R.D. (1989). 'Leave-k-out diagnostics for time seri.es, , Journal of the Royal Statistical Society B, 51, 3, 363-424. Chang, l., Tiao, G.C. and Chen Ch. (1988). 'Estimation of( time series parameters in the presence of outliers, ' Technometrics, 30, 2, 193-204. Cleveland, W.P. (1972). 'The inverse autocorrelations of a time series and their applications, , Technometrics, 14, 277-293. ",._-Grenander, U. and Rosenblatt, M. (1957). Statistical Analysis of Stationary Time Series, New York: J. Wiley. Harvey, A.C. and Pierse, R.G. (1984). 'Estimating missing observations in economic time series,' Journal of the American Statistical Association, 79, 125-132. Jones, R.H. (1980). 'Maximum likelihood fitting of ARMA models to time series with missing observations,' Technometrics, 22, 3, 389-395. Kato, A.J. (1984). 'A characterization of the inverse autocorrelation function,' Communications in Statistics, Theory and Methods, 13, 20, 2503-2510. Khon, R. and Ansley, C.F. (1983). 'Fixed interval estimation in state space models when some of the data are missing or aggregated,' Biometrika, 70,3, 683-8. 11 ----------------,------------------( Ljung, G.M. (1982). 'The likelihood function for a stationary Gaussian autoregressive-moving average e process with missing observations,' Biometrika, 69, 1, 265-8. Ljung, G.M. (1989). 'A note on the estimation of missing values in time series,' Communication in Statistics, Simulation and Computation, 18, 2, 459-465. e Peña, O. (1987). 'Measuring the importance of outliers in ARlMA models,' in New Perspectives in Theoretical and Applied Statistics, eds. M.L. Puri et al, Wiley, 109 - 118. Peña, O. (1990). 'Influential observations in time series, , Journal of Business and Economic Statistics, 8, 2, 235-241. Pourahmadi, M. (1989). 'Estimation and interpolation of missing values of a stationary time series,' Journal e of Time Series Analysis, 10, 2, 149-169. Wincek, M.A. and Reinsel, G.C. (1986). 'An exact maximum likelihood estimation procedure for regression-ARMA time series models with possibly nonconsecutive data,' Journal of the Royal Statistical Society, 48, e 3, 303-313. ACKNOWLEDGEMENTS Thanks are due to Professor Gregory Reinsel and to two anonymous referees for helpful comments. The first author acknowledges support from OGICYT, project PB86-0538. 12 

A characterization of the inverse autocorrelation function,'

A note on the estimation of missing values in time series,'

An algorithm for the exact likelihood of a mixed autoregressive-moving average process,'

An exact maximum likelihood estimation procedure for regression-ARMA time series models with possibly nonconsecutive data,'

Estimating missing observations in economic time series,'

Estimation and interpolation of missing values of a stationary time series,'

Estimation of( time series parameters in the presence of outliers, '

Estimation ofe the interpolation error variance and an index of linear determinism,'

Fixed interval estimation in state space models when some of the data are missing or aggregated,'

Influential observations in time series,

Interpolating time series with application to the estimation of holiday effects on electricity demand,

Intervention Analysis with applications to economic and environmental problems,

Maximum likelihood fitting of ARMA models to time series with missing observations,'

Measuring the importance of outliers in ARlMA models,'

The inverse autocorrelations of a time series and their applications,

The likelihood function for a stationary Gaussian autoregressive-moving average e process with missing observations,'

Interpolation, outliers and inverse autocorrelations

The paper addresses the problem of estimating missing observations in an infinite realization of a linear, possibly nonstationary, stochastic processes when the model is known.  The general case of any possible distribution of missing observations in the time series is considered, and analytical expressions for the optimal estimators and their associated mean squared errors are obtained.  These expressions involve solely the elements of the inverse or dual autocorrelation function of the series. This optimal estimator -the conditional expectation of the missing observations given the available ones- is equal to the estimator that results from filling the missing values in the series with arbitrary numbers, treating these numbers as additive outliers, and removing with intervention analysis the outlier effects from the invented numbers

Interpolation, outliers and inverse autocorrelations

Abstract

Similar works

Full text

Available Versions

Universidad Carlos III de Madrid e-Archivo

Cadmus, EUI Research Repository

Name not available

Name not available