Missing values in time series can be treated as unknown parameters and estimated by maximum likelihood, or as random variables and predicted by the expectation of the unknown values given the data. The difference between these two procedures is illustrated by an example. It is argued that the second procedure is, in general, more relevant for estimating missing values in time series.ARIMA models; Interpolation; Mean Square Error;

Peña, Daniel

Tiao, George C.

English

Research Papers in Economics

Working Paper 91  - 01  Departamento de Economía 
February 1991  Universidad Carlos III de Madrid 
Calle Madrid, 126 
28903  Getafe (Spain) 
e 
e 
A NOTE ON LIKELIHOOD ESTIMATION OF
 
MISSING VALUES  IN  TIME SERIES
 
Daniel Peña and George C.  Tiao* 
Abstract  .  _ 
Missing  values  in  time series  can be treated as  unknown parameters and estimated 
by  maximum likelihood, or as  random variables and predicted by the expectation of the 
unknown values given the data. The difference between these two procedures is illustrated 
by an example.  It is  argued that the second  procedure is,  in  general, more relevant for 
estimating missing values in time series. 
Key Words
 
ARIMA models; interpolation; Mean Square Error.
 
*  Daniel Peña is Proíessor of Statistics, Universidad Carlos III de Madrid and Laboratorio 
de Estadística~ ETSlI, Universidad Politécnica de Madrid, and George C. Tiao is W. Allen 
Wallis Professor of Statistics, Graduate School oí Business, University of Chicago. ( 
The use of the maximum likelihood method to estimate missing observations 01', in  gen­
eral, unobserved values of random variables is a controversial topic because different authors 
use different likelihoods to obtain the estimators (see  Bayarri, DeGroot and Kadane, 1986 
and the discussion of the papel', and Fuller, 1988). 
To illustrate the problem in  time series models. suppose that the time series  {Zt} follo\\'s 
the stationary first order autoregressive process 
191  < 1, 
where the at's are i.i.d.  N(O.  ( 2).  For simplicity, let us  assume that 4J  and a 2  are knowll. 
Suppose that out ofn observations Zt,  t =  L ... ,n, the observation ZT is  missing. 1 :S T  ::;  n. 
Then, denoting Zn as the n  x  1 vector Zn  =  (ZI"'"  zn)' and Z(T) as the (n - 1) x 1 vector 
obtained from  Zn  by dropping  ZT.  the joint density function of the available data Z(T) for 
given ZT is: 
e  (1) 
where: 
and 
(3) 
In (1),  ZT  is  now  an unknown parameter of the model for  Z(T)'  The likelihood function 
of ZT can be written as 
where it is  understood that in the exponent a term disappears if the range of summatioll is 
not positive. 
Hence, the maximum likelihood estimator of ZT takes the form: 
(5) 
1 e
 
with {;  =  1 if T  =  1 and {;  = -1 if T =  n.  It is  easy to verify from (4) that the mean sCluare 
error (MSE) of T  is: 
(6) 
Several  authors  have  computed  least  squares or  "maximum likelihood"  estimators  by 
maximizing the joint density function  f(Zn)  in  (2)  with  respect  to  missing  observations 
e  for  known data.  (See, for instance, Brubacher and Tunniclife-Wilson,  1977).  Then, in  the 
present case, it is  easy to verify that the estimator is  given by 
T = 1 or n 
(7) 
l<T<n e 
However, this estimator Zt  cannot be called a maximum likelihood estimator because the 
function  f(Zn) for  ZT  considered as an unknown parameter is  not a joint density function, 
and, therefore, f(Zn) for unobserved ZT  and known  Z(T)  is  not a likelihood fundion as it is 
usually defined in standard texts. 
To  interpret the meaning 01'  (7).  [et  us  consider  ZT  as  a  random  \'ariable I'ollowing  the 
probabilistic structure in  (2).  Then, the distribution of ZT  given the data Z(T)  is: 
(8) 
where f(Z(T))  can be obtained by integrating out ZT  from f(Zn)'  As  is  ",ell known (see e.g. 
Peña (1987))  the distribution in  (8)  is  normal with: 
T = 1 or n  (9) 
1 < T  < n. 
'vVe  see that the conditional expectation, E(ZTIZ(T)), is  equal to ZT  in (7); this is  because 
f(Zn)  is  proportional to f(ZTIZ(T))  and, for  the present example, the mode and  the mean 
of the distribution f(ZTIZ(T))  are identical.  1'vlore  important, \Ve  see that E(ZTIZ(T)),  which 
is  the minimum !vlSE estimator of ZT.  can  be very  different from the maximum likelihood 
estimator ZT in (5).  Indeed, the !vlSE of::T  in (6) always exceeds, and can be very much larger 
than, Var(zTIZ(T)) in (9), which is, of course, also the :rv1SE of the estimator E(ZTIZ(T))' 
2 
---------_.~_._..._---_. The difference  between the  two  estimators.  ZT  and  ZT,  is  not surprising if  we  look  at 
the problern fro\11  a  Bayesian  point of view.  The estimator  ZT  is  the mean  (01'  mode)  of 
the  posterior  distribution  f(ZTIZ(T))  in  (8).  v.·hich  is  proportional  to  the  product  of the 
likelihood function  (zTIZ(T))  in  (4)  and the prior distribution f(ZT)  in  (:3).  On  the other 
hand, the estimator ZT  can be regarded as the mean  (01'  mode) of a  posterior distribution 
of ZT  proportional to  the product (zTIZ(T))pO(ZT),  where PO(ZT)  is  a  "Iocally  uniform"  01' 
e  noninformative prior distribution (Box and Tiao, 1973).  Thus, in the stationary case,  14>1  < 1, 
the two means can be very different because very different prior distributions are employed. 
This also explains the fact  that when  <ti  goes  to  1 (the model approaches a  non§tationary 
one) the difference between these two estimators goes to zero simply because in this case the 
e  prior distribution f( ZT)  also becomes nearly locally uniformo 
It may be argued from a frequentist point 01' view that the optimal properties of Zr and ZT 
in  (6) and (9)  respectively are not really comparable, because they are obtained under very 
different assumptions.  For the maximum likelihood estimator ZT  the unkno\\'n observation 
ZT  is  regarded as  a fixed  pararneter, and the  ~ISE(ZT) in  (6)  is  obtained under  (01' at least 
motivated by) such an assumption: while 1'01'  the estimator ZT,  the lvISE(zT)  =  Var(zTIZ(T)) 
in  (9)  is  obtained when  ZT  is  regarded as  random follo",ing  the structure in  (2).  Indeed, it 
e  can be verified from (4)  that, 1'01' fixed  ZT,  the ~lSE 01'  ZT  is: 
T = 1 01'  11 
(10) 
l<T<n 
e 
so that 1'01'  sorne values 01' ZT,  !\lSe(.:T) can  be larger than MSE(ZT)' 
The point 01'  the above discussion is  to show  that in  estimating missing values in  time 
series,  the  method  01'  maximum likelihood  can  lead  to  results  very  different  from  those 
obtained by optimal prediction under stationary assumptions.  Except 1'01'  the initial value 
at t  = 0,  we  do  not  think,  however,  that it  is  appropriate to  treat  missing  observatiolls 
as fixed  parameters.  This seems almost a  contradiction in  terms.  In  time series  analysis 
we  believe it is  natural in most applications to regard the missing observations as random 
variables following  the same probabilistic structure as  the remaining ones, and hence adopt 
e  the conditional expectation 01' posterior mean as their optima! estimator. 
3 
e REFERENCES 
•￿  Bayarri, M.J., DeCroot, M.H. and Kadane, J.B. (1986), "What is  the likelihood func-
tion?", in Proceedings of the Fourth Purdue Symposium on Statistical Decision  Theory 
and Related Topics, eds.  S.S. Cupta and J. Berger, Springer-Verlag. 
•￿  Box, C.E.P. and Tiao, C.C. (1973), Bayesian [nference in Statisf1:cal Analysis, Addison-
Wesley. 
•￿  Brubacher. S.R. y Tunnicliffe-Wilson, C. (1976),  "Interpolating Time Series-with  Ap-
plication to the Estimation of Holiday Effects on Electricity Demand", Appl.  Statist., 
25,2, 107-116. 
•￿  Fuller, \V.A. (1987),  Measunment EITOTS Models, John \Viley and Sonso 
•￿  Peña, D.  (198/),  "Measuring the importance of outliers in  ARHvlA  models", in  .\'ew  e 
Perspeetivts in  Theoretical and Appliul Statistics, eds.  rv1.L.  Purí et al, Wíley, 109-118. 
e 
(, 
(: 
4 

A Note on likelihood estimation of missing values in time series.

http://e-archivo.uc3m.es/bitstream/10016/2748/1/we9101.pdf

A Note on likelihood estimation of missing values in time series.

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics