In this note we analyze the relationship between one-step ahead prediction errors and interpolation errors in time series. We obtain an expression of the prediction errors in terms of the interpolation errors and then we show that minimizing the sum of squares of the one step-ahead standardized prediction errors is equivalent to minimizing the sum of squares of standardized interpolation errors.

Daniel Peña

Pedro Galeano

English

Research Papers in Economics

 
 
Working Paper  04-27 
Statistics and Econometrics Series 10 
September 2004 
 
Departamento de Estadística
Universidad Carlos III de Madrid
Calle Madrid, 126
28903 Getafe (Spain)
Fax (34) 91 624-98-49
 
 
 A NOTE ON PREDICTION AND INTERPOLATION ERRORS IN TIME SERIES 
 
   Pedro Galeano and Daniel Peña* 
 
 
Abstract 
 
In this note we analyze the relationship between one-step ahead prediction errors and 
interpolation errors in time series. We obtain an expression of the prediction errors in terms 
of the interpolation errors and then we show that minimizing the sum of squares of the one 
step-ahead standardized prediction errors is equivalent to minimizing the sum of squares of 
standardized interpolation errors. 
 
 
Keywords: Fixed-Point Smoothing; Interpolation Error; Kalman Filter; Prediction Error. 
 
 
 
 
 
 
* Galeano, Departamento de Estadística, Universidad Carlos III de Madrid, c/Madrid, 126, 
28903 Getafe (Madrid), e-mail:pedro.galeano@uc3m.es. Peña, Departamento de Estadística, 
Universidad Carlos III de Madrid, c/Madrid, 126, 28903 Getafe (Madrid), e-
mail:daniel.pena@uc3m.es. We acknowledge financial support from BEC2000-0167, MCYT, 
Spain. 
 A Note on Prediction and Interpolation Errors in Time Series
Pedro Galeano, Daniel Pe~ na¤
Departamento de Estad¶ ³stica, Universidad Carlos III de Madrid,
C/ Madrid 126, Getafe, 28903 Madrid, Spain
Abstract
In this note we analyze the relationship between one-step ahead prediction errors and interpolation
errors in time series. We obtain an expression of the prediction errors in terms of the interpolation errors
and then we show that minimizing the sum of squares of the one step-ahead standardized prediction
errors is equivalent to minimizing the sum of squares of standardized interpolation errors.
Keywords: Fixed-Point Smoothing; Interpolation Error; Kalman Filter; Prediction Error.
1 Introduction
It is well known that the likelihood function of an ARMA(p;q) process can be written in terms of the one step
ahead prediction errors using the conditional distribution of each observation given the previous data. This
is called the prediction error decomposition. The Maximum Likelihood Estimate (MLE) of the parameters
can be computed by minimizing the concentrated likelihood function, which depends on the one-step ahead
prediction errors. The interpolation problem consists in the estimation of a missing observation by using the
past and future values of the time series. The interpolator which minimizes the mean squared error criterion
is computed by the expected value of the observation given the rest of the sample. The interpolation error
is the di®erence between the interpolated value and the true value of the observation. In the state-space
form of ARMA models, the interpolator is obtained with some smoothing algorithm, such as the ¯xed point
smoother (FPS) (see Anderson and Moore, 1979).
The aim of this note is to show the relationship between prediction errors and interpolation errors and
to prove that the parameter values which minimize the mean squared prediction error are the same as those
¤Corresponding author.
E-mail address: daniel.pena@uc3m.es (D. Pe~ na)
1which minimize the mean squared interpolation errors. This note is organized as follows. In section 2 we
introduce the notation and brie°y review the Fixed Point Smoothing algorithm. In section 3, we ¯rst obtain
an expression of the one step ahead prediction error in terms of the interpolation errors, second we derive
the covariances between interpolation errors and third we show that minimizing the sum of squares of the
one step-ahead standardized prediction errors leads to the same result than minimizing the sum of squares
of the standardized interpolation errors. Section 4 illustrates the result in the simplest case of a ¯rst order
autoregresssive process.
2 Kalman Filter and ¯xed point smoothing
Let fztg be a process following a zero mean stationary and invertible ARMA(p;q) model,
Á(B)zt = µ(B)ut; (1)
where Á(B) = 1 ¡ Á1B ¡ ::: ¡ ÁpBp, µ(B) = 1 ¡ µ1B ¡ ::: ¡ µqBq and futg is a sequence of independent
N(0;¾2) variables. We denote the vector of ARMA parameters in (1) by ¯ =
¡
Á1;:::;Áp;µ1;:::;µq
¢0
and a
sample generated by this process by z = (z1;:::;zT)
0, where T is the sample size. Let §z be the covariance
matrix of z, then the likelihood function is:
L(zj¯;¾2) = (2¼)
¡ T
2 j§zj
¡ 1
2 exp
µ
¡
z0§¡1
z z
2
¶
:
Let ztjt¡1 = E [ztjzt¡1;:::;z1] for t = 1;:::;T, be the one step ahead predictions obtained by minimizing
the mean squared errors, where z1j0 = E [z1], and let et = zt ¡ ztjt¡1 be the corresponding one step
ahead prediction errors with variances E
h¡
zt ¡ ztjt¡1
¢2i
= ¾2v2
tjt¡1, and where var(z1) = ¾2v2
1j0. The
log-likelihood, `
¡
zj¯;¾2¢
= logL
¡
zj¯;¾2¢
, can be written as:
`(zj¯;¾2) = ¡
T
2
log2¼¾2 ¡
1
2
T X
t=1
logv2
tjt¡1 ¡
1
2¾2
T X
t=1
e2
t
v2
tjt¡1
;
and the maximum likelihood estimate of ¾2 is given by,
b ¾
2
MLE =
1
T
T X
t=1
e2
t
v2
tjt¡1
; (2)
2and, using (2), the maximum likelihood estimate of ¯, b ¯MLE, maximizes the concentrated log-likelihood
given by,
S (¯) =
1
T
T X
t=1
logv2
tjt¡1 + log
Ã
T X
t=1
e2
t
v2
tjt¡1
!
:
The state-space representation for ARMA(p;q) models proposed by Jones (1980) is obtained by de¯ning
r = maxfp;q + 1g, with:
zt = H0xt;
xt = Fxt¡1 + Gut;
(3)
where H = (1;0;:::;0)
0, xt =
¡
zt;zt+1jt;:::;zt+r¡1jt
¢0
, G =
¡
1;Ã1;:::;Ãr¡1
¢0
, and:
F =
2
6 6
6 6
6
6 6
6 6
6
6
4
0 1 0 ¢¢¢ 0
. . .
. . .
. . .
...
. . .
. . .
. . .
. . .
. . . 0
0 0 0 ¢¢¢ 1
Ár Ár¡1 Ár¡2 ¢¢¢ Á1
3
7 7
7 7
7
7 7
7 7
7
7
5
where Ã(B) = Á(B)
¡1 µ(B) = 1 +
P1
i=1 ÃiBi, and zt+jjt = E [zt+jjzt;:::;z1] with E
h¡
zt+j ¡ zt+jjt
¢2i
=
¾2v2
t+jjt. With this representation, the Kalman Filter computes the log-likelihood through the recursions:
xtjt¡1 = Fxt¡1jt¡1
§tjt¡1 = F§t¡1jt¡1F0 + Q
Kt = §tjt¡1H
³
v2
tjt¡1
´¡1
xtjt = xtjt¡1 + Kt
¡
zt ¡ ztjt¡1
¢
§tjt = (I ¡ KtH0)§tjt¡1;
(4)
for t = 1;:::;T, where ztjt¡1 = H0xtjt¡1, v2
tjt¡1 = H0§tjt¡1H, Q = GG0 and,
xtjs = E [xtjz1;:::;zs]
¾2§tjs = cov [xtjz1;:::;zs]:
s;t = 1;:::;T
The initial conditions are x1j0 = x0j0 = 0 and ¾2§1j0 = ¾2§0j0 = cov(x0) and ¾2 is estimated with (2).
Suppose now that we want to interpolate the observation at time t = h. The interpolated value,
3E
£
zhjz(h)
¤
= z
(h)
hjT, where z(h) = fzi : i = 1;:::;T; i 6= hg is obtained in two steps. First, we assume that
the value zh is missing and compute the estimation of the state variables with the Kalman Filter under this
condition. Second, we compute the interpolated value by going backwards with the Fixed Point Smoothing
algorithm. The situation in which zh is not observed can be represented by the state-space model,
zt =
³
1 ¡ I
(h)
t
´
H0xt + I
(h)
t wt;
xt = Fxt¡1 + Gut;
(5)
where I
(h)
t is a dummy variable such that I
(h)
t = 0, t 6= h and I
(h)
h = 1 and wt represents independent N (0;1)
variables, independent of zh. The Kalman Filter applied to this situation is given by,
x
(h)
tjt¡1 = Fx
(h)
t¡1jt¡1
§
(h)
tjt¡1 = F§
(h)
t¡1jt¡1F0 + Q
K
(h)
t =
³
1 ¡ I
(h)
t
´
§
(h)
tjt¡1H
³
v
2;(h)
tjt¡1
´¡1
x
(h)
tjt = x
(h)
tjt¡1 + K
(h)
t
³
zt ¡ z
(h)
tjt¡1
´
§
(h)
tjt = (I ¡
³
1 ¡ I
(h)
t
´
K
(h)
t H0)§
(h)
tjt¡1;
(6)
for t = 1;:::;T, where z
(h)
tjt¡1 =
³
1 ¡ I
(h)
t
´
H0x
(h)
tjt¡1 + I
(h)
t wt, v
2;(h)
tjt¡1 =
³
1 ¡ I
(h)
t
´
H0§
(h)
tjt¡1H + I
(h)
t , and,
x
(h)
tjs = E [xtjz1;:::;zs]
§
(h)
tjs = cov [xtjz1;:::;zs]:
s;t = 1;:::;T
All the values have the subscript h in order to distinguish between the Kalman Filter with the observation
at t = h and without it. Of course, for t < h, x
(h)
tjt¡1 = xtjt¡1. Note that for t = h, v
2;(h)
hjh¡1 = 1.
Second, we use the Fixed Point Smoothing (FPS) algorithm to obtain the interpolated value, that can
be derived as follows. Consider the augmented process yt = [ x0
t xa0
t ]0, such that, xa
t = xa
t¡1 and xa
h = xh
with state-space form given by,
2
6
4
xt
xa
t
3
7
5 =
2
6
4
F 0
0 I
3
7
5
2
6
4
xt¡1
xa
t¡1
3
7
5 +
2
6
4
G
0
3
7
5ut
zt =
· ³
1 ¡ I
(h)
t
´
H0 0
¸
2
6
4
xt
xa
t
3
7
5 + I
(h)
t wt
4Now, applying the Kalman Filter to the augmented process yt, for t ¸ h, with the initial condition
§a
hjh¡1 = §
(h)
hjh¡1, the FPS works as follows, (see G¶ omez and Maravall (1994) for more details):
Ka
t =
³
1 ¡ I
(h)
t
´³
§a
tjt¡1
´0
H
³
v
2;(h)
tjt¡1
´¡1
x
(h)
hjt = x
(h)
tjt¡1 + Ka
t
³
zt ¡ z
(h)
tjt¡1
´
§
(h)
hjt = §
(h)
hjt¡1 ¡
³
1 ¡ I
(h)
t
´
§a
tjt¡1H (Ka
t )
0
§a
t+1jt = §a
tjt¡1
³
F ¡
³
1 ¡ I
(h)
t
´
FK
(h)
t H0
´0
(7)
We note that for t = h, the FPS gives Ka
h = 0, x
(h)
hjh = x
(h)
hjh¡1, §
(h)
hjh = §
(h)
hjh¡1 and §a
h+1jh = §a
hjh¡1F0.
The interpolation value for zh is z
(h)
hjT = H0x
(h)
hjT and the corresponding interpolated error is given by,
ih = zh ¡ z
(h)
hjT; (8)
with E
£
i2
h
¤
= ¾2
³
H0§
(h)
hjTH
´
.
3 A relationship between prediction and interpolation errors in
ARMA processes
In this section, we analyze the relationship between interpolation and prediction errors and obtain an ex-
pression for the prediction error in terms of the interpolation errors. This relationship allows us to obtain
the covariance matrix of the interpolation errors. We also show that the parameter values which minimize
the sum of squares of the standardized one step-ahead prediction errors are the same that minimize the sum
of squares of the standardized interpolation errors. This main result is summarized in the following theorem:
Theorem 1 Let z = (z1;:::;zT)
0 be a time series generated by the stationary and invertible ARMA(p;q)
process in (1). Let ih be the interpolation error of the observation at t = h and let eh;:::;eT be one step
ahead prediction errors assuming that all the observations are known. Then, ih can be written as follows,
ih = c
(h)
h eh + c
(h)
h+1eh+1 + ::: + c
(h)
T eT (9)
5where the coe±cients c
(h)
h ;:::;c
(h)
T are given by,
c
(h)
h = 1 ¡ H0
T X
i=h+1
Ka
i bh
i c
(h)
t = ¡H0
Ã
Ka
t ¡
T X
i=t+1
Ka
i bh
i
!
; t = h + 1;:::;T; (10)
and the coe±cients bs
t are given by,
b
h+i
t = H0Ft¡h¡iKh ¡ H0F
t¡h¡i X
j=1
K
(h)
h+jb
h+i
h+j; bh
t = H0Ft¡hKh ¡ H0F
t¡h¡1 X
j=1
K
(h)
h+jb
h+i
h+j; (11)
for i = 0;:::;t ¡ h ¡ 1.
Proof. Let,
e
(h)
t = zt ¡ z
(h)
tjt¡1; t = h + 1;:::;T (12)
be the prediction errors assuming that observation at t = h is missing. The relationships in (7) provide the
following expression for the interpolated value z
(h)
hjT:
z
(h)
hjT = z
(h)
hjh¡1 + H0Ka
t+1
³
zh+1 ¡ z
(h)
h+1jh
´
+ ::: + H0Ka
T
³
zT ¡ z
(h)
TjT¡1
´
and by (8) and (12),
ih = eh ¡ H0Ka
t+1e
(h)
h+1 ¡ ::: ¡ H0Ka
Te
(h)
T : (13)
We will obtain an expression of the errors e
(h)
t in terms of the prediction errors et, for t > h. For (12), we
have:
e
(h)
t = zt ¡ ztjt¡1 + ztjt¡1 ¡ z
(h)
tjt¡1 = et + ztjt¡1 ¡ z
(h)
tjt¡1:
Using the Kalman Filters in (4) and (6), it can be shown that:
ztjt¡1 ¡ z
(h)
tjt¡1 = H0Ft¡hKheh + H0Ft¡h¡1
³
Kh+1eh+1 ¡ K
(h)
h+1e
(h)
h+1
´
+ ::: + H0F
³
Kt¡1et¡1 ¡ K
(h)
t¡1eh
t¡1
´
;
and, therefore,
e
(h)
t = H0Ft¡hKheh + H0Ft¡h¡1
³
Kh+1eh+1 ¡ K
(h)
h+1e
(h)
h+1
´
+ ::: + H0F
³
Kt¡1et¡1 ¡ K
(h)
t¡1e
(h)
t¡1
´
+ et: (14)
6Consequently, starting from e
(h)
h+1 = eh+1 + H0FKheh, we obtain the values of e
(h)
t in terms of eh;:::;et
as:
e
(h)
t = et + b
t¡1
t et¡1 + ¢¢¢ + bh
t eh
where the coe±cients bs
t are obtained recursively from (13) and (14), and are given by,
b
h+i
t = H0Ft¡h¡iKh ¡ H0F
t¡h¡i X
j=1
K
(h)
h+jb
h+i
h+j; bh
t = H0Ft¡hKh ¡ H0F
t¡h¡1 X
j=1
K
(h)
h+jb
h+i
h+j;
for i = 0;:::;t ¡ h ¡ 1, which shows (9) with the coe±cients in (10).
Some consequences are as follows. First, E [ih] = 0. Second, the variance of ih is given by,
var(ih) = E
¡
i2
h
¢
= E
2
4
Ã
T X
t=h
c
(h)
t et
!23
5 = ¾2
T X
t=h
³
c
(h)
t
´2
v2
tjt¡1:
Third, if m > h, then,
cov (ih;im) = ¾2
T X
t=m
c
(h)
t c
(m)
t v2
tjt¡1:
Let e=(e1;:::;eT)
0 and i = (i1;:::;iT)
0 be the vectors of prediction and interpolation errors. The vector
e has a diagonal covariance matrix §e with elements ¾2v2
tjt¡1, t = 1;:::;T. The vectors e and i are related by
i = Ce, where C is an upper triangular T £T matrix with elements cuv = c
(v)
u , u;v = 1;:::;T. Consequently,
the covariance matrix of i is §i = C§eC0, and taking into account that i = Ce, we also have that,
i0§
¡1
i i=(Ce)
0 (C§eC0)
¡1 (Ce) = e0§¡1
e e.
As a consequence, the parameters that minimize the sum of squares of standardized interpolation errors,
i0§
¡1
i i, are the ML estimates, that is the parameters which minimize the sum of squares of standardized
prediction errors, e0§¡1
e e. As a by-product, we get that,
@
¡
e0§¡1
e e
¢
@
¡
Á1;:::Áp;µ1;:::;µq;¾2¢ =
@
¡
i0§
¡1
i i
¢
@
¡
Á1;:::Áp;µ1;:::;µq;¾2¢:
74 Illustration
As an illustration, consider an stationary AR(1) model with zero mean and autoregressive parameter Á.
Running the Kalman ¯lter for a realization of this process, z = (z1;:::;zT)
0, with initial conditions x1j0 =
x0j0 = 0 and §1j0 = §0j0 =
¡
1 ¡ Á
2¢¡1
, we get the prediction errors e = (e1;:::;eT)
0 and their conditional
variances ¾2v2
1j0 = ¾2 ¡
1 ¡ Á
2¢¡1
and ¾2v2
tjt¡1 = ¾2, t > 1. Running the Kalman ¯lter assuming that the
observation t = h is missing, and then, the FPS algorithm, we get,
§a
t+1jt =
8
> <
> :
Á t = h
0 t > h
Ka
t =
8
> <
> :
0 t 6= h + 1
Á
1+Á2 t = h + 1
x
(h)
hjt =
8
> <
> :
Áxh¡1 t = h
Á
1+Á2 (zh¡1 + zh+1) t > h
§
(h)
hjt =
8
> <
> :
1 t = h
1
1+Á2 t > h
;
implying that the interpolated value is z
(h)
hjT =
Á
1+Á2 (zh¡1 + zh+1) with interpolation error ih = zh ¡
Á
1+Á2 (zh¡1 + zh+1). From (13) and (11), we get,
ih =
8
> > > > <
> > > > :
¡
1 ¡ Á
2¢
e1 ¡ Áe2 h = 1
1
1+Á2eh ¡
Á
1+Á2eh+1 h = 2;:::;T ¡ 1
eT h = T
; (15)
which gives the variance of ih,
var(ih) =
8
> <
> :
¾
2
1+Á2 h = 2;:::;T ¡ 1
¾2 h = 1;T
and the covariances between interpolation errors,
cov (ih;im) =
8
> > > > <
> > > > :
¡¾
2Á
(1+Á2) m = h + 1; h = 1; T ¡ 1
¡¾
2Á
(1+Á2)
2 m = h + 1; h = 2;:::;T ¡ 2
0 m > h + 1; h = 1;:::;T
implying that the interpolation errors are uncorrelated if m ¡ h > 1.
Finally, we show the equality i0§
¡1
i i = e0§¡1
e e in the case of an AR(1) model. For that, we note that §e
8can be written as,
§e = ¾2
µ
I +
Á
2
1 ¡ Á
2U
¶
where U is a matrix which all its elements are 0 except the (1,1) element that is 1. From (15) we have that
i = Ce where the matrix C has elements,
C(i;j) =
8
> > > > > > > > > > > > > > <
> > > > > > > > > > > > > > :
¡
1 ¡ Á
2¢
(i;j) = (1;1)
¡Á (i;j) = (1;2)
1
1+Á2 j = i; 2 · i · T ¡ 1
¡
Á
1+Á2 j = i + 1; 2 · i · T ¡ 1
1 (i;j) = (T;T)
0 otherwise
As i0§
¡1
i i = e0C0 (C§eC0)
¡1 Ce, we only need to show that C0 (C§eC0)
¡1 C = §¡1
e . For that,
C§eC0 = C¾2
µ
I +
Á
2
1 ¡ Á
2U
¶
C0 = ¾2
µ
CC0I +
Á
2
1 ¡ Á
2CUC0
¶
=
= ¾2CC0
µ
I +
Á
2
1 ¡ Á
2 (C0)
¡1 UC0
¶
= ¾2CC0
µ
I +
Á
2
1 ¡ Á
2U
¶
Therefore,
C0 (C§eC0)
¡1 C =
1
¾2C0
µ
I +
Á
2
1 ¡ Á
2U
¶¡1
(CC0)
¡1 C =
1
¾2C0 ¡
I ¡ Á
2U
¢
(C0)
¡1 =
=
1
¾2
³
I ¡ Á
2C0U (C0)
¡1´
=
1
¾2
¡
I ¡ Á
2U
¢
= §¡1
e :
Acknowledgment
We acknowledge the ¯nancial support by project DGICYT BEC2000-0167, Spain.
References
Anderson, B. and Moore, J., 1979. Optimal Filtering, Englewood Cli®s, NJ: Prentice Hall.
G¶ omez, V. and Maravall, A., 1994. Estimation, prediction, and interpolation for nonstationary series with
the Kalman ¯lter. J. Am. Statist. Assoc. 89, 611-624.
Jones, R. H., 1980. Maximum likelihood ¯tting of ARMA models to time series with missing observations.
9Technometrics 22, 389-395.
10

Maximum likelihood ¯tting of ARMA models to time series with missing observations. 9Technometrics 22,

Optimal Filtering,

A NOTE ON PREDICTION AND INTERPOLATION ERRORS IN TIME SERIES

http://docubib.uc3m.es/WORKINGPAPERS/WS/ws042710.pdf

A NOTE ON PREDICTION AND INTERPOLATION ERRORS IN TIME SERIES

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics