We study alternative models for capturing abrupt structural changes (level shifts) in a times series. The problem is confounded by the presence of transient outliers. We compare the performance of non-Gaussian time-varying parameter models and multiprocess mixture models within a Monte Carlo experimental setup. Our findings suggest that once we incorporate shocks with thick-tailed probability distributions, the superiority of the multiprocess mixture models over the time-varying parameter models, reported in an earlier study, disappears. The behavior of the two models, both in adapting to level shifts and in reacting to transient outliers, is very similar.time-varying parameter (TVP) models, non-Gaussian state space models, multiprocess mixture models, level shifts, outliers

Prasad Bidarkota

English

Research Papers in Economics

A Comparison of Two Alternative Approaches to 
Modeling Level Shifts in the Presence of Outliers 
 
 
Prasad V. Bidarkota
* 
 
Department of Economics, University Park, DM-316 
Florida International University, Miami Florida, 33199, USA 
Tel: +1-305-348-3281; Fax: +1-305-348-1524; E-mail: bvp@iitbombay.org 
 
 
 
 
Abstract: We study alternative models for capturing abrupt structural changes (level 
shifts) in a times series. The problem is confounded by the presence of transient outliers. 
We compare the performance of non-Gaussian time-varying parameter models and 
multiprocess mixture models within a Monte Carlo experimental setup. Our findings 
suggest that once we incorporate shocks with thick-tailed probability distributions, the 
superiority of the multiprocess mixture models over the time-varying parameter models, 
reported in an earlier study, disappears. The behavior of the two models, both in adapting 
to level shifts and in reacting to transient outliers, is very similar. 
 
Key phrases: time-varying parameter (TVP) models; non-Gaussian state space models; 
multiprocess mixture models; level shifts; outliers. 
 
JEL Codes: C22, E31, C53 
 
 
First version: September 15, 2002 
Revised: July 16, 2003 
 
                                                 
* Comments from an anonymous referee and conversations with Tom Marsh proved very 
helpful and are gratefully acknowledged. 
  1Introduction 
The problem of modeling structural changes (such as level shifts) in a time series 
has been studied by a number of scholars. Several approaches have been proposed for 
detecting and dealing with such behavior. One early technique is the variable parameter 
regression (VPR), or the time varying parameter (TVP) model as it is more popularly 
known, proposed by Garbade (1977). Although still widely used in applied time series 
analysis, it has been recognized that this model has difficulty in adapting quickly to 
sudden and abrupt shifts (see, for instance, Kitagawa (1987) and Bidarkota and 
McCulloch (1998)). This is essentially due to the assumption that all the stochastic 
shocks driving this model are Gaussian. 
Kitagawa (1987) proposed a non-Gaussian state space model, which encompasses 
the time varying parameter model, for modeling abrupt changes in regime. A lot of 
literature has appeared since the publication of this paper, dealing with estimation and 
modeling of non-Gaussian and nonlinear state space models. Durbin and Koopman 
(2000) is a recent important addition to this literature. A general conclusion from these 
numerous studies is that non-Gaussian state space models deal with abrupt regime 
changes better than the Gaussian TVP models. 
Further limitations of the TVP model have been reported by Gamble and LeSage 
(1993), particularly when the data series contains outliers in addition to abrupt regime 
changes. These authors compare the performance of a multiprocess mixture model, 
originally proposed by Gordon and Smith (1990), with a TVP model in the context of a 
Monte Carlo study and conclude that the multiprocess mixture model is promising for 
simultaneously modeling abrupt regime changes and outliers.  
  2An important assumption in the study by Gamble and LeSage (1993) is that all 
shocks in both the TVP and the multiprocess mixture models are Gaussian. This is not 
innocuous, given the deliberate contamination of the experimental data series analyzed by 
transient outliers and given the findings in Kitagawa (1987). We therefore seek to 
investigate the robustness of the reported superiority of the multiprocess mixture models 
over the TVP models in such a setting, but assuming that all shocks driving the two 
models have probability distribution functions with thick tails.  
  This paper is structured as follows. Section 2 outlines the time varying parameter 
model and the multiprocess mixture model. Section 3 describes the procedure used to 
estimate the two models. Section 4 discusses the experimental setup and results. Section 
5 offers brief conclusions on our findings. 
 
Section 2: Alternative Models  
To motivate the specifications of the models that follow, we plot in Figure 1 
artificially generated data that illustrate the level shifts phenomenon that we are trying to 
model. A set of 100 observations  ,   t y 100 ,..., 2 , 1 t = , were generated in the following way. 
A set of 100   random variables was first drawn. Two outliers were introduced 
at observation numbers 35 and 75 by multiplying the random numbers drawn at these 
periods by a factor of 20. Two level changes were introduced in the mean of the series by 
adding the number 1 to the first 25 of these random numbers, subtracting the number 1 
for observations 26 through 50, and once again adding the number 1 to the remaining 
observations. This data generating mechanism is very similar to the one in Gamble and 
Le Sage (1993) and Kitagawa (1987). 
) 1 , 0 ( N . d . i . i
  3We are especially interested in modeling the shifting mean value of the time 
series. Note that the entire framework in this paper is univariate modeling. We abstract 
from considering the role of any explanatory variables here but this can easily be 
addressed in the methods that follow. Where such an extension may present added 
nontrivial difficulties, we explicitly point this out below and suggest well-known and 
adequate methods of surmounting them. 
 
2.1 Non-Gaussian State Space Models 
  Consider a state space model for an observable series    t y:
t t t x y ε + =             ( 1 a )  
t 1 t t x x η + = −          ( 1 b )  
where   and η  are independent white noise sequences with some specified probability 
density functions  and   centered around zero, respectively. In this model, the 
mean of the series is tracked by the random variable  . It is a constant when the 
disturbance   is trivially distributed with a mass point at zero. Otherwise, the mean 
varies over time and can, hence, track level shifts. 
t ε t
t
) ( r ε ) ( q η
t x
η
When   and   are both Gaussian, we obtain the time varying parameter (TVP) 
model of Garbade (1977). As noted in the introduction, this model has been widely used 
to model level shifts (as well as slope shifts) in regression models. However, Kitagawa 
(1987) recommends using thick tailed non-Gaussian distributions from which to draw 
either the disturbance η  alone or both 
t ε t η
t t ε  and  t η . He demonstrates the added efficacy of 
such models in tracking abrupt level shifts quickly and more accurately. 
  4While several candidate distributions exist from which to draw the disturbances 
 and η , Kitagawa (1987) makes no recommendations among these. We shall use 
symmetric stable distributions wherever we entertain thick tailed error disturbances in 
this paper.
t ε t
1 
A random variable   is said to have a symmetric stable distribution Sc  if 
its log-characteristic function can be expressed as: 
X α δ (,)
ln exp( ) | | Ei X t i t c t =− δ α.          ( 2 )  
The parameters   and  c > 0 δ∈−∞ ∞ ( , ) are measures of scale and location, respectively, 
and   is the characteristic exponent governing the tail behavior with a smaller 
value of α indicating thicker tails. The normal distribution belongs to the symmetric 
stable family with  , and is the only member with finite variance, equal to 2  
α∈(,] 0 2
α=2 2 c.
 
2.2 Multiprocess Mixture Models 
A simple multiprocess mixture model, somewhat along the lines of the one 
proposed by Gordon and Smith (1990), takes the following form: 
) c , 0 ( S ~ , x y t t t t α ε ε + =       ( 3 a )  
, x x t 1 t t η + = −    (3b) 



λ − ρ
λ
η
α ) 1 ( y probabilit with ) c , 0 ( S
y probabilit with 0
~ t
                                                 
1 While there is little justification that anyone can provide at present for using the 
symmetric stable or some other specific distribution here, a comparison of the 
performance of alternative leptokurtic distributions in modeling level shifts should be 
useful in this regard. However, such an undertaking goes beyond the scope of the present 
work.  
  5In the state space models discussed in the previous subsection, a regime change (or a 
level shift) in the mean occurs every period (except in the degenerate case when the 
disturbance   is trivially distributed with a mass point at zero). In the multiprocess 
mixture model, however, a regime change need not occur every period. Instead, a 
Bernoulli process governs whether a regime change occurs in any given period.  
t η
The error term   dictating the regime changes has a continuous distribution with 
density   that integrates to a probability 
t η
) c , 0 ( S ρ α 1 ) 1 ( < λ −  and a mass point with 
probability   at zero. Thus,   has a compound distribution. There is a non-trivial 
probability   that η  in which case there is no regime change, and a probability 
 that a regime change occurs every period.  When a regime change does occur, the 
magnitude of the change is drawn from a continuous symmetric stable distribution with 
density  .  
λ
0 > λ
) c , 0 ( S ρ α
t η
0 = t
) 1 ( λ −
  The model in Equations (3) is a mixture of two models – one with   and the 
other with η  having a non-degenerate distribution. In the more general multiprocess 
mixture models proposed by Gordon and Smith (1990), the mixture could consist of 
several more component models such as, for instance, a component that tracks changes in 
the slope coefficient on an explanatory variable.  
0 t = η
t
In the following sections, our objective is to compare the performance of the non-
Gaussian state space models and the multiprocess mixture models in describing the type 
of time series as plotted in Figure 1. We also report on and compare the performance of 
the Gaussian versions of both the state space and the multiprocess mixture models in this 
context. 
  6Section 3: Estimation Issues 
Maximum likelihood estimation of the non-Gaussian state space models and the 
multiprocess mixture models is facilitated by the Sorenson-Alspach (1971) filtering 
algorithm. In what follows in this section, we provide details on the Sorenson-Alspach 
algorithm and its numerical implementation.  
  In the non-Gaussian state space models and the multiprocess mixture models, the 
conditional density of the unobserved state  , given the observations 
, is denoted by  . The Sorenson-Alspach (1971) recursive 
formulae for obtaining one step-ahead prediction 
t x
m
m m 2 1 Y ) y ,...., y , y ( = ) Y | x ( p m t
) 1 t ( − =  and filtering ( ) t m =  
densities are as follows: 
px Y px x px Y d x tt tt t t t (| ) (| ) ( | ) −− − −
−∞
∞
= ∫ 11 1 1 − 1
− 1
− 1
t 1
,        ( 4 a )  
px Y py x px Y py Y tt tt tt tt (|) (|) (| ) /(| ) = − 1 ,      ( 4 b )  
py Y py x px Y d x tt tt tt t (| ) (|) (| ) −
−∞
∞
= ∫ 1 .      ( 4 c )  
These formulae have been extended to include a smoother   formula by Kitagawa 
(1987). The log-likelihood function, conditional on the hyperparameters of the model, is 
given by: 
) t m ( >
log ( ,..., ) log ( | ). py y py Y Tt
t
T
1
1
= −
= ∑       ( 5 )  
  When both disturbance terms  t ε  and  t η  in Equations (1) are normally distributed, 
the Sorenson-Alspach filter in Equations (4) above collapses to the famous Kalman filter. 
  7Thus, the Kalman filter provides optimal, linearly adaptive estimates of   under these 
distributional assumptions. 
t x
  Under more general assumptions about the distribution of the two error terms, as 
in the non-Gaussian state space models and the multiprocess mixture models, however, 
the Kalman filter does not provide globally optimal estimates of the state vector. The 
integrals in Equations (4) cannot usually be solved in closed form under these 
circumstances. 
  One approach is to evaluate these integrals numerically, as in Kitagawa (1987), or 
Hodges and Hale (1993). In this paper, we estimate the non-Gaussian state space models 
and the multiprocess mixture models by evaluating these integrals with the numerical 
integration techniques in Bidarkota and McCulloch (1998). They provide details on the 
accuracy of their approximation procedure. 
  It is worth noting at this point that when the unobserved state variable   in the 
non-Gaussian state space and multiprocess mixture models contains, in addition to the 
intercept term alone that we have here, a set of slope coefficients on some explanatory 
variables as well, then the integrals in Equations (4) will be multidimensional. In this 
situation, evaluating the high dimensional integrals numerically, as is being done here in 
this paper, is not feasible. A useful alternative is to evaluate the integrals using Monte 
Carlo techniques, as in Tanizaki and Mariano (1998) or Durbin and Koopman (2000). 
t x
Finally, the probability density for the symmetric stable distributions required for 
maximum likelihood estimation of all the non-Gaussian stable models is computed using 
the numerical algorithm in McCulloch (1996).  
 
  8Section 4: Monte Carlo Experiment and Results 
  A set of 1000 samples was generated using the same process that was used to 
produce Figure 1. The two models in Equations (1) and (3) were estimated using the 
methods described in Section 3 for these 1000 samples. For each sample, the mean 
 was computed at each point in time using the filter density obtained from 
Equation (4b) for both the models. The behavior of this quantity   at each point 
in time over the 1000 samples is compared for the two models.  
) Y | x ( E t t
) Y | x ( E t t
Figure 2 plots the mean of   at each point in time over the 1000 samples 
for the two models. As the two panels indicate, there is little perceptible difference in the 
mean of   across the two models at any point in time. Figure 3 plots the same 
quantity obtained with the Gaussian versions of the two models. It is quite evident from 
the two panels here that the multiprocess mixture model adapts more rapidly to the abrupt 
level shifts than the TVP model. Thus, the first key finding in Gamble and LeSage (1993) 
that the multiprocess mixture model adapts more rapidly to the abrupt level shifts at 
observations 26 and 51 than the TVP model disappears once we admit thick-tailed non-
Gaussian disturbances.  
) Y | x ( E t t
) Y | x ( E t t
Notice the impact of the outliers at observations 35 and 75 on the mean of 
. As the two panels of Figure 2 indicate, there is once again little noteworthy 
difference in the impact of the outliers on the mean of the filter density across the two 
non-Gaussian models. However, the two panels of Figure 3 show that the outliers have 
greater impact on the estimates obtained with the TVP model. Thus, the second key 
finding in Gamble and LeSage (1993) that the outliers at observations 35 and 75 have 
only a slight impact on the mixture models estimates but influence the TVP model 
) Y | x ( E t t
  9estimates much more profoundly disappears once we drop the Gaussian assumption for 
the disturbances.  
Figures 4 and 5 plot the quantiles of E  across the 1000 samples for the 
two models at each point in time with non-Gaussian and Gaussian disturbances, 
respectively. These provide an indication of the variation of   across the 1000 
samples for the two models at each point in time. As the two panels of Figure 4 indicate, 
there is once again little noteworthy difference in the quantiles of the mean of the filter 
density across the two non-Gaussian models. The two panels of Figure 5 however 
indicate significant differences in the quantiles obtained with Gaussian versions of the 
models. Note in particular the differences in the scales on the y-axis in the two panels. 
) Y | x ( t t
) Y | x ( E t t
Overall, for tracking the mean of the time series subject to abrupt level shifts and 
confounded by transient outliers, our results show that the behavior of the non-Gaussian 
state space model, the non-Gaussian multiprocess mixture model and the Gaussian 
multiprocess mixture model is very similar and clearly better than that of the Gaussian 
TVP model. The interquartile range for the filter mean obtained with the non-Gaussian 
state space and mulitprocess mixture models is much narrower in the neighborhood of 
large outliers compared to that obtained with the Gaussian multiprocess mixture model. 
 
Section 5: Concluding Remarks 
Our experiment suggests that the key finding in Gamble and LeSage (1993), that 
multiprocess mixture models provide a better alternative to TVP models in capturing 
level shifts in the presence of outliers, hinges critically on their assumption that all shocks 
driving the two models are Gaussian. Once we allow shocks in these models to be drawn 
  10from probability distributions that exhibit fat tails, both models display very similar 
behavior in learning about level shifts and in reacting to outliers. Compared to the 
Gaussian models, the non-Gaussian models track the mean of the time series both more 
accurately and more precisely.  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  11  12 
 
 
 
 
  13 
  14 
 
 
 
 
  15 
 
 
 
 
  16REFERENCES 
 
Bidarkota, P.V. and J.H. McCulloch (1998),‘Optimal univariate inflation forecasting 
with symmetric stable shocks,’ Journal of Applied Econometrics, Vol.13, No.6, 659-670. 
 
Durbin, J. and S.J. Koopman (2000), ‘Time series analysis of non-Gaussian 
observations based on state space models from both classical and Bayesian perspectives,’ 
Journal of The Royal Statistical Society, Series B, 62, Part 1, 3-56. 
 
Gamble, J.A. and J.P. LeSage (1993), ‘A Monte Carlo comparison of time varying 
parameter and multiprocess mixture models in the presence of structural shifts and 
outliers,’ The Review of Economics and Statistics, Vol.75, No.3, 515-519. 
 
Garbade, K. (1977), ‘Two methods for examining the stability of regression 
coefficients,’ Journal of the American Statistical Association, Vol. 72, No. 357, 54-63. 
 
Gordon, K. and A.F.M. Smith (1990), ‘Modeling and monitoring biomedical time 
series,’ Journal of the American Statistical Association, Vol. 85, No. 410, 328-337. 
 
Hodges, P.E. and D.F. Hale (1993), ‘A computational method for estimating densities of 
non-Gaussian nonstationary univariate time series,’ Journal of Time Series Analysis, 
Vol.14, No.2, 163-178. 
 
  17  18
Kitagawa, G. (1987), ‘Non-Gaussian state space modeling of nonstationary time series,’ 
Journal of the American Statistical Association, Vol.82, No.400, 1032-63. 
 
McCulloch, J.H. (1996), ‘Numerical approximation of the symmetric stable distribution 
and density,’ in R. Adler, R. Feldman, and M.S. Taqqu (eds.), A Practical Guide to 
Heavy Tails: Statistical Techniques for Analyzing Heavy Tailed Distributions, Boston: 
Birkhauser. 
 
Sorenson, H.W. and D.L. Alspach (1971), ‘Recursive Bayesian estimation using 
Gaussian sums,’ Automatica, 7, 465-479. 
 
Tanizaki, H. and R.S. Mariano (1998), ‘Nonlinear and non-Gaussian state-space 
modeling with Monte Carlo simulations,’ Journal of Econometrics, 83, 263-290. 
 
 

Comparison of Two Alternative Approaches to Modeling Level Shifts in the Presence of Outliers

http://casgroup.fiu.edu/pages/docs/2248/1280267800_03-07.pdf

Comparison of Two Alternative Approaches to Modeling Level Shifts in the Presence of Outliers

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics