Autoregressive (AR), moving-average (MA), and autoregressive- moving average (ARMA) models are very popular in time-series analysis. Many problems of estimating their parameters and testing hypotheses are only solved asymptotically. The derivation of asymptotic results is usually not easy. An alternative approach to such problems is the Bayesian approach. It is assumed that the parameters of the models are random variables. There are theorems ensuring that under general assumptions the asymptotic posterior distribution does not depend on the prior distribution. As the derivation of the results is usually easier in the Bayesian approach, we can use this procedure particularly for the statistical analysis of more complicated models

Andel, J.

International Institute for Applied Systems Analysis (IIASA)

Bayesian Estimates in Time-Series ModelsAndel, J. IIASA Collaborative PaperJune 1985Andel, J. (1985) Bayesian Estimates in Time-Series Models. IIASA Collaborative Paper. Copyright © June 1985 by the author(s). http://pure.iiasa.ac.at/2719/ All rights reserved. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage. All copies must bear this notice and the full citation on the first page. For other purposes, to republish, to post on servers or to redistribute to lists, permission must be sought by contacting repository@iiasa.ac.at NOT FOR QUOTATION WITHOUT PERMISSION OF THE AUTHOR BAYESIAN ESTIMATES I N  Tm-SERIES MODELS June 1985 CP-85-29 CoZZaborative Papers r e p o r t  work which h a s  n o t  been performed s o l e l y  a t  t h e  I n t e r n a t i o n a l  I n s t i t u t e  f o r  Applied Systems Ana lys i s  and which h a s  r e c e i v e d  o n l y  l i m i t e d  review. V i e w s  o r  o p i n i o n s  expres sed  h e r e i n  do n o t  n e c e s s a r i l y  r e p r e s e n t  t h o s e  of  t h e  I n s t i t u t e ,  i ts  National.Member Organ iza t ions ,  o r  o t h e r  o rgan i -  z a t i o n s  s u p p o r t i n g  t h e  work. INTERNATIONAL INSTITUTE FOR APPLIED SYSTEMS ANALYSIS A-2361 Laxenburg, A u s t r i a  Within the framework of the Economic Structural Change Proqram, a cooperative research activity of IIASA and the uni;ersity of Bonn, FRG, a project is carried out on "Statis- tical and Econometric Identification of Structural Change"; the project involves studies on the formal aspects of the analysis of structural changes. On the one hand, they include statistical methods to detect non-constancies, such as sta- bility tests, detection criteria, etc., and on the other hand, methods which are suitable for models which incorporate non- constancy of the parameters, such as estimation techniques for time-varying parameters, adaptive methods, etc. The present paper discusses the application of Bayesian estimation methods in the context of ARMA-models such as the periodic autoregression. Anatoli Smyshlyaev Acting Leader Economic Structural Change Program BAYESIAN ESTIMATES I N  TIME-SERIES MODELS J i r i  Andel C h a r l e s  U n i v e r s i t y ,  SokoZovska 83,  1 8 6  00  Prague 8 ,  Czechos Zovakia INTRODUCTION Autoregress ive  (AR) , moving-average (MA) , and au tore -  gressive-moving average  (ARMA) models a r e  very  popular  i n  time-series a n a l y s i s .  Many problems of e s t i m a t i n g  t h e i r  param- eters and t e s t i n g  hypotheses  a r e  on ly  so lved  a sympto t i ca l ly .  The d e r i v a t i o n  of  asympto t ic  r e s u l t s  i s  u s u a l l y  n o t  easy.  An a l t e r n a t i v e  approach t o  such problems i s  t h e  Bayesian approach. It  i s  assumed t h a t  t h e  parameters  of  t h e  models a r e  random v a r i a b l e s .  There a r e  theorems ensu r ing  t h a t  under g e n e r a l  assumptions t h e  asymptot ic  p o s t e r i o r  d i s t r i b u t i o n  does n o t  de- pend on t h e  p r i o r  d i s t r u b i t o n .  A s  t h e  d e r i v a t i o n  of t h e  re- s u l t s  i s  u s u a l l y  e a s i e r  i n  t h e  Bayesian approach,  we can use  t h i s  procedure  p a r t i c u l a r l y  f o r  t h e  s t a t i s t i c a l  a n a l y s i s  of more complicated models. PERIODIC AUTOREGRESSION I n  many a p p l i c a t i o n s  w e  encounter  t i m e  series wi th  a s easona l  behavior .  W e  may assume t h a t  t h e  l e n g t h  of season p i s  known (e .g . ,  f o r  monthJydata e x h i b i t i n g  a one-year season- a l i t y  we have p = 1 2 ) .  I n  t h e  c l a s s i c a l  approach Box and Jenkins  (1970, Ch. .9) recommend t o  s t a r t  w i th  t h e  d i f f e r e n c e s  V X  = X t - X  and to apply an ARMA model to the differences P t t-P d D of the type VIVpXt. As such differencing is, however, not sufficient for real time series, a nonlinear transformation is sometimes applied before. In the well-known example of the number of airline passengers the log-transformation is applied at the beginning without taking into account its influence on other statistical procedures. If the length of periodicity p = 2, we can consider the following modification of an AR model: where {Y is a white noise. Simulations show (Andel 1983) t that the realization of such a model is very similar to some economic time series. In some cases the model has an explosive behavior. To take account of this behavoir, we put Then our model can be expressed in the form of Let zl, z2 denote the roots of the polynomial If lzl\<l, 1z21<1, the process {Et) is stationary; other- wise {E ) has an explosive behavior. t Our example was a special case of periodic autoregression. We used a model of the second order with p = 2. Generally, the periodic autoregression is given by If {Yt) is the usual white noise, we have a model with equal variances. If var Y 2 = Ok' 2 n+(j-l)p+k where ol, ..., . . . ,cr2 are not all the same, we have a model with periodic P variances. The periodic autoregression is a special case of periodically correlated random sequences, whkch were intro- duced by Gladyshev (1961). Jones and Brelsford (1967) expanded bkl , . . . ,bkn into a Fourier series. Pagano (1 978) considered estimators for bki obtained by modified Yule-Walker equations. He also showed that a periodic AR can be rewritten into a multidimensional AR model. A periodic ARMA model was intro- duced by Cleveland and Tiao (1979). The problem of the periodic AR is also treated by Troutman (19791. Tiao and Grupe (1980) investigated the errors of nlsclassLfication when the periodic structure of an ARMA process was neglected. Newton (1982) shows that a periodic AR can substantially simplify the numerical procedures for estimating parameters in multiple AR models. Andel (1983) presents some results of the Bayesian analysis of the periodic AR model including tests on whether the t2me series can be described by the classical AR model. THE BAYESIAN APPROACH If 8 E 9 is a random (multidimensional) parameter with a prior density p(0), and if a random vector X has a conditional density of ~ ( ~ 1 0 )  given 0, then according to the Bayes theorem, the posterior density ~ ( 0 1 ~ )  of 0 is given by where cx i s  a  c o n s t a n t .  Usual ly  t h e  modus of t h e  p o s t e r i o r  d e n s i t y  is  taken a s  an e s t i m a t o r  of 8. This  i s  a gene ra l i za -  t i o n  of t h e  maximum l i k e l i h o o d  e s t i m a t o r ,  which w e  would g e t  f o r  p ( 8 )  = cons t .  (Another e s t ima to r  of 8  could be t h e  p o s t e r i o r  expec ta t ion . )  The main problem wi th  using t h e  BayesTan approach is t h e  choice  of t h e  p r i o r  d e n s i t y  p(0)  . The fol lowing t h r e e  poss i -  b i l i t i e s  a r e  most popular:  a .  Conjugate p r i o r  dens2ty.  L e t  p(xl 0 )  be given.  A system M P ( X !  8 )  is c a l l e d  a  conjugate  system, i f  p(8  [ x )  E M ( x l  holds  f o r  every p ( e )  E M P ( X I  8)  . Usually w e  t a k e  t h e  minimal system M P ( X (  8)  wi th  t h i s  proper ty ,  o r  t h e  so-ca l led  n a t u r a l  system. Although a  conjugate  p r i o r  d e n s i t y  i s  convenient from t h e  mathematical p o i n t  of view, t h e r e  i s  no l o g i c a l  reason f o r  us ing  it i n  a  given case .  Moreover, t h i s  procedure does not  s p e c i f y  which d e n s i t y  from t h e  system M p(xl  e )  should be taken.  b. Uncer ta in ty  p r i n c i p l e .  I n  t h e  case  of " f u l l  i g -  norance" it i s  recommended t o  pu t  p ( 8 )  = 1 f o r  8  E 52. This  o f t e n  l e a d s  t o  improper (o r  vague) d e n s i t i e s ,  e .g .  i f  52 = Rk (Euclidean k-dimensional s p a c e ) .  The advantage of  t h i s  method i s  t h a t  t h e  d e n s i t y  p ( 8 )  = 1 can a sympto t i ca l ly  s u b s t i t u t e  any o t h e r  reasonable  p r i o r  dens2ty.  Theorem (DeGroot 1970. 5 10.4) . L e t  no (8 1x1 = cp (x 1 0 )  and l e t  .rr(8(x) = c 1 p ( x [ 9 ) p ( 8 ) .  L e t  A c 52 be  such a set  t h a t  m = i n f o  A ~ ( 8 )  > 0. L e t  a , b , c  > 0 s a t i s f y  scp  p ( 8 )  = ( l+c)m . 8 E R-A Then where -1 E = max [ (1-a) (a+b) , (l+a+b+ac) -' (a-tbtab) J + In the typical situation the posterior distribution TT o is the normal one with a variance matrix of the order N-' (where N is the number of observations). Then a and b are nearly zero and E is very small. Unfortunately, the uncertainty principle is not logically consistent. For example, let ~ ( ~ 1 8 )  = 1 ex (1-0) n-x , 0 c 0 e l  . If 8 is completely unknawn, we choose its prAor density p (8) = 1 and compute p (0 1x1 by means of the Bayea theorem. -I We find that w = €JL has a posterlor density of c w  (X-1)/2(~-w1/2)n-X.   ow ever, 1/2 n-x p(xlw) = wXI2 (1-w ) If w is considered to be a completely unknown parameter with a constant prior density, then Its posterior density is c~w~/*.(~-w 1/2)n-x c. JeffreyFs principle. To avoid inconsistency it is 1 /2 necessary to choose the prtor density proport2onal to IJ(o)[ , where J is the Fisher information matrix. In practice the prior denstty is chosen by combining the uncertainty principle and Jeffrey's principle. SOME RESULTS CONCERNING TRE PERXODIC AR Andel (1983) proved the following assertions: Theorem. Assume model ( x )  , where XI, ..., Xn are given 2 constants and {ytl are i.i.d. N(0,o ) varfables. If xl, ..., xN is a known realization, put [N-n-k " = k ] + 1 (where [ 3 is the integer part), P (kJ = "k qij 1 Xn+k+ (h-1 ) p-i x n+k+ (h-1 ) p-j I h= 1 Let the prior density of a,b be a-I for a > (I. Then (npv) (N-n-np) f (bk-b:) ' Q ~  (bk-b:) has a posterior F k= 1 np , N-n-np distribution, and [n (p-1 ) v] [N-n (p+l ) 3 A t H A  has a posterior F n(p-1) ,N-n(p+l) distribution, where Similar assertions are also derived for the model with periodic variances. GENERALIZATIONS Model (x) can be genera l lzed  t o  a  model wi th  exogenous and endogenous v a r i a b l e s *  The computations a r e  s l i g h t l y  e a s i e r  when we write t h e  abso lu te  term (pk) sepa ra te ly .  The model r eads  Quite  analogously w e  can g e t  a mul t ip le  model f o r  t h e  case  t h a t  Xt a r e  random vectors .  PROBLEMS OF STRUCTURAL CHANGES For reasons of s i m p l i c i t y ,  we only  d l s c u s s  he re  t h e  main i d e a s  i n  t h e  f i e l d  of c l a s s i c a l  A'R models. Their  genera l iza-  t i o n  t o  a  p e r i o d i c  AR model is then obvious. L e t  Xt be c r e a t e d  by t h e  model 2 where Yt a r e  independent v a r i a b l e s ,  Yt - N ( 0 , ~ ~ )  . W e  assume t h a t  i n  some unknownmoment, t = 8 ,  t h e r e  can be a change of parameters: Some methods for the detection of point 8 are described by Segen and Sanderson (1980) and by Basseville and Benveniste (1983). Their procedures are Based on the cumulative sum (CUSUM) technique. Simulations show, howwer, that the CUSUM method is in many cases not sensit2ve enough. Another promising model for detecting structural changes can be the model with exogenous and endogenous variables, e.g. where The problem is to estimate parameters p, bl, ..., bn, a, 2 8, and a = var Yt. The change occurs in tine t = 8 ,  mainly influencing the mean value of the process. This formulation reminds of the piece-wise linear regression with unknown break- points, where the solution is often Based on the maximum likelihood method. Both cases mentioned here can be generalized to more complicated models. Further research in th2s field is neces- sary, because the problem is interesting and important. It is clear that the theoretical results should be complemented by available computer programs. REFERENCES Andel, J. (1983). Statistical analysis of periodic auto- regression. Appl. Mat. 28:364-385. Basseville, M., and A. Benveniste (19831. Sequential detection of abrupt changes in spectral characteristics of digital signals. IEEE Trans. of Inf. Th. IT-29:709-724. Box, G.E.P., and G.M. Jenkins (1970). Time Series Analysis. Forecasting and Control. San Francisco: Holden-Day. Cleveland, W.P., and G.C. Tiao (1979). Modeling seasonal time series. Rev. Economie Appllquee 32:107-129. DeGroot, M.H. (1970). Optimal Statistical Decisions. New York: Mc Graw-Hill. Gladyshev,E.G. (1961). Periodically correlated random sequences. Soviet Math. 2:385-388. Jones, R.H., and W.M. Brelsford (1967). Time series with periodic structure. Biometrika 54:403-408. Newton, H.J. (1982). Using periodic autoregression for multiple spectral estimation. Technometrics 24:109-116. Pagano, M. (1978). On periodic and mulitple autoregression. Ann. Statist. 6:1310-1317. Segen, J., and A.C. Sanderson (1980). Detecting change in a time series. IEEE Trans. on Tnf. Th. IT-26:249-255. Tiao, C.G., and M.R. Grupe (1980). Hidden periodic auto- regressive-moving average models tn time series data. Biometrika 67:365-373. Troutman, B.M. (1979). Some results in periodic autoregression. Biometrika 66:219-228. 

Bayesian Estimates in Time-Series Models

Abstract

Similar works

Full text

Available Versions

International Institute for Applied Systems Analysis (IIASA)