Using well-known GARCH models for density prediction of daily S&P 500 and Nikkei 225 index returns, a comparison is provided between frequentist and Bayesian estimation. No significant difference is found between the qualities of the forecasts of the whole density, whereas the Bayesian approach exhibits significantly better left-tail forecast accuracy

Corre, N. (Nienke)

David, D. (David)

Hoogerheide, L.F. (Lennart)

Erasmus University Digital Repository

TI 2011-020/4 Tinbergen Institute Discussion Paper   Stock Index Returns' Density Prediction using GARCH Models: Frequentist or Bayesian Estimation?   Lennart F. Hoogerheidea David Ardiab Nienke Corré  a Tinbergen and Econometric Institutes, Erasmus University Rotterdam, The Netherlands; b aeris CAPITAL AG, Switzerland.   Tinbergen Institute is the graduate school and research institute in economics of Erasmus University Rotterdam, the University of Amsterdam and VU University Amsterdam.  More TI discussion papers can be downloaded at http://www.tinbergen.nl  Tinbergen  Institute has two locations:  Tinbergen Institute Amsterdam Gustav Mahlerplein 117 1082 MS Amsterdam The Netherlands Tel.: +31(0)20 525 1600  Tinbergen Institute Rotterdam Burg. Oudlaan 50 3062 PA Rotterdam The Netherlands Tel.: +31(0)10 408 8900 Fax: +31(0)10 408 9031  Duisenberg school of finance is a collaboration of the Dutch financial sector and universities, with the ambition to support innovative research and offer top quality academic education in core areas of finance. DSF research papers can be downloaded at: http://www.dsf.nl/  Duisenberg school of finance Gustav Mahlerplein 117 1082 MS Amsterdam The Netherlands Tel.: +31(0)20 525 8579   Stock index returns' density prediction using GARCH models:Frequentist or Bayesian estimation?Lennart F. Hoogerheidea,, David Ardiab, Nienke CorreaTinbergen and Econometric Institutes, Erasmus University Rotterdam, The Netherlandsbaeris CAPITAL AG, SwitzerlandAbstractUsing well-known GARCH models for density prediction of daily S&P 500 and Nikkei 225 indexreturns, a comparison is provided between frequentist and Bayesian estimation. No signicant dif-ference is found between the qualities of the forecasts of the whole density, whereas the Bayesianapproach exhibits signicantly better left-tail forecast accuracy.JEL classication: C11, C52, C53, C58.Keywords: GARCH, Bayesian, KLIC, censored likelihood1. IntroductionThe modeling of volatility has been at the heart of nancial econometrics for decades. One of itscorner-stones is the GARCH model of Bollerslev (1986). Popular extensions include the ExponentialGARCH (EGARCH) model by Nelson (1991) and the GJR model by Glosten et al. (1993).The common approach for estimation in GARCH-type models is Maximum Likelihood (ML). Theuse of ML is appealing because of the intuitive principle underlying the approach and its ease of im-plementation. Nevertheless, ML estimation entails some diculties in practice due to the numericaloptimization which involves inequality constraints. The Bayesian approach oers an attractive alter-native which enables small sample results, robust estimation, model discrimination and probabilisticstatements on (possibly nonlinear functions of) model parameters (see Ardia, 2008).The comparison of frequentist and Bayesian GARCH has received little attention in research. Ina study by Nakatsuma and Tsurumi (1996) some attention is paid to the comparison. Based on asmall sample Monte Carlo experiment, they conclude that the Bayesian approach performs betterthan the frequentist approach in terms of smaller mean square errors of the posterior mean versusthe MLE of the parameters in an ARMA-GARCH model. Their comparison is however not extendedto an empirical application or to the evaluation of the prediction of returns' densities.This note aims at comparing the performance of well-known GARCH models via frequentist andBayesian approaches. We evaluate the forecasting performance of the GARCH, GJR and EGARCHmodel using index data. The performance of the models is evaluated based on the relative predictiveaccuracy; both the entire density and the left tail, which is more relevant for risk managementpurposes, are considered. Our results indicate that the Bayesian approach outperforms its frequentistcounterpart when forecasting the left-tail of the return's distribution.Corresponding author. Telephone: +31104081360. Fax: +31104089162 Address: P.O. Box 1738, NL-3000 DRRotterdam, The Netherlands.Email addresses: lhoogerheide@ese.eur.nl (Lennart F. Hoogerheide), da@aeris-capital.com (David Ardia)Preprint submitted to SSRN January 31, 20112. Frequentist and Bayesian estimation of GARCH modelsWe consider the GARCH(1,1), GJR(1,1) and EGARCH(1,1) models, with Student-t innovationsto account for conditional excess kurtosis (see Geweke and Amisano, 2010). The mean equation isdened asyt = + "t t = 1; 2; : : : ; T"tjIt 1  S(0; ht  2 ; ) ;(1)with yt the log-return at time t and where S is used for the Student-t distribution. The sequence ofinnovations, given past information It 1, follows a Student-t distribution with mean zero, varianceht and  degrees of freedom. The restriction  > 2 guarantees existence of the conditional variance.In the GARCH(1,1) model the conditional variance ht is a linear function of the past squaredinnovation and conditional variance:ht = ! +  "2t 1 +  ht 1 : (2)The GJR(1,1) model is similar, except for the indicator variable added to allow for asymmetric eectsdepending on the sign of the innovation (i.e., typically to allow for a larger eect of a negative shockthan a positive shock of the same magnitude):ht = ! + 1 If"t 10g + 2 If"t 1<0g"2t 1 +  ht 1 : (3)The indicator variable Ifg takes on a value of one if the condition holds and zero otherwise. Finally,the EGARCH(1,1) model is given by:log(ht) = ! + j"t 1jpht 1+ "t 1pht 1+  log(ht 1) ; (4)where the term "t 1=pht 1 is known as the standardized residual; the occurrence of this term withoutj:j allows for the asymmetric behavior of the conditional variance.For the GARCH and GJR models the positivity constraints imposed on the model parameters inthe variance equation are ! > 0, ;   0.In what follows, we regroup the model parameters into  and the observed log-returns into y =(y1; : : : ; yT )0. The likelihood function is given by p(yj) which is the product of the conditional densityvalues at yt.In the frequentist approach  is assumed to be a xed unknown constant, which is typically esti-mated by ML. The Bayesian approach incorporates the likelihood in a broader framework. Parametersin  are assumed to be stochastic variables described by a posterior density p(jy) given by Bayes'theorem asp(jy) = p(yj)p()Rp(yj)p()d ; (5)where p() is the prior density for , reecting the prior beliefs before having observed the data.For the parameters , !,  and  we use non-informative at (uniform) prior on the parameter do-main. For , we use a non-informative proper prior (as an improper prior would yield an improperposterior, see Bauwens and Lubrano (1998)): a translated exponential density as in Geweke (1993);   exp( ( 2)) with  = 0:05. We adopt an independence chain Metropolis-Hastings algorithm(with Student-t candidate distribution around the posterior mode, with scale matrix equal to minusthe inverse Hessian of the log-posterior at the mode, and four degrees of freedom for fat tails) todraw random samples from the posterior density: 25000 draws after a burn-in of 5000 draws. Ac-ceptance rates were always around 50%, indicating the good performance of the posterior simulator.Prediction results under a more exible candidate distribution, the mixture of Student-t distributionsof Hoogerheide et al. (2007), were qualitatively the same. Further, results stay almost the same forsomewhat dierent values of .23. Tests for dierence in forecasting performanceThis research focuses on one-day-ahead density forecasts for daily returns rather than point fore-casts. These density forecasts provide an estimate of the probability distribution of future returns.Our objective is to nd the density forecast that comes closest to the true but unobserved density fordaily returns by comparing the relative performance of two competing density forecasts denoted byf^t(yt+1) and g^t(yt+1). Given (1) the forecast densities can be expressed asyt+1j;M; It  S(; ht+1; ) ; (6)where the forecast density is conditional on  and modelM. In the frequentist approach we simply usethe ML estimator ^MLE, implicitly ignoring parameter uncertainty. On the other hand, the Bayesianapproach takes into account parameter uncertainty, which is integrated out: the Bayesian densityforecast is obtained as the mean of the density forecasts corresponding to the posterior draws.To compare the performance of the dierent forecast densities we follow Mitchel and Hall (2005)and Diks et al. (2008) and use the well-known Kullback-Leibler information criterion (KLIC). Usingthe KLIC a loss dierential series is constructed with dt+1 = ln g^t(yt+1)   ln f^t(yt+1), which can beinterpreted as the dierence in the score of both models. Obviously, a high score is preferred as thisimplies that the model assigns a high `probability' to the observed value. Given a sequence of densityforecasts and corresponding realizations for the observed return, densities are compared based on theaverage score d. Under the null hypothesis of equal forecast accuracy the loss dierential should bezero on average. This can be tested by a Diebold and Mariano (1995) type test statistic with robustNewey-West standard error.We also consider the evaluation of forecast performance in the left-tail of the forecast density,which is of special interest for risk management applications. For testing, we rely on the censoredlikelihood (CSL) scoring rule by Diks et al. (2008). For a particular model the CSL equals the totalnon-tail probability (1  F^t(r)) if yt+1 is not a tail event and the forecasted density value f^t(yt+1) ifyt+1 is a tail event. Hence the CSL score function is obtained asCSL(f^tjyt+1) = Ifyt+1rg ln f^t(yt+1) + Ifyt+1>rg ln(1  F^t(r)) ; (7)where F^t(r) is the Student-t cumulative density at time t evaluated at the threshold r. This termimplies that the shape of the forecasted density is ignored for observations outside the region ofinterest. To evaluate the CSL of competing models a procedure similar to the evaluation procedurebased on KLIC scores is employed. This left-tail test is a natural extension of the well-known KLIC-based test for the whole density forecasts (for r ! 1 it obviously reduces to the same test). TheCDF term Ifyt+1>rg ln(1   F^t(r)) is required, since its exclusion would yield a criterion that wouldalways favor density forecasts with a fatter left-tail, even if the true density were thin-tailed like thenormal distribution. Note that a proper threshold value r needs to be specied, determining whichobservations pertain to the left tail. For a fair evaluation, the number of pdf-terms and CDF-termsincluded in the CSL should be the same across all competing models such that a model-independentthreshold is required. Though the threshold is allowed to be time-varying, the threshold is xed at-2.5% in this study, which is the 7.7% to 8.8% quantile of the unconditional return distribution duringthe out-of-sample period for the S&P 500 and Nikkei 225.4. Results and conclusionsLog-returns of the S&P 500 and Nikkei 225 stock indices are examined. Models are estimatedusing a ten-year rolling window. The out-of-sample period consists of 600 daily forecasts, rangingfrom January 1, 2008, to April 19, 2010.The top panel of Table 1 reports the results of the pairwise forecast accuracy comparison of com-peting density forecasts as measured by the KLIC. Superior predictive accuracy of the model/methodin the row (as compared with the model/method in the column) is indicated by a positive value.Our primary interest lies in the comparison of the frequentist and Bayesian approach. Across allmodels and datasets the test statistic is not signicant, implying that the null hypothesis of equalpredictive accuracy of the frequentist and Bayesian approach cannot be rejected.3Table 1: Diebold and Mariano (1995) t-statistic (using Newey-West standard error) corresponding to average loss dierentialfor KLIC (whole density) or CSL (left tail of density), based on 600 density forecasts of the GARCH(1,1), GJR(1,1) andEGARCH(1,1) models for daily log-returns of the S&P 500 and Nikkei 225. * and ** indicates (two-sided) signicance at 5%and 1% respectively. Positive (negative) values signify better forecast performance of the model specied in the correspondingrow (column).Frequentist (MLE) Bayesian (MH)GARCH GJR EGARCH GARCH GJR EGARCHKLICS&P 500Frequentist (MLE) GARCH -2.632* -1.602 1.007GJR 2.632* 0.409 1.685EGARCH 1.602 -0.409 -0.881Bayesian (MH) GARCH -1.007 -2.376* -4.053**GJR -1.685 2.376* -2.732**EGARCH 0.881 4.053** 2.732**Nikkei 225Frequentist (MLE) GARCH -2.622** -0.477 -0.041GJR 2.622** 0.220 0.373EGARCH 0.477 -0.220 -0.494Bayesian (MH) GARCH 0.041 -2.033* -2.011*GJR -0.373 2.033* -0.408EGARCH 0.494 2.011* 0.408CSLS&P 500Frequentist (MLE) GARCH -2.143* -0.373 -5.274**GJR 2.143* 1.454 -5.480**EGARCH 0.373 -1.454 -4.227**Bayesian (MH) GARCH 5.274** -1.686 -0.798GJR 5.480** 1.686 0.246EGARCH 4.227** 0.798 -0.246Nikkei 225Frequentist (MLE) GARCH -1.506 -0.007 -5.277**GJR 1.506 0.243 -5.054**EGARCH 0.007 -0.243 -5.422**Bayesian (MH) GARCH 5.277** -1.743 -0.875GJR 5.054** 1.743 0.916EGARCH 5.422** 0.875 -0.916The testing approach also oers a straightforward framework to compare the predictive accuracyof the dierent GARCH-type models. The results indicate that the EGARCH and GJR modelsperform better (signicantly in the Bayesian framework) than the standard GARCH model.The bottom panel of Table 1 reports results for the left-tail forecast accuracy quantied by theCSL. The null hypothesis of equal predictive accuracy of the frequentist and Bayesian approach in theleft tail is convincingly rejected for all models and datasets in favor of the Bayesian approach. Thedierence is mainly caused by the rst term on the right-hand side of (7). Our nding makes intuitivelysense: Bayesian inference has been often found to be more appropriate in small samples, where itis more important that the present parameter uncertainty is taken into account. Although we havemore than 2500 observations, we still have few observations from the (extreme) left tail. Especiallythe degrees of freedom parameter  may be strongly inuenced by a few extreme observations inthe estimation window. In addition, note that the dierences in CSL between the models are lesspronounced (if any dierence exists) than the dierences between frequentist and Bayesian estimation,which further stresses the clearness of the superiority of Bayesian inference in this application.Our results suggest that in risk management applications the Bayesian approach should be pre-ferred, because of its superior predictive accuracy in the left tail. This result is remarkable, since4we use large data sets (over 2500 observations) and non-informative priors. For smaller data sets orinformative priors the dierence between Bayesian and frequentist inference may be even larger. Infuture research we will focus on dierent data sets and priors, alternative model specications (e.g.,dierent error distributions such as the skewed t distribution), model combination (see Ardia andHoogerheide, 2010), and multi-step-ahead forecasting. In the latter case, the importance samplingmethod of Hoogerheide and Van Dijk (2010) may provide a substantial reduction of the computingtime.AcknowledgementsThe authors acknowledge Guido Bolliger, Michel Dubois, Dusan Isakov, Michael Rockinger andDick van Dijk for useful comments. The views expressed in this paper are the sole responsibility ofthe authors and do not necessarily reect those of aeris CAPITAL AG or any of its aliates. Anyremaining errors or shortcomings are the authors' responsibility.ReferencesArdia, D., 2008. Financial Risk Management with Bayesian Estimation of GARCH Models: Theory and Applications.Vol. 612 of Lecture Notes in Economics and Mathematical Systems. Springer-Verlag, Berlin, Germany.Ardia, D., Hoogerheide, L. F., 2010. Ecient Bayesian estimation and combination of GARCH-type models. In: Bocker,K. (Ed.), Rethinking Risk Measurement and Reporting - Volume II: Examples and Applications from Finance. RiskBooks, London, Ch. 1.Bauwens, L., Lubrano, M., 1998. Bayesian inference on GARCH models using the Gibbs sampler. Econometrics Journal1, C23{C46.Bollerslev, T., 1986. Generalized autoregressive conditional heteroskedasticity. Journal of Econometrics 31 (3), 307{327.Diebold, F. X., Mariano, R. S., 1995. Comparing predictive accuracy. Journal of Business & Economic Statistics 13 (3),253{263.Diks, C., Panchenko, V., Van Dijk, D., 2008. Partial likelihood-based scoring rules for evaluating density forecasts intails. Tinbergen Institute Discussion Paper TI 2008-050/4.Geweke, J., Amisano, G., 2010. Comparing and evaluating Bayesian predictive distributions of asset returns. Interna-tional Journal of Forecasting 26, 216{230.Geweke, J. F., 1993. Bayesian treatment of the independent Student-t linear model. Journal of Applied Econometrics8 (S1), S19{S40.Glosten, L. R., Jaganathan, R., Runkle, D. E., 1993. On the relation between the expected value and the volatility ofthe nominal excess return on stocks. Journal of Finance 48 (5), 1779{1801.Hoogerheide, L. F., Kaashoek, J. F., Van Dijk, H. K., 2007. On the shape of posterior densities and credible sets ininstrumental variable regression models with reduced rank: An application of exible sampling methods using neuralnetworks. Journal of Econometrics 139 (1), 154{180.Hoogerheide, L. F., Van Dijk, H. K., 2010. Bayesian forecasting of Value at Risk and Expected Shortfall using adaptiveimportance sampling. International Journal of Forecasting 26, 231{247.Mitchel, J., Hall, S. G., 2005. Evaluating, comparing and combining density forecasts using the KLIC with an applicationto the Bank of England and NIESR `fan' charts of ination. Oxford Bulletin of Economics and Statistics 67, 995{1033.Nakatsuma, T., Tsurumi, H., 1996. ARMA-GARCH models: Bayes estimation versus MLE, and Bayes non-stationaritytest. Working Paper No. 9619, Department of Economics, Rutgers University, New Jersey.Nelson, D. B., 1991. Conditional heteroskedasticity in asset returns: A new approach. Econometrica 59 (2), 347{370.5

Stock Index Returns' Density Prediction using GARCH Models: Frequentist or Bayesian Estimation?

http://repub.eur.nl/pub/22344/2011-0204.pdf

Stock Index Returns' Density Prediction using GARCH Models: Frequentist or Bayesian Estimation?

Abstract

Similar works

Full text

Available Versions

Erasmus University Digital Repository