Banca d'Italia)

Hashem Pesaran

Paolo Zaffaroni

English

Research Papers in Economics

Model Averaging and Value-at-Risk based
Evaluation of Large Multi Asset Volatility
Models for Risk Management∗
M. Hashem Pesaran
University of Cambridge and USC
Paolo Zaﬀaroni
University of Cambridge and Banca d´Italia
March 3, 2004
Abstract
This paper considers the problem of model uncertainty in the case
of multi-asset volatility models and argues in favour of the use of
Bayesian-type model averaging techniques to deal with the risk of
inadvertently using false models in portfolio management. In particu-
lar, it is shown that portfolio returns based on an average of Gaussian
models with diﬀerent volatilities is more fat-tailed than an individual
Gaussain model with the same average volatility measure. Evalua-
tion of volatility models is also considered and a simple Value-at-Risk
(VaR) diagnostic test is proposed for individual as well as ‘average’
models and its exact and asymptotic properties are established. The
model averaging idea and the VaR diagnostic tests are illustrated by
an application to portfolios of daily returns based on twenty two of
Standard & Poor’s 500 industry group indices over the period January
2, 1995 to October 13, 2003, inclusive.
JEL Classiﬁcations: C32, C52, C53, G11
Key Words: Model Averaging, Value-at-Risk, Decision Based
Evaluations.
∗We would like to thank Vanessa Smith for her assistance with the data used in this
study.
11 Introduction
Multivariate models of conditional volatility are of crucial importance for
optimal asset allocation, risk management, derivative pricing and dynamic
hedging. Yet there are few published empirical studies of the performance
of multivariate volatility models as applied to portfolios with a relatively
large number of assets. Although, many alternative parametric and semi-
parametric multivariate volatility models have been advanced in the aca-
demic literature, until recently most have been limited in the number of
assets that they can handle. In an attempt to provide operationally feasible
volatility models for the analysis of portfolios with a large number of assets,
many investigators (in ﬁnancial markets as well as in academia) have fo-
cused on highly restricted versions of multivariate generalized autoregressive
conditional heteroscedastic (GARCH) speciﬁcation. These include the con-
ditionally constant correlation (CCC) model of Bollerslev (1990), the Risk-
metrics speciﬁcations popularized by J.P.Morgan/Reuters (1996), and used
predominantly by practitioners, the orthogonal GARCH model of Alexander
(2001), and more the dynamic conditional correlation (DCC) advanced by
Engle (2002). Multivariate volatility models have also been considered in
the stochastic volatility (SV) literature. See Ghysels, Harvey, and Renault
(1995) and Shephard (2004) for reviews. However, so far the focus of this
literature has been on univariate and multivariate models with a small num-
ber of assets, with the notable exceptions of Diebold and Nerlove (1989),
Engle, Ng, and Rothschild (1990), King, Sentana, and Wadhwani (1994) and
Harvey, Ruiz, and Shephard (1994). These models are similar in structure
to a class of factor orthogonal GARCH model discussed below.
The highly restricted nature of the multivariate volatility models could
present a high degree of model uncertainty which ought to be recognized
at the outset. This is particularly important since due to data limitations
and operational considerations it is not possible to subject these models to
rigorous statistical testing either. Application of model selection procedures
also face additional diﬃculties when the number of assets is moderately large,
and might very well be that no single model choice would be satisfactory in
practice and could carry risks that are diﬃcult to asses a priori.
In this paper we advocate the use of model averaging in order to minimize
the risk associated with model uncertainty. We apply recent developments
in model evaluation and model averaging techniques to multi-asset volatil-
ity models, in the case where the number of assets under consideration is
relatively large. Based on this framework, we develop simple criteria for
evaluation of alternative volatility forecasts by examining the Value-at-Risk
(VaR) performance of their associated portfolios over an evaluation period.
The approach is general and can be applied to strategic asset allocation prob-
lems that require volatility forecasts over relatively long periods as well as
more traditional VaR problems with horizons ranging from a single day to a
month. Our method is relevant both for evaluation of existing portfolios as
2well as for the construction of eﬃcient portfolios.
The probability forecast combination approach also attempts to avoid
the pre-testing problem associated with the standard two-stage procedure
where the decision problem is based on a probability model selected as the
‘best’ from a given set of candidate models according to a suitable criteria.
Frequently used selection criteria are Akaike Information Criteria (AIC) and
the Schwartz Bayesian Information Criteria (SBC). However, such a two-
step procedure is subject to the pre-test (selection) bias problem and tends
to under-estimate the uncertainty that surrounds the forecast. Second, usu-
ally point forecasts are analyzed and the evaluation of their out-of-sample
performance is based on standard metrics such as root mean square forecast
errors (RMSFE). However, this approach runs into diﬃculties when consider-
ing volatility models. In fact, volatility is not directly observable and is often
proxied by square of daily returns or more recently by the standard error
of daily returns using intra-daily observations, known as realized volatility
(see, for example, Andersen, Bollerslev, Diebold, and Labys (2003)). In
multi-asset contexts the use of standard metrics such as RMSFE is further
complicated by the need to select weights to be attached to diﬀerent types
of errors in forecasts of individual asset volatilities and their cross-volatility
correlations and choice of such weights is not innocuous in a multivariate
framework (see Pesaran and Skouras (2002)).
The use of model averaging techniques in econometrics is not new and
dates back to the seminal work of Granger and Newbold (1977) on forecast
combination.1 However, this literature focusses on combining point forecasts
and does not address the problem of combining forecast probability distri-
bution functions which is relevant in the risk management literature. The
problem is not trivial even in the simplest case of average of normal distribu-
tions with diﬀerent parameters since the average model will not have a normal
distribution, except when for the degenerate case of zero-one coeﬃcients.
From a methodological perspective, following Granger and Pesaran (2000b),
our approach aims to represent a more uniﬁed treatment of the empirical
portfolio analysis from a decision-theoretic perspective rather than from a
merely statistical one. Granger and Pesaran (2000a) clarify the importance
of concentrating on probability forecasts rather than on just event forecasts.
Within a risk-management perspective the motivations for doing so appear
even stronger since the ultimate goal is not simply to ﬁnding the best approx-
imating volatility model but how to best approximate the entire predictive
density of asset returns, or at least its tail behaviour.
The remainder of the paper is organised as follows: the decision problem
that underlies the VaR analysis and the associated diagnostic tests is set out
in Section 2. Section 3 provides a brief outline of the diﬀerent types of multi-
variate volatility models considered in the paper. Bayesian and non-Bayesian
1For reviews of the forecast combination literature see Clemen (1989), Granger (1989),
Diebold and Lopez (1996) and Hendry and Clements (2002).
3approaches to model averaging are reviewed and discussed in Section 4. Sec-
tion 5 introduces a simple Value-at-Risk (VaR) diagnostic test and establishes
its distribution. Section 6 provides a detailed empirical analysis using daily
returns on twenty two of Standard and Poor’s 500 industry indices over the
period January 2 1995 to October 13 2003. Section 7 concludes with a sum-
mary of the main results and provides suggestions for future research. The
mathematical proofs are collected in the Appendix.
4References
Aiolfi, M., and C. Favero (2002): “Model uncertainty, thick modelling and
the predictability of stock returns,” IGIER Working Paper 221, IGIER.
Aiolfi, M., C. Favero, and G. Primiceri (2001): “Recursive ‘thick’ modeling
of excess returns and portfolio allocation,” IGIER Working Paper 197, IGIER.
Alexander, C. (2001): “Orthogonal GARCH,” in Mastering risk, ed. by
C. Alexander, vol. 2, pp. 21–38. London: Financial Times - Prentice Hall.
Andersen, A., T. Bollerslev, F. Diebold, and P. Labys (2003): “Modeling
and forecasting realized volatility,” Econometrica, 71, 579–626.
Billingsley, P. (1986): Probability and measure. New York: Wiley, second edn.
Bollerslev, T. (1986): “Generalized autoregressive conditional heteroskedastic-
ity,” Journal of Econometrics, 31, 302–327.
Bollerslev, T. (1990): “Modelling the coherence in short-run nominal exchange
rates: a multivariate generalized ARCH,” Review of Economics and Statistics,
72, 498–505.
Bollerslev, T., R. Engle, and J. Wooldridge (1988): “A capital asset
pricing model with time varying covariances,” Journal of Political Economy,
96, 116–131.
Brown, B. (1971): “Martingale central limit theorems,” The Annals of Mathe-
matical Statistics, 42, 59–66.
Burnham, K., and D. Anderson (1998): Model Selection and Inference: A
Practical Information-Theoretic Approach. New York: Springer Verlag.
Clemen, R. (1989): “Combining forecasts: a review and annotated bibliography,”
International Journal of Forecasting, 5, 559–583.
De Santis, G., and B. Gerard (1997): “International asset pricing and port-
folio diversiﬁcation with time-varying risk,” Journal of Finance, 52, 1881–1912.
De Santis, G., R. Litterman, A. Vesval, and K. Winkelmann (2003):
“Covariance matrix estimation,” in Modern investment management: an equi-
librium approach, ed. by R. Litterman. London: Wiley.
Diebold, F., T. Gunther, and A. Tay (1998): “Evaluating density forecasts
with application to ﬁnancial risk management,” International Economic Review,
39, 863–883.
Diebold, F., J. Hahn, and A. Tay (1999): “Multivariate density forecast eval-
uation and calibration in ﬁnancial risk management: high-frequency returns on
foreign exchange,” The Review of Economics and Statistics, 81, 661–673.
Diebold, F., and J. Lopez (1996): “Forecast evaluation and combination,”
in Handbook of Statistics, ed. by G. G.S. Maddala,and C. Rao, pp. 214–268.
Amsterdam: North Holland.
31Diebold, F., and M. Nerlove (1989): “The dynamics of exchange rate volatil-
ity: a multivariate latent factor ARCH model,” Journal of Applied Economet-
rics, 4, 1–21.
Draper, D. (1995): “Assessment and propagation of model uncertainty (with
discussion),” Journal of the Royal Statistical Society, Series B, 57, 45–97.
Engle, R. (2002): “Dynamic conditional correlation - a simple class of multivari-
ate generalized autoregressive conditional heteroskedasticity models,” Journal
of Business & Economic Statistics, 20, 339–350.
Engle, R., V. Ng, and M. Rothschild (1990): “Asset pricing with a
FACTOR-ARCH covariance structure: empirical estimates for treasury bills,”
Journal of Econometrics, 45, 213–237.
Fernandez, C., E. Ley, and M. F. J. Steel (2001a): “Benchmark priors for
bayesian model averaging,” Journal of Econometrics, 100, 381–427.
Fernandez, C., E. Ley, and M. F. J. Steel (2001b): “Model uncertainty in
cross-country growth regressions,” Journal of Applied Econometrics, 16, 563–
576.
Garratt, A., K. Lee, M. H. Pesaran, and Y. Shin (2003): “Forecast uncer-
tainties in macroeconometric modelling: an application to the UK economy,”
Journal of the American Statistical Association , forthcoming.
Ghysels, E., A. Harvey, and E. Renault (1995): “Stochastic Volatility,” in
Handbook of Statistics vol 14. Amsterdam: North Holland.
Granger, C. (1989): “Combining forecasts - Twenty years later,” Journal of
Forecasting, 8, 167–173.
Granger, C., and Y. Jeon (2004): “Thick modeling,” Economic Modeling, 21,
323–343.
Granger, C., and P. Newbold (1977): Forecasting economic time series. New
York; London: Academic Press.
Granger, C., and M. Pesaran (2000a): “A decision theoretic approach to
forecast evaluation,” in Statistics and Finance: An Interface, ed. by W. Chan,
W. Li,and H. Tong, pp. 261–278. London: Imperial College Press.
Granger, C., and M. Pesaran (2000b): “Economic and statistical measures
of forecast accuracy,” Journal of Forecasting, 19, 537–560.
Harvey, A., E. Ruiz, and N. Shephard (1994): “Multivariate Stochastic Vari-
ance Models,” Review of Economic Studies, 61, 247–264.
Hendry, D., and M. Clements (2002): “Pooling of forecasts,” Econometrics
Journal, 5, 1–26.
Hoeting, J. A., D. Madigan, A. E. Raftery, and C. T. Volinsky (1999):
“Bayesian model averaging: a tutorial,” Statistical Science, 14, 382–417.
32J.P.Morgan/Reuters (1996): “RiskMetricsTM - Technical Document, Fourth
Edition,” .
King, M., E. Sentana, and S. Wadhwani (1994): “Volatility and links between
national stock markets,” Econometrica, 62, 901–933.
Ledoit, O., P. Santa-Clara, and M. Wolf (2003): “Flexible multivariate
GARCH modeling with an application to international stock markets,” Review
of Economics and Statistics forthcoming.
Ling, S., and M. McAleer (2003): “Asymptotic theory for a vector ARCH-
GARCH model,” Econometric Theory, 19, 280–310.
Lopez, J. (1999): “Methods for evaluating value-at-risk estimates,” FRBSF Eco-
nomic Review, 2.
McCracken, M. (2000): “Robust out-of-sample inference,” Journal of Econo-
metrics, 99, 195–223.
Nelson, D., and C. Cao (1992): “Inequality constraints in the univariate
GARCH model,” Journal of Business & Economic Statistics, 10, 229–235.
Pesaran, M., and S. Skouras (2002): “Decision-based methods for forecast
evaluation,” in A Companion to Economic Forecasting, ed. by M. Clements,and
D. Hendry, pp. 241–267. Oxford: Basil Blackwell.
Pesaran, M., and A. Timmermann (1995): “Predictability of stock returns:
robustness and economic signiﬁcance,” Journal of Finance, 50, 1201–1228.
Rosenblatt, M. (1952): “Remarks on a multivariate transformation,” Annals of
Mathematical Statistics, 23, 470–472.
Shephard, N. (2004): Stochastic volatility: selected readings. Oxford: Oxford
University Press, forthcoming.
Sheppard, K. (2002): “UCSD GARCH Toolbox, Version 2.0.4 19-JUN-2002,” .
Stephens, M. (1970): “Use of Kolmogorov-Smirnov, Cramer-Von Mises and re-
lated statistics without extensive tables,” Journal of the Royal Statistical Soci-
ety, Series B (Methodological), 32, 115–122.
West, K. (1996): “Asymptotic inference about predictive ability,” Econometrica,
64, 1067–1084.
Zaffaroni, P. (2003): “Estimating and forecasting volatility with large scale
models: theoretical appraisal of professionals’ practice,” Preprint .
33

Model Averaging and Value-at-Risk based Evaluation of Large Multi Asset Volatility Models for Risk Management

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics