Calibration tests for multivariate Gaussian forecasts

Forecasts by nature should take the form of probabilistic distributions. Calibration, the statistical consistency of forecast distributions and observations, is a central property of good probabilistic forecasts. Calibration of univariate forecasts has been widely discussed, and significance tests are commonly used to investigate whether a prediction model is miscalibrated. However, calibration tests for multivariate forecasts are rare. In this paper, we propose calibration tests for multivariate Gaussian forecasts based on two types of the Dawid–Sebastiani score (DSS): the multivariate DSS (mDSS) and the individual DSS (iDSS). Analytic results and simulation studies show that the tests have sufficient power to detect miscalibrated forecasts with incorrect mean or incorrect variance. But for forecasts with incorrect correlation coefficients, only the tests based on mDSS are sensitive to miscalibration. As an illustration, we apply the methodology to weekly data on Norovirus disease incidence among males and females in Germany, in 2011–2014. The results further show that tests for multivariate forecasts are useful tools and superior to univariate calibration tests for correlated multivariate forecasts

Statistical post-processing of hydrological forecasts using Bayesian model averaging

Accurate and reliable probabilistic forecasts of hydrological quantities like runoff or water level are beneficial to various areas of society. Probabilistic state-of-the-art hydrological ensemble prediction models are usually driven with meteorological ensemble forecasts. Hence, biases and dispersion errors of the meteorological forecasts cascade down to the hydrological predictions and add to the errors of the hydrological models. The systematic parts of these errors can be reduced by applying statistical post-processing. For a sound estimation of predictive uncertainty and an optimal correction of systematic errors, statistical post-processing methods should be tailored to the particular forecast variable at hand. Former studies have shown that it can make sense to treat hydrological quantities as bounded variables. In this paper, a doubly truncated Bayesian model averaging (BMA) method, which allows for flexible post-processing of (multi-model) ensemble forecasts of water level, is introduced. A case study based on water level for a gauge of river Rhine, reveals a good predictive skill of doubly truncated BMA compared both to the raw ensemble and the reference ensemble model output statistics approach.Comment: 19 pages, 6 figure

Measuring output gap uncertainty

We propose a methodology for producing density forecasts for the output gap in real time using a large number of vector autoregessions in inflation and output gap measures. Density combination utilizes a linear mixture of experts framework to produce potentially non-Gaussian ensemble densities for the unobserved output gap. In our application, we show that data revisions alter substantially our probabilistic assessments of the output gap using a variety of output gap measures derived from univariate detrending filters. The resulting ensemble produces well-calibrated forecast densities for US inflation in real time, in contrast to those from simple univariate autoregressions which ignore the contribution of the output gap. Combining evidence from both linear trends and more flexible univariate detrending filters induces strong multi-modality in the predictive densities for the unobserved output gap. The peaks associated with these two detrending methodologies indicate output gaps of opposite sign for some observations, reflecting the pervasive nature of model uncertainty in our US data

Real-Time Multivariate Density Forecast Evaluation and Calibration: Monitoring the Risk of High-Frequency Returns on Foreign Exchange

We provide a framework for evaluating and improving multivariate density forecasts. Among other things, the multivariate framework lets us evaluate the adequacy of density forecasts involving cross-variable interactions, such as time-varying conditional correlations. We also provide conditions under which a technique of density forecast "calibration" can be used to improve deficient density forecasts. Finally, motivated by recent advances in financial risk management, we provide a detailed application to multivariate high-frequency exchange rate density forecasts. Copyright © 1998 F.X. Diebold, J. Hahn, and A.S. Tay. This paper is also available at

Forecasting and prequential validation for time varying meta-elliptical distributions

We consider forecasting and prequential (predictive sequential) validation of meta-elliptical distributions with time varying parameters. Using the weak prequential principle of Dawid, we conduct model validation avoiding nuisance parameter problems. Results rely on the structure of meta-elliptical distributions and we allow for discontinuities in the marginals and time varying parameters. We illustrate the ideas of the paper using a large data set of 16 commodity prices

Generation and Evaluation of Space-Time Trajectories of Photovoltaic Power

In the probabilistic energy forecasting literature, emphasis is mainly placed on deriving marginal predictive densities for which each random variable is dealt with individually. Such marginals description is sufficient for power systems related operational problems if and only if optimal decisions are to be made for each lead-time and each location independently of each other. However, many of these operational processes are temporally and spatially coupled, while uncertainty in photovoltaic (PV) generation is strongly dependent in time and in space. This issue is addressed here by analysing and capturing spatio-temporal dependencies in PV generation. Multivariate predictive distributions are modelled and space-time trajectories describing the potential evolution of forecast errors through successive lead-times and locations are generated. Discrimination ability of the relevant scoring rules on performance assessment of space-time trajectories of PV generation is also studied. Finally, the advantage of taking into account space-time correlations over probabilistic and point forecasts is investigated. The empirical investigation is based on the solar PV dataset of the Global Energy Forecasting Competition (GEFCom) 2014.Comment: 33 pages, 11 Figure

The past, present, and future of macroeconomic forecasting

Broadly defined, macroeconomic forecasting is alive and well. Nonstructural forecasting, which is based largely on reduced-form correlations, has always been well and continues to improve. Structural forecasting, which aligns itself with economic theory and, hence, rises and falls with theory, receded following the decline of Keynesian theory. In recent years, however, powerful new dynamic stochastic general equilibrium theory has been developed, and structural macroeconomic forecasting is poised for resurgence.Forecasting