Detecting and Predicting Forecast Breakdowns

Abstract

We propose a theoretical framework for assessing whether a forecast model estimated over one period can provide good forecasts over a subsequent period. We formalize this idea by defining a forecast breakdown as a situation in which the out-of-sample performance of the model, judged by some loss function, is significantly worse than its in-sample performance. Our framework, which is valid under general conditions, can be used not only to detect past forecast breakdowns but also to predict future ones. We show that main causes of forecast breakdowns are instabilities in the data generating process and relate the properties of our forecast breakdown test to those of existing structural break tests. The main differences are that our test is robust to the presence of unstable regressors and that it has greater power than previous tests to capture systematic forecast errors caused by recurring breaks that are ignored by the forecast model. As a by-product, we show that our results can be applied to forecast rationality tests and provide the appropriate asymptotic variance estimator that corrects the size distortions of previous forecast rationality tests. The empirical application finds evidence of a forecast breakdown in the Phillipsí curve forecasts of U.S. inflation, and links it to inflation volatility and to changes in the monetary policy reaction function of the Fed.