In this paper, we present a comparison of the forecasting perfomance of selected static and dynamic factor models on two large monthly data panels. The first dataset contains EU variables, whereas the other contains US variables. These data panels are split into two parts: the first subsample (the calibration sample) is used to select the most performing specification for each class of models in a in- sample environment and the second subsample (the proper sample) is used to compare the performances of the selected models in an out-of-sample environment. In the calibration sample, genetic algorithms are employed to achieve an efficient exploration of the parameter space. We find that dynamic factor models are globally the most performing methods on both data panels

Della Marra, Fabio

English

Archivio Istituzionale della Ricerca

Macroeconomic forecasting: a non-standard optimisation approach to the calibration of dynamic factor models.

Archivio Ricerca Ca'Foscari

Archivio istituzionale della ricerca - Università degli Studi di Venezia Ca' Foscari

MACROECONOMIC FORECASTING: A NON-STANDARDOPTIMISATION APPROACH TO THE CALIBRATION OFDYNAMIC FACTOR MODELSFabio Della Marra 11 Department of Economics, University of Parma, (e-mail:fabio.dellamarra@iispascal.it)ABSTRACT: In this paper, we present a comparison of the forecasting perfomanceof selected static and dynamic factor models on two large monthly data panels. Thefirst dataset contains EU variables, whereas the other contains US variables. Thesedata panels are split into two parts: the first subsample (the calibration sample) isused to select the most performing specification for each class of models in a in-sample environment and the second subsample (the proper sample) is used to comparethe performances of the selected models in an out-of-sample environment. In thecalibration sample, genetic algorithms are employed to achieve an efficient explorationof the parameter space. We find that dynamic factor models are globally the mostperforming methods on both data panels.KEYWORDS: macreoconomic forecasting, dynamic factor models, genetic algorithms.1 IntroductionIn this paper, a comparative analysis of the forecasting performance of threeLarge-Dimensional Dynamic Factor Models is presented. As a key feature,Dynamic Factor Models represent each variable in a dataset as the sum oftwo orthogonal terms: a common component χt , driven by a reduced (as com-pared to the number of series in the dataset) number of common factors, andan idiosyncratic component ξt , which represents measurement errors or localfeatures. Among the different versions of the Dynamic Factor Models we se-lected:(i) SW model. This time-domain method was introduced in Stock & Watson,(2002a), Stock & Watson, (2002b). The factors are estimated by com-puting static principal components of the variables in the dataset. Let yitbe the variable of the dataset to be forecasted at time t, its h-step-aheadprediction equation (also called Diffusion Forecast Index) is obtained byregressing yit+h on the factors and on yit itself. Lags of the factors and ofyit may be added.(ii) FHLR model. This frequency-domain method was proposed in Forni et al., (2000), Forni et al. , (2005) and requires the computation of two steps.In a first step, the common component χt , the idiosyncratic componentξt and their covariances are estimated using a frequency-domain methodintroduced in Forni et al. , (2000) named Dynamic Principal Component.In the second step, the factors are estimated by computing GeneralizedPrincipal Components.(iii) FHLZ model. This frequency-domain method was proposed in Forni et al., (2015), Forni et al. , (2016b). Here, the underlying assumption in (i)and (ii) that the common components span a finite-dimensional space asn tends to infinity is relaxed.There exists some literature comparing the forecasting performances ofSW and FHLR, but universal consensus still does not seem to have been reached.Theoretically, time-domain methods consider only relations among the vari-ables at the same time, whereas frequency-domain methods exploit leaded andlagged relations among the variables. However time-domain methods requireless parameters to be calibrated. Hence they are more robust to misspecifi-cation than frequency-domain methods. Instead, a systematic comparison ofthe forecasting performances of SW, FHLR and FHLZ can be found only inForni et al. , (2016a), Della Marra, (2017). Forni et al. , (2016a) conducted aforecasting exercise on a US macroeconomic dataset, taking an autoregressiveprocess of order 4 as a benchmark. They showed that FHLZ oupterforms SW,FHLR and the benchmark both for Industrial Production and Inflation duringthe Great Moderation. In the Great Recession, the forecasting performancesof the Industrial Production change dramatically: all factor models are outper-formed by the benchmark. SW and FHLR outperform FHLZ. Hence, Forniet al. concluded that, due to its more dynamical structure, FHLZ tends to bethe best performing method in ”stationary periods”, but it loses ground duringregime changes. Also, they showed that FHLZ tends to be outperforming onnominal variables and FHLR on real variables. Della Marra, (2017) conducteda forecasting exercise on an EU macroeconomic dataset. The global settingsof his exercise are basically the same as in Forni et al. , (2016a), but also thelength of the rolling window is suboptimally selected during the calibrationprocess. He found that, on the proper sample, FHLZ is the most performingfor the Inflation. However, mixed evidencies appear over the proper samplefor the Industrial Production.In this paper, the EU dataset is the same employed in Della Marra, (2017).This dataset is split into two subsamples. To guarantee balancedness, timeseries with missing data are discarded. The former, from February 1986 toDecember 2000, is used to calibrate the models, i.e. to produce in-sampleforecasts of the variables of the EU dataset for several specifications of SW,FHLR and FHLZ. Then, for each class of models, we select the specificationwhich shows the minimum mean square forecast error (MSFE). These mod-els are then run and compared in the remaining sample, from January 2001to November 2015. Instead, the US dataset employed in our exercise is ac-curately described in McCracken & Ng, (2016). This dataset is split into twosubsamples. The former, from February 1959 to December 1984, is used tocalibrate the models. Then, for each class of models, we select the specifi-cation which shows the minimum mean square forecast error (MSFE). Thesemodels are then run and compared in the remaining sample, from January 1985to October 2016.The paper is structured as follows. In Section 2, the calibration processof the models is described. In Section 3, results are discussed and Section 4concludes.2 Description of the two datasets and of the calibration processBoth dataset contain real variables (import/export price indexes, employment,Industrial Production) and nominal variables (money aggregates, consumerprice indexes, wages), asset prices (stock prices and exchange rates) and sur-veys. To achieve stationarity, several series are deseasonalized and trans-formed. No treatment for outliers is applied. In addition to SW, FHLR, FHLZ,the forecasts of an autoregressive process (AR) are computed. The order pof the AR process is determined in the calibration process. As in Stock &Watson, (2002b), D’Agostino & Giannone, (2012), to assess the forecastingperformances, the variables which are taken into account are the level of thelogarithm of the Industrial Production (IP) and the yearly change of the loga-rithm of the Consumer Price Index (CPI). Forecasts are computed h-monthsahead, with h ∈ {1,3,6,12,24}. For each methods, we employ a rolling-window scheme [t− l, t], whose size l is determined in the calibration sample.To assess the forecasting performance of each model, the mean-square forecasterror (MSFE) is employed as a metric.Since each method is characterized by several parameters, an exhaustiveexploration of the parameter space would be computationally infeasbile. Hence,we employ genetic algorithms to explore more efficiently the parameter spacein the calibration sample of each dataset. At each epoque, the population ofthe genetic algorithm is a subset of the strings containing all the possibile con-figurations of the parameters. We set the fitness as the inverse for its MSFE.For each method, we iterate the genetic algorithm ten times on the calibrationsample of the two datasets. The fitness of each individual is stored in a datastructure. Eventually, for each method we select as the most performing con-figuration the one endowed with the greatest fitness. The convergence of eachiteration of the genetic algorithms is graphically shown by plotting the boxplotof the results. These plots are not reported here.3 ResultsThe forecasting performance of the three dynamic factor models over the IPand CPI are compared on the proper sample of each dataset. As in Forni et al., (2016a) and in Della Marra, (2017), to assess the forecasting performance ofeach couple of methods locally, each time series of the dataset is smoothed by acentered moving average of length m= 61 (with coefficients equal to 1/m) andthen the Fluctuation test (Giacomini & Rossi, (2010)) is run, at 5% significancelevel. The results for the IP at horizon h = 12 are reported in figure1.As to the EU dataset, all methods outperform AR significatively from thecrisis on (which, according to CEPR, starts in April 2008). Globally, FHLRand FHLZ outperforms SW from the crisis on. As to the IP, FHLR tends tooutperform FHLZ from the crisis on. Instead, as to the CPI, FHLZ tends tooutperform FHLR from the crisis on, but evidencies are less significative. Asin Forni et al. , (2016a) and in Della Marra, (2017), this exercise has beenextended to the other variables in the dataset. The results achieved are omit-ted here, but it can be seen that FHLR tends globally to outperform the othermethods on the real variables and that FHLZ tends globally to outperform theother methods on the nominal variables. On the US dataset, all methods tend,instead, to lose ground against AR significatively during the Great Recession.FHLR tends globally to outperform the other methods on the real variablesand that FHLZ tends globally to outperform the other methods on the nominalvariables.4 ConclusionsIn this paper, we have shown that FHLR tends globally to outperform the othermethods on the real variables and that FHLZ tends globally to outperformthe other methods on the nominal variables. As to EU dataset, Della Marra,(2017) found similar results for the CPI, but mixed evidencies appeared for1990-01-01 2000-01-01 2010-01-01Time-15-10-5051015Fluctuation TestIP SWvsAR -12-step aheadFluctuation TestCritical Value1990-01-01 2000-01-01 2010-01-01Time-5051015Fluctuation TestIP FHLRvsAR -12-step aheadFluctuation TestCritical Value1990-01-01 2000-01-01 2010-01-01Time010203040Fluctuation TestIP FHLZvsAR -12-step aheadFluctuation TestCritical Value1990-01-01 2000-01-01 2010-01-01Time-50510152025Fluctuation TestIP FHLRvsSW -12-step aheadFluctuation TestCritical Value1990-01-01 2000-01-01 2010-01-01Time-10-505101520Fluctuation TestIP FHLZvsSW -12-step aheadFluctuation TestCritical Value1990-01-01 2000-01-01 2010-01-01Time-15-10-505Fluctuation TestIP FHLZvsFHLR -12-step aheadFluctuation TestCritical ValueFigure 1: Fluctuation test for the IP.the IP. As to the US dataset, Forni et al. , (2016a) found similar but lesssignificative results. Hence, we have empirically shown that the calibrationprocess plays a crucial role in these applications, since a more efficient explo-ration of the paramter space allowed us to empirically prove the superiority offrequence-domain dynamic factor models against time-domain factor modelsin a macroeconomic forecasting setting.ReferencesBOIVIN, J., & NG, S. (2005). Understanding and comparing factor-basedforecasts. International Journal of Central Banking, 1(3), 117 – 151.D’AGOSTINO, A., & GIANNONE, D. (2012). Comparing Alternative Predic-tors Based on Large-Panel Factor Models. Oxford bulletin of economicsand statistics, 74(2), 306–326.DELLA MARRA, F. (2017). A forecasting performance comparison of dy-namic factor models based on static and dynamic methods. Communica-tions in Applied and Industrial Mathematics, 8(1), 44–66.FORNI, M., & LIPPI, M. (2001). The generalized dynamic factor model:representation theory. Econometric theory, 17(06), 1113–1141.FORNI, M., HALLIN, M., LIPPI, M., & REICHLIN, L. (2000). The gener-alized dynamic factor model: Identification and estimation. Review ofEconomics and Statistics, 82(4), 540 – 554.FORNI, M., HALLIN, M., LIPPI, M., & REICHLIN, L. (2005). The general-ized dynamic factor model: One-sided estimation and forecasting. Jour-nal of the American Statistical Association, 100, 830 – 840.FORNI, M., GIANNONE, D., LIPPI, M., & REICHLIN, L. (2009). Openingthe black box: Structural factor models with large cross sections. Econo-metric Theory, 25(05), 1319–1347.FORNI, M., HALLIN, M., LIPPI, M., & ZAFFARONI, P. (2015). Dynamicfactor model with infinite dimensional factor space: representation. Jour-nal of Econometrics, 185, 359–371.FORNI, M., GIOVANNELLI, A., LIPPI, M., & SOCCORSI, S. (2016a). Dy-namic Factor Model with infinite dimensional factor space: forecasting.Tech. rept. CEPR Discussion Paper No. DP11161.FORNI, M., HALLIN, M., LIPPI, M., & ZAFFARONI, P. (2016b). Dynamicfactor model with infinite dimensional factor space: Asymptotic analysis.Tech. rept. EIEF Working Paper 2016/07, Einaudi Institute for Economicsand Finance.GIACOMINI, R., & ROSSI, B. (2010). Forecast comparisons in unstable envi-ronments. Journal of Applied Econometrics, 25(4), 595–620.HALLIN, M., & LIPPI, M. (2013). Factor models in high-dimensional timeseriesA time-domain approach. Stochastic Processes and their Applica-tions, 123(7), 2678–2695.MCCRACKEN, M. W., & NG, S. (2016). FRED-MD: A monthly databasefor macroeconomic research. Journal of Business & Economic Statistics,34(4), 574–589.STOCK, J. H., & WATSON, M. W. (2002a). Forecasting using principal com-ponents from a large number of predictors. Journal of the American Sta-tistical Association, 97(460), 1167–1179.STOCK, J. H., & WATSON, M. W. (2002b). Macroeconomic forecasting usingdiffusion indexes. Journal of Business & Economic Statistics, 20(2), 147–162.

Name not available

https://iris.unive.it/retrieve/handle/10278/3691299/110654/CLADAG_2017_paper_36.pdf

Macroeconomic forecasting: a non-standard optimisation approach to the calibration of dynamic factor models.

Abstract

Similar works

Full text

Available Versions

Archivio Istituzionale della Ricerca

Archivio Ricerca Ca'Foscari

Archivio istituzionale della ricerca - Università degli Studi di Venezia Ca' Foscari

Name not available