Climate change impact studies inform policymakers on the estimated damages of
future climate change on economic, health and other outcomes. In most studies,
an annual outcome variable is observed, e.g. annual mortality rate, along with
higher-frequency regressors, e.g. daily temperature and precipitation.
Practitioners use summaries of the higher-frequency regressors in fixed effects
panel models. The choice over summary statistics amounts to model selection.
Some practitioners use Monte Carlo cross-validation (MCCV) to justify a
particular specification. However, conventional implementation of MCCV with
fixed testing-to-full sample ratios tends to select over-fit models. This paper
presents conditions under which MCCV, and also information criteria, can
deliver consistent model selection. Previous work has established that the
Bayesian information criterion (BIC) can be inconsistent for non-nested
selection. We illustrate that the BIC can also be inconsistent in our
framework, when all candidate models are misspecified. Our results have
practical implications for empirical conventions in climate change impact
studies. Specifically, they highlight the importance of a priori information
provided by the scientific literature to guide the models considered for
selection. We emphasize caution in interpreting model selection results in
settings where the scientific literature does not specify the relationship
between the outcome and the weather variables.Comment: Additional simulation results available from authors by reques