Towards Machine Wald

The past century has seen a steady increase in the need of estimating and predicting complex systems and making (possibly critical) decisions with limited information. Although computers have made possible the numerical evaluation of sophisticated statistical models, these models are still designed \emph{by humans} because there is currently no known recipe or algorithm for dividing the design of a statistical model into a sequence of arithmetic operations. Indeed enabling computers to \emph{think} as \emph{humans} have the ability to do when faced with uncertainty is challenging in several major ways: (1) Finding optimal statistical models remains to be formulated as a well posed problem when information on the system of interest is incomplete and comes in the form of a complex combination of sample data, partial knowledge of constitutive relations and a limited description of the distribution of input random variables. (2) The space of admissible scenarios along with the space of relevant information, assumptions, and/or beliefs, tend to be infinite dimensional, whereas calculus on a computer is necessarily discrete and finite. With this purpose, this paper explores the foundations of a rigorous framework for the scientific computation of optimal statistical estimators/models and reviews their connections with Decision Theory, Machine Learning, Bayesian Inference, Stochastic Optimization, Robust Optimization, Optimal Uncertainty Quantification and Information Based Complexity.Comment: 37 page

Counterfactual Sensitivity and Robustness

Researchers frequently make parametric assumptions about the distribution of unobservables when formulating structural models. Such assumptions are typically motived by computational convenience rather than economic theory and are often untestable. Counterfactuals can be particularly sensitive to such assumptions, threatening the credibility of structural modeling exercises. To address this issue, we leverage insights from the literature on ambiguity and model uncertainty to propose a tractable econometric framework for characterizing the sensitivity of counterfactuals with respect to a researcher's assumptions about the distribution of unobservables in a class of structural models. In particular, we show how to construct the smallest and largest values of the counterfactual as the distribution of unobservables spans nonparametric neighborhoods of the researcher's assumed specification while other `structural' features of the model, e.g. equilibrium conditions, are maintained. Our methods are computationally simple to implement, with the nuisance distribution effectively profiled out via a low-dimensional convex program. Our procedure delivers sharp bounds for the identified set of counterfactuals (i.e. without parametric assumptions about the distribution of unobservables) as the neighborhoods become large. Over small neighborhoods, we relate our procedure to a measure of local sensitivity which is further characterized using an influence function representation. We provide a suitable sampling theory for plug-in estimators and apply our procedure to models of strategic interaction and dynamic discrete choice