Theory and Applications of Proper Scoring Rules

We give an overview of some uses of proper scoring rules in statistical inference, including frequentist estimation theory and Bayesian model selection with improper priors.Comment: 13 page

Rejoinder to "Bayesian Model Selection Based on Proper Scoring Rules"

We are deeply appreciative of the initiative of the editor, Marina Vanucci, in commissioning a discussion of our paper, and extremely grateful to all the discussants for their insightful and thought-provoking comments. We respond to the discussions in alphabetical order [arXiv:1409.5291].Comment: Published at http://dx.doi.org/10.1214/15-BA942REJ in the Bayesian Analysis (http://projecteuclid.org/euclid.ba) by the International Society of Bayesian Analysis (http://bayesian.org/

Bayesian Model Selection Based on Proper Scoring Rules

Bayesian model selection with improper priors is not well-defined because of the dependence of the marginal likelihood on the arbitrary scaling constants of the within-model prior densities. We show how this problem can be evaded by replacing marginal log-likelihood by a homogeneous proper scoring rule, which is insensitive to the scaling constants. Suitably applied, this will typically enable consistent selection of the true model.Comment: Published at http://dx.doi.org/10.1214/15-BA942 in the Bayesian Analysis (http://projecteuclid.org/euclid.ba) by the International Society of Bayesian Analysis (http://bayesian.org/

Minimum scoring rule inference

Proper scoring rules are methods for encouraging honest assessment of probability distributions. Just like likelihood, a proper scoring rule can be applied to supply an unbiased estimating equation for any statistical model, and the theory of such equations can be applied to understand the properties of the associated estimator. In this paper we develop some basic scoring rule estimation theory, and explore robustness and interval estimation properties by means of theory and simulations.Comment: 27 pages, 3 figure

A Note on Bayesian Model Selection for Discrete Data Using Proper Scoring Rules

We consider the problem of choosing between parametric models for a discrete observable, taking a Bayesian approach in which the within-model prior distributions are allowed to be improper. In order to avoid the ambiguity in the marginal likelihood function in such a case, we apply a homogeneous scoring rule. For the particular case of distinguishing between Poisson and Negative Binomial models, we conduct simulations that indicate that, applied prequentially, the method will consistently select the true model.Comment: 8 pages, 2 figure

Comparisons of Hyv\"arinen and pairwise estimators in two simple linear time series models

The aim of this paper is to compare numerically the performance of two estimators based on Hyv\"arinen's local homogeneous scoring rule with that of the full and the pairwise maximum likelihood estimators. In particular, two different model settings, for which both full and pairwise maximum likelihood estimators can be obtained, have been considered: the first order autoregressive model (AR(1)) and the moving average model (MA(1)). Simulation studies highlight very different behaviours for the Hyv\"arinen scoring rule estimators relative to the pairwise likelihood estimators in these two settings.Comment: 14 pages, 2 figure