377 research outputs found
Entropy and inference, revisited
We study properties of popular near-uniform (Dirichlet) priors for learning
undersampled probability distributions on discrete nonmetric spaces and show
that they lead to disastrous results. However, an Occam-style phase space
argument expands the priors into their infinite mixture and resolves most of
the observed problems. This leads to a surprisingly good estimator of entropies
of discrete distributions.Comment: LaTex2e, 9 pages, 5 figures; references added, minor revisions
introduced, formatting errors correcte
A Parsimonious Tour of Bayesian Model Uncertainty
Modern statistical software and machine learning libraries are enabling
semi-automated statistical inference. Within this context, it appears easier
and easier to try and fit many models to the data at hand, reversing thereby
the Fisherian way of conducting science by collecting data after the scientific
hypothesis (and hence the model) has been determined. The renewed goal of the
statistician becomes to help the practitioner choose within such large and
heterogeneous families of models, a task known as model selection. The Bayesian
paradigm offers a systematized way of assessing this problem. This approach,
launched by Harold Jeffreys in his 1935 book Theory of Probability, has
witnessed a remarkable evolution in the last decades, that has brought about
several new theoretical and methodological advances. Some of these recent
developments are the focus of this survey, which tries to present a unifying
perspective on work carried out by different communities. In particular, we
focus on non-asymptotic out-of-sample performance of Bayesian model selection
and averaging techniques, and draw connections with penalized maximum
likelihood. We also describe recent extensions to wider classes of
probabilistic frameworks including high-dimensional, unidentifiable, or
likelihood-free models
Prediction of particle type from measurements of particle location: A physicist's approach to Bayesian classification
The Bayesian approach to the prediction of particle type given measurements
of particle location is explored, using a parametric model whose prior is based
on the transformation group. Two types of particle are considered, and
locations are expressed in terms of a single spatial coordinate. Several cases
corresponding to different states of prior knowledge are evaluated, including
the effect of measurement uncertainty. Comparisons are made to nearest neighbor
classification and kernel density estimation. How one can evaluate the
reliability of the prediction solely from the available data is discussed.Comment: 22 pages, 11 figures, 4 tables, minor revisio
- …