9,366 research outputs found
Entropic Priors for Discrete Probabilistic Networks and for Mixtures of Gaussians Models
The ongoing unprecedented exponential explosion of available computing power,
has radically transformed the methods of statistical inference. What used to be
a small minority of statisticians advocating for the use of priors and a strict
adherence to bayes theorem, it is now becoming the norm across disciplines. The
evolutionary direction is now clear. The trend is towards more realistic,
flexible and complex likelihoods characterized by an ever increasing number of
parameters. This makes the old question of: What should the prior be? to
acquire a new central importance in the modern bayesian theory of inference.
Entropic priors provide one answer to the problem of prior selection. The
general definition of an entropic prior has existed since 1988, but it was not
until 1998 that it was found that they provide a new notion of complete
ignorance. This paper re-introduces the family of entropic priors as minimizers
of mutual information between the data and the parameters, as in
[rodriguez98b], but with a small change and a correction. The general formalism
is then applied to two large classes of models: Discrete probabilistic networks
and univariate finite mixtures of gaussians. It is also shown how to perform
inference by efficiently sampling the corresponding posterior distributions.Comment: 24 pages, 3 figures, Presented at MaxEnt2001, APL Johns Hopkins
University, August 4-9 2001. See also http://omega.albany.edu:8008
Learning OT constraint rankings using a maximum entropy model
Abstract. A weakness of standard Optimality Theory is its inability to account for grammar
Entropic Priors and Bayesian Model Selection
We demonstrate that the principle of maximum relative entropy (ME), used
judiciously, can ease the specification of priors in model selection problems.
The resulting effect is that models that make sharp predictions are
disfavoured, weakening the usual Bayesian "Occam's Razor". This is illustrated
with a simple example involving what Jaynes called a "sure thing" hypothesis.
Jaynes' resolution of the situation involved introducing a large number of
alternative "sure thing" hypotheses that were possible before we observed the
data. However, in more complex situations, it may not be possible to explicitly
enumerate large numbers of alternatives. The entropic priors formalism produces
the desired result without modifying the hypothesis space or requiring explicit
enumeration of alternatives; all that is required is a good model for the prior
predictive distribution for the data. This idea is illustrated with a simple
rigged-lottery example, and we outline how this idea may help to resolve a
recent debate amongst cosmologists: is dark energy a cosmological constant, or
has it evolved with time in some way? And how shall we decide, when the data
are in?Comment: Presented at MaxEnt 2009, the 29th International Workshop on Bayesian
Inference and Maximum Entropy Methods in Science and Engineering (July 5-10,
2009, Oxford, Mississippi, USA
On the Prior and Posterior Distributions Used in Graphical Modelling
Graphical model learning and inference are often performed using Bayesian
techniques. In particular, learning is usually performed in two separate steps.
First, the graph structure is learned from the data; then the parameters of the
model are estimated conditional on that graph structure. While the probability
distributions involved in this second step have been studied in depth, the ones
used in the first step have not been explored in as much detail.
In this paper, we will study the prior and posterior distributions defined
over the space of the graph structures for the purpose of learning the
structure of a graphical model. In particular, we will provide a
characterisation of the behaviour of those distributions as a function of the
possible edges of the graph. We will then use the properties resulting from
this characterisation to define measures of structural variability for both
Bayesian and Markov networks, and we will point out some of their possible
applications.Comment: 28 pages, 6 figure
Bayesian Credibility for GLMs
We revisit the classical credibility results of Jewell and B\"uhlmann to
obtain credibility premiums for a GLM using a modern Bayesian approach. Here
the prior distributions can be chosen without restrictions to be conjugate to
the response distribution. It can even come from out-of-sample information if
the actuary prefers.
Then we use the relative entropy between the "true" and the estimated models
as a loss function, without restricting credibility premiums to be linear. A
numerical illustration on real data shows the feasibility of the approach, now
that computing power is cheap, and simulations software readily available
- ā¦