77 research outputs found
Computing Entropies With Nested Sampling
The Shannon entropy, and related quantities such as mutual information, can
be used to quantify uncertainty and relevance. However, in practice, it can be
difficult to compute these quantities for arbitrary probability distributions,
particularly if the probability mass functions or densities cannot be
evaluated. This paper introduces a computational approach, based on Nested
Sampling, to evaluate entropies of probability distributions that can only be
sampled. I demonstrate the method on three examples: a simple gaussian example
where the key quantities are available analytically; (ii) an experimental
design example about scheduling observations in order to measure the period of
an oscillating signal; and (iii) predicting the future from the past in a
heavy-tailed scenario.Comment: Accepted for publication in Entropy. 21 pages, 3 figures. Software
available at https://github.com/eggplantbren/InfoNes
The Implications of the Early Formation of Life on Earth
One of the most interesting unsolved questions in science today is the
question of life on other planets. At the present time it is safe to say that
we do not have much of an idea as to whether life is common or exceedingly rare
in the universe, and this will probably not be solved for certain unless
definitive evidence of extraterrestrial life is found in the future. Our
presence on Earth is just as consistent with the hypothesis that life is
extremely rare as it is with the hypothesis that it is common, since if there
was only one planet with intelligent life, we would find ourselves on it.
However, we have more information than this, such as the the surprisingly short
length of time it took for life to arise on Earth. Previous authors have
analysed this information, concluding that it is evidence that the probability
of abiogenesis is moderate ( 13% with 95% probability) and cannot be
extremely small. In this paper I use simple probabilistic model to show that
this conclusion was based more on an unintentional assumption than on the data.
While the early formation of life on Earth provides some evidence in the
direction of life being common, it is far from conclusive, and in particular
does not rule out the possibility that abiogenesis has only occurred once in
the history of the universe.Comment: Submitted, but seems to have fallen into a black hole sinc
Inference for Trans-dimensional Bayesian Models with Diffusive Nested Sampling
Many inference problems involve inferring the number of components in
some region, along with their properties , from a
dataset . A common statistical example is finite mixture
modelling. In the Bayesian framework, these problems are typically solved using
one of the following two methods: i) by executing a Monte Carlo algorithm (such
as Nested Sampling) once for each possible value of , and calculating the
marginal likelihood or evidence as a function of ; or ii) by doing a single
run that allows the model dimension to change (such as Markov Chain Monte
Carlo with birth/death moves), and obtaining the posterior for directly. In
this paper we present a general approach to this problem that uses
trans-dimensional MCMC embedded within a Nested Sampling algorithm, allowing us
to explore the posterior distribution and calculate the marginal likelihood
(summed over ) even if the problem contains a phase transition or other
difficult features such as multimodality. We present two example problems,
finding sinusoidal signals in noisy data, and finding and measuring galaxies in
a noisy astronomical image. Both of the examples demonstrate phase transitions
in the relationship between the likelihood and the cumulative prior mass,
highlighting the need for Nested Sampling.Comment: Only published here for the time being. 17 pages, 10 figures.
Software available at https://github.com/eggplantbren/RJObjec
- β¦