Monte-Carlo techniques are standard numerical tools for exploring
non-Gaussian and multivariate likelihoods. Many variants of the original
Metropolis-Hastings algorithm have been proposed to increase the sampling
efficiency. Motivated by Ensemble Monte Carlo we allow the number of Markov
chains to vary by exchanging particles with a reservoir, controlled by a
parameter analogous to a chemical potential μ, which effectively
establishes a random process that samples microstates from a macrocanonical
instead of a canonical ensemble. In this paper, we develop the theory of
macrocanonical sampling for statistical inference on the basis of Bayesian
macrocanonical partition functions, thereby bringing to light the relations
between information-theoretical quantities and thermodynamic properties.
Furthermore, we propose an algorithm for macrocanonical sampling,
Avalanche Sampling, and apply it to various toy problems as well as
the likelihood on the cosmological parameters Ωm and w on the basis
of data from the supernova distance redshift relation.Comment: 13 pages, 9 figure