23 research outputs found
A Repelling-Attracting Metropolis Algorithm for Multimodality
Although the Metropolis algorithm is simple to implement, it often has
difficulties exploring multimodal distributions. We propose the
repelling-attracting Metropolis (RAM) algorithm that maintains the
simple-to-implement nature of the Metropolis algorithm, but is more likely to
jump between modes. The RAM algorithm is a Metropolis-Hastings algorithm with a
proposal that consists of a downhill move in density that aims to make local
modes repelling, followed by an uphill move in density that aims to make local
modes attracting. The downhill move is achieved via a reciprocal Metropolis
ratio so that the algorithm prefers downward movement. The uphill move does the
opposite using the standard Metropolis ratio which prefers upward movement.
This down-up movement in density increases the probability of a proposed move
to a different mode. Because the acceptance probability of the proposal
involves a ratio of intractable integrals, we introduce an auxiliary variable
which creates a term in the acceptance probability that cancels with the
intractable ratio. Using several examples, we demonstrate the potential for the
RAM algorithm to explore a multimodal distribution more efficiently than a
Metropolis algorithm and with less tuning than is commonly required by
tempering-based methods
Thermodynamic assessment of probability distribution divergencies and Bayesian model comparison
Within path sampling framework, we show that probability distribution
divergences, such as the Chernoff information, can be estimated via
thermodynamic integration. The Boltzmann-Gibbs distribution pertaining to
different Hamiltonians is implemented to derive tempered transitions along the
path, linking the distributions of interest at the endpoints. Under this
perspective, a geometric approach is feasible, which prompts intuition and
facilitates tuning the error sources. Additionally, there are direct
applications in Bayesian model evaluation. Existing marginal likelihood and
Bayes factor estimators are reviewed here along with their stepping-stone
sampling analogues. New estimators are presented and the use of compound paths
is introduced
Parallel Tempering with Equi-Energy Moves
The Equi-Energy Sampler (EES) introduced by Kou et al [2006] is based on a
population of chains which are updated by local moves and global moves, also
called equi-energy jumps. The state space is partitioned into energy rings, and
the current state of a chain can jump to a past state of an adjacent chain that
has energy level close to its level. This algorithm has been developed to
facilitate global moves between different chains, resulting in a good
exploration of the state space by the target chain. This method seems to be
more efficient than the classical Parallel Tempering (PT) algorithm. However it
is difficult to use in combination with a Gibbs sampler and it necessitates
increased storage. In this paper we propose an adaptation of this EES that
combines PT with the principle of swapping between chains with same levels of
energy. This adaptation, that we shall call Parallel Tempering with Equi-Energy
Moves (PTEEM), keeps the original idea of the EES method while ensuring good
theoretical properties, and practical implementation even if combined with a
Gibbs sampler. Performances of the PTEEM algorithm are compared with those of
the EES and of the standard PT algorithms in the context of mixture models, and
in a problem of identification of gene regulatory binding motifs