62,248 research outputs found
Extreme Quantum Advantage for Rare-Event Sampling
We introduce a quantum algorithm for efficient biased sampling of the rare
events generated by classical memoryful stochastic processes. We show that this
quantum algorithm gives an extreme advantage over known classical biased
sampling algorithms in terms of the memory resources required. The quantum
memory advantage ranges from polynomial to exponential and when sampling the
rare equilibrium configurations of spin systems the quantum advantage diverges.Comment: 11 pages, 9 figures;
http://csc.ucdavis.edu/~cmg/compmech/pubs/eqafbs.ht
Determination of the chemical potential using energy-biased sampling
An energy-biased method to evaluate ensemble averages requiring test-particle
insertion is presented. The method is based on biasing the sampling within the
subdomains of the test-particle configurational space with energies smaller
than a given value freely assigned. These energy-wells are located via unbiased
random insertion over the whole configurational space and are sampled using the
so called Hit&Run algorithm, which uniformly samples compact regions of any
shape immersed in a space of arbitrary dimensions. Because the bias is defined
in terms of the energy landscape it can be exactly corrected to obtain the
unbiased distribution. The test-particle energy distribution is then combined
with the Bennett relation for the evaluation of the chemical potential. We
apply this protocol to a system with relatively small probability of low-energy
test-particle insertion, liquid argon at high density and low temperature, and
show that the energy-biased Bennett method is around five times more efficient
than the standard Bennett method. A similar performance gain is observed in the
reconstruction of the energy distribution.Comment: 10 pages, 4 figure
Free Energy Methods for Bayesian Inference: Efficient Exploration of Univariate Gaussian Mixture Posteriors
Because of their multimodality, mixture posterior distributions are difficult
to sample with standard Markov chain Monte Carlo (MCMC) methods. We propose a
strategy to enhance the sampling of MCMC in this context, using a biasing
procedure which originates from computational Statistical Physics. The
principle is first to choose a "reaction coordinate", that is, a "direction" in
which the target distribution is multimodal. In a second step, the marginal
log-density of the reaction coordinate with respect to the posterior
distribution is estimated; minus this quantity is called "free energy" in the
computational Statistical Physics literature. To this end, we use adaptive
biasing Markov chain algorithms which adapt their targeted invariant
distribution on the fly, in order to overcome sampling barriers along the
chosen reaction coordinate. Finally, we perform an importance sampling step in
order to remove the bias and recover the true posterior. The efficiency factor
of the importance sampling step can easily be estimated \emph{a priori} once
the bias is known, and appears to be rather large for the test cases we
considered. A crucial point is the choice of the reaction coordinate. One
standard choice (used for example in the classical Wang-Landau algorithm) is
minus the log-posterior density. We discuss other choices. We show in
particular that the hyper-parameter that determines the order of magnitude of
the variance of each component is both a convenient and an efficient reaction
coordinate. We also show how to adapt the method to compute the evidence
(marginal likelihood) of a mixture model. We illustrate our approach by
analyzing two real data sets
An Exact Auxiliary Variable Gibbs Sampler for a Class of Diffusions
Stochastic differential equations (SDEs) or diffusions are continuous-valued
continuous-time stochastic processes widely used in the applied and
mathematical sciences. Simulating paths from these processes is usually an
intractable problem, and typically involves time-discretization approximations.
We propose an exact Markov chain Monte Carlo sampling algorithm that involves
no such time-discretization error. Our sampler is applicable to the problem of
prior simulation from an SDE, posterior simulation conditioned on noisy
observations, as well as parameter inference given noisy observations. Our work
recasts an existing rejection sampling algorithm for a class of diffusions as a
latent variable model, and then derives an auxiliary variable Gibbs sampling
algorithm that targets the associated joint distribution. At a high level, the
resulting algorithm involves two steps: simulating a random grid of times from
an inhomogeneous Poisson process, and updating the SDE trajectory conditioned
on this grid. Our work allows the vast literature of Monte Carlo sampling
algorithms from the Gaussian process literature to be brought to bear to
applications involving diffusions. We study our method on synthetic and real
datasets, where we demonstrate superior performance over competing methods.Comment: 37 pages, 13 figure
Discriminative Density-ratio Estimation
The covariate shift is a challenging problem in supervised learning that
results from the discrepancy between the training and test distributions. An
effective approach which recently drew a considerable attention in the research
community is to reweight the training samples to minimize that discrepancy. In
specific, many methods are based on developing Density-ratio (DR) estimation
techniques that apply to both regression and classification problems. Although
these methods work well for regression problems, their performance on
classification problems is not satisfactory. This is due to a key observation
that these methods focus on matching the sample marginal distributions without
paying attention to preserving the separation between classes in the reweighted
space. In this paper, we propose a novel method for Discriminative
Density-ratio (DDR) estimation that addresses the aforementioned problem and
aims at estimating the density-ratio of joint distributions in a class-wise
manner. The proposed algorithm is an iterative procedure that alternates
between estimating the class information for the test data and estimating new
density ratio for each class. To incorporate the estimated class information of
the test data, a soft matching technique is proposed. In addition, we employ an
effective criterion which adopts mutual information as an indicator to stop the
iterative procedure while resulting in a decision boundary that lies in a
sparse region. Experiments on synthetic and benchmark datasets demonstrate the
superiority of the proposed method in terms of both accuracy and robustness
Efficient Dynamic Importance Sampling of Rare Events in One Dimension
Exploiting stochastic path integral theory, we obtain \emph{by simulation}
substantial gains in efficiency for the computation of reaction rates in
one-dimensional, bistable, overdamped stochastic systems. Using a well-defined
measure of efficiency, we compare implementations of ``Dynamic Importance
Sampling'' (DIMS) methods to unbiased simulation. The best DIMS algorithms are
shown to increase efficiency by factors of approximately 20 for a
barrier height and 300 for , compared to unbiased simulation. The
gains result from close emulation of natural (unbiased), instanton-like
crossing events with artificially decreased waiting times between events that
are corrected for in rate calculations. The artificial crossing events are
generated using the closed-form solution to the most probable crossing event
described by the Onsager-Machlup action. While the best biasing methods require
the second derivative of the potential (resulting from the ``Jacobian'' term in
the action, which is discussed at length), algorithms employing solely the
first derivative do nearly as well. We discuss the importance of
one-dimensional models to larger systems, and suggest extensions to
higher-dimensional systems.Comment: version to be published in Phys. Rev.
Speeding Up MCMC by Delayed Acceptance and Data Subsampling
The complexity of the Metropolis-Hastings (MH) algorithm arises from the
requirement of a likelihood evaluation for the full data set in each iteration.
Payne and Mallick (2015) propose to speed up the algorithm by a delayed
acceptance approach where the acceptance decision proceeds in two stages. In
the first stage, an estimate of the likelihood based on a random subsample
determines if it is likely that the draw will be accepted and, if so, the
second stage uses the full data likelihood to decide upon final acceptance.
Evaluating the full data likelihood is thus avoided for draws that are unlikely
to be accepted. We propose a more precise likelihood estimator which
incorporates auxiliary information about the full data likelihood while only
operating on a sparse set of the data. We prove that the resulting delayed
acceptance MH is more efficient compared to that of Payne and Mallick (2015).
The caveat of this approach is that the full data set needs to be evaluated in
the second stage. We therefore propose to substitute this evaluation by an
estimate and construct a state-dependent approximation thereof to use in the
first stage. This results in an algorithm that (i) can use a smaller subsample
m by leveraging on recent advances in Pseudo-Marginal MH (PMMH) and (ii) is
provably within of the true posterior.Comment: Accepted for publication in Journal of Computational and Graphical
Statistic
- …