20,484 research outputs found
Metropolis Sampling
Monte Carlo (MC) sampling methods are widely applied in Bayesian inference,
system simulation and optimization problems. The Markov Chain Monte Carlo
(MCMC) algorithms are a well-known class of MC methods which generate a Markov
chain with the desired invariant distribution. In this document, we focus on
the Metropolis-Hastings (MH) sampler, which can be considered as the atom of
the MCMC techniques, introducing the basic notions and different properties. We
describe in details all the elements involved in the MH algorithm and the most
relevant variants. Several improvements and recent extensions proposed in the
literature are also briefly discussed, providing a quick but exhaustive
overview of the current Metropolis-based sampling's world.Comment: Wiley StatsRef-Statistics Reference Online, 201
Reduced Complexity Filtering with Stochastic Dominance Bounds: A Convex Optimization Approach
This paper uses stochastic dominance principles to construct upper and lower
sample path bounds for Hidden Markov Model (HMM) filters. Given a HMM, by using
convex optimization methods for nuclear norm minimization with copositive
constraints, we construct low rank stochastic marices so that the optimal
filters using these matrices provably lower and upper bound (with respect to a
partially ordered set) the true filtered distribution at each time instant.
Since these matrices are low rank (say R), the computational cost of evaluating
the filtering bounds is O(XR) instead of O(X2). A Monte-Carlo importance
sampling filter is presented that exploits these upper and lower bounds to
estimate the optimal posterior. Finally, using the Dobrushin coefficient,
explicit bounds are given on the variational norm between the true posterior
and the upper and lower bounds
Dynamic importance sampling for uniformly recurrent markov chains
Importance sampling is a variance reduction technique for efficient
estimation of rare-event probabilities by Monte Carlo. In standard importance
sampling schemes, the system is simulated using an a priori fixed change of
measure suggested by a large deviation lower bound analysis. Recent work,
however, has suggested that such schemes do not work well in many situations.
In this paper we consider dynamic importance sampling in the setting of
uniformly recurrent Markov chains. By ``dynamic'' we mean that in the course of
a single simulation, the change of measure can depend on the outcome of the
simulation up till that time. Based on a control-theoretic approach to large
deviations, the existence of asymptotically optimal dynamic schemes is
demonstrated in great generality. The implementation of the dynamic schemes is
carried out with the help of a limiting Bellman equation. Numerical examples
are presented to contrast the dynamic and standard schemes.Comment: Published at http://dx.doi.org/10.1214/105051604000001016 in the
Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute
of Mathematical Statistics (http://www.imstat.org
Asymptotic optimality of the cross-entropy method for Markov chain problems
The correspondence between the cross-entropy method and the zero-variance
approximation to simulate a rare event problem in Markov chains is shown. This
leads to a sufficient condition that the cross-entropy estimator is
asymptotically optimal.Comment: 13 pager; 3 figure
Non-reversible Metropolis-Hastings
The classical Metropolis-Hastings (MH) algorithm can be extended to generate
non-reversible Markov chains. This is achieved by means of a modification of
the acceptance probability, using the notion of vorticity matrix. The resulting
Markov chain is non-reversible. Results from the literature on asymptotic
variance, large deviations theory and mixing time are mentioned, and in the
case of a large deviations result, adapted, to explain how non-reversible
Markov chains have favorable properties in these respects.
We provide an application of NRMH in a continuous setting by developing the
necessary theory and applying, as first examples, the theory to Gaussian
distributions in three and nine dimensions. The empirical autocorrelation and
estimated asymptotic variance for NRMH applied to these examples show
significant improvement compared to MH with identical stepsize.Comment: in Statistics and Computing, 201
Calculating principal eigen-functions of non-negative integral kernels: particle approximations and applications
Often in applications such as rare events estimation or optimal control it is
required that one calculates the principal eigen-function and eigen-value of a
non-negative integral kernel. Except in the finite-dimensional case, usually
neither the principal eigen-function nor the eigen-value can be computed
exactly. In this paper, we develop numerical approximations for these
quantities. We show how a generic interacting particle algorithm can be used to
deliver numerical approximations of the eigen-quantities and the associated
so-called "twisted" Markov kernel as well as how these approximations are
relevant to the aforementioned applications. In addition, we study a collection
of random integral operators underlying the algorithm, address some of their
mean and path-wise properties, and obtain error estimates. Finally,
numerical examples are provided in the context of importance sampling for
computing tail probabilities of Markov chains and computing value functions for
a class of stochastic optimal control problems.Comment: 38 pages, 4 figures, 1 table; to appear in Mathematics of Operations
Researc
Scalable Inference for Markov Processes with Intractable Likelihoods
Bayesian inference for Markov processes has become increasingly relevant in
recent years. Problems of this type often have intractable likelihoods and
prior knowledge about model rate parameters is often poor. Markov Chain Monte
Carlo (MCMC) techniques can lead to exact inference in such models but in
practice can suffer performance issues including long burn-in periods and poor
mixing. On the other hand approximate Bayesian computation techniques can allow
rapid exploration of a large parameter space but yield only approximate
posterior distributions. Here we consider the combined use of approximate
Bayesian computation (ABC) and MCMC techniques for improved computational
efficiency while retaining exact inference on parallel hardware
- …