7,747 research outputs found
Stochastic methods for solving high-dimensional partial differential equations
We propose algorithms for solving high-dimensional Partial Differential
Equations (PDEs) that combine a probabilistic interpretation of PDEs, through
Feynman-Kac representation, with sparse interpolation. Monte-Carlo methods and
time-integration schemes are used to estimate pointwise evaluations of the
solution of a PDE. We use a sequential control variates algorithm, where
control variates are constructed based on successive approximations of the
solution of the PDE. Two different algorithms are proposed, combining in
different ways the sequential control variates algorithm and adaptive sparse
interpolation. Numerical examples will illustrate the behavior of these
algorithms
Adaptive Simulation Using Perfect Control Variates
Adaptive Simulation Using Perfect Control Variate
Efficient posterior sampling for high-dimensional imbalanced logistic regression
High-dimensional data are routinely collected in many areas. We are
particularly interested in Bayesian classification models in which one or more
variables are imbalanced. Current Markov chain Monte Carlo algorithms for
posterior computation are inefficient as and/or increase due to
worsening time per step and mixing rates. One strategy is to use a
gradient-based sampler to improve mixing while using data sub-samples to reduce
per-step computational complexity. However, usual sub-sampling breaks down when
applied to imbalanced data. Instead, we generalize piece-wise deterministic
Markov chain Monte Carlo algorithms to include importance-weighted and
mini-batch sub-sampling. These approaches maintain the correct stationary
distribution with arbitrarily small sub-samples, and substantially outperform
current competitors. We provide theoretical support and illustrate gains in
simulated and real data applications.Comment: 4 figure
Speeding Up MCMC by Delayed Acceptance and Data Subsampling
The complexity of the Metropolis-Hastings (MH) algorithm arises from the
requirement of a likelihood evaluation for the full data set in each iteration.
Payne and Mallick (2015) propose to speed up the algorithm by a delayed
acceptance approach where the acceptance decision proceeds in two stages. In
the first stage, an estimate of the likelihood based on a random subsample
determines if it is likely that the draw will be accepted and, if so, the
second stage uses the full data likelihood to decide upon final acceptance.
Evaluating the full data likelihood is thus avoided for draws that are unlikely
to be accepted. We propose a more precise likelihood estimator which
incorporates auxiliary information about the full data likelihood while only
operating on a sparse set of the data. We prove that the resulting delayed
acceptance MH is more efficient compared to that of Payne and Mallick (2015).
The caveat of this approach is that the full data set needs to be evaluated in
the second stage. We therefore propose to substitute this evaluation by an
estimate and construct a state-dependent approximation thereof to use in the
first stage. This results in an algorithm that (i) can use a smaller subsample
m by leveraging on recent advances in Pseudo-Marginal MH (PMMH) and (ii) is
provably within of the true posterior.Comment: Accepted for publication in Journal of Computational and Graphical
Statistic
Control Variates for Reversible MCMC Samplers
A general methodology is introduced for the construction and effective
application of control variates to estimation problems involving data from
reversible MCMC samplers. We propose the use of a specific class of functions
as control variates, and we introduce a new, consistent estimator for the
values of the coefficients of the optimal linear combination of these
functions. The form and proposed construction of the control variates is
derived from our solution of the Poisson equation associated with a specific
MCMC scenario. The new estimator, which can be applied to the same MCMC sample,
is derived from a novel, finite-dimensional, explicit representation for the
optimal coefficients. The resulting variance-reduction methodology is primarily
applicable when the simulated data are generated by a conjugate random-scan
Gibbs sampler. MCMC examples of Bayesian inference problems demonstrate that
the corresponding reduction in the estimation variance is significant, and that
in some cases it can be quite dramatic. Extensions of this methodology in
several directions are given, including certain families of Metropolis-Hastings
samplers and hybrid Metropolis-within-Gibbs algorithms. Corresponding
simulation examples are presented illustrating the utility of the proposed
methods. All methodological and asymptotic arguments are rigorously justified
under easily verifiable and essentially minimal conditions.Comment: 44 pages; 6 figures; 5 table
- …