9,634 research outputs found
Efficient Sequential Monte-Carlo Samplers for Bayesian Inference
In many problems, complex non-Gaussian and/or nonlinear models are required
to accurately describe a physical system of interest. In such cases, Monte
Carlo algorithms are remarkably flexible and extremely powerful approaches to
solve such inference problems. However, in the presence of a high-dimensional
and/or multimodal posterior distribution, it is widely documented that standard
Monte-Carlo techniques could lead to poor performance. In this paper, the study
is focused on a Sequential Monte-Carlo (SMC) sampler framework, a more robust
and efficient Monte Carlo algorithm. Although this approach presents many
advantages over traditional Monte-Carlo methods, the potential of this emergent
technique is however largely underexploited in signal processing. In this work,
we aim at proposing some novel strategies that will improve the efficiency and
facilitate practical implementation of the SMC sampler specifically for signal
processing applications. Firstly, we propose an automatic and adaptive strategy
that selects the sequence of distributions within the SMC sampler that
minimizes the asymptotic variance of the estimator of the posterior
normalization constant. This is critical for performing model selection in
modelling applications in Bayesian signal processing. The second original
contribution we present improves the global efficiency of the SMC sampler by
introducing a novel correction mechanism that allows the use of the particles
generated through all the iterations of the algorithm (instead of only
particles from the last iteration). This is a significant contribution as it
removes the need to discard a large portion of the samples obtained, as is
standard in standard SMC methods. This will improve estimation performance in
practical settings where computational budget is important to consider.Comment: arXiv admin note: text overlap with arXiv:1303.3123 by other author
Unbiased and Consistent Nested Sampling via Sequential Monte Carlo
We introduce a new class of sequential Monte Carlo methods called Nested
Sampling via Sequential Monte Carlo (NS-SMC), which reframes the Nested
Sampling method of Skilling (2006) in terms of sequential Monte Carlo
techniques. This new framework allows convergence results to be obtained in the
setting when Markov chain Monte Carlo (MCMC) is used to produce new samples. An
additional benefit is that marginal likelihood estimates are unbiased. In
contrast to NS, the analysis of NS-SMC does not require the (unrealistic)
assumption that the simulated samples be independent. As the original NS
algorithm is a special case of NS-SMC, this provides insights as to why NS
seems to produce accurate estimates despite a typical violation of its
assumptions. For applications of NS-SMC, we give advice on tuning MCMC kernels
in an automated manner via a preliminary pilot run, and present a new method
for appropriately choosing the number of MCMC repeats at each iteration.
Finally, a numerical study is conducted where the performance of NS-SMC and
temperature-annealed SMC is compared on several challenging and realistic
problems. MATLAB code for our experiments is made available at
https://github.com/LeahPrice/SMC-NS .Comment: 45 pages, some minor typographical errors fixed since last versio
Langevin and Hamiltonian based Sequential MCMC for Efficient Bayesian Filtering in High-dimensional Spaces
Nonlinear non-Gaussian state-space models arise in numerous applications in
statistics and signal processing. In this context, one of the most successful
and popular approximation techniques is the Sequential Monte Carlo (SMC)
algorithm, also known as particle filtering. Nevertheless, this method tends to
be inefficient when applied to high dimensional problems. In this paper, we
focus on another class of sequential inference methods, namely the Sequential
Markov Chain Monte Carlo (SMCMC) techniques, which represent a promising
alternative to SMC methods. After providing a unifying framework for the class
of SMCMC approaches, we propose novel efficient strategies based on the
principle of Langevin diffusion and Hamiltonian dynamics in order to cope with
the increasing number of high-dimensional applications. Simulation results show
that the proposed algorithms achieve significantly better performance compared
to existing algorithms
Particle Efficient Importance Sampling
The efficient importance sampling (EIS) method is a general principle for the
numerical evaluation of high-dimensional integrals that uses the sequential
structure of target integrands to build variance minimising importance
samplers. Despite a number of successful applications in high dimensions, it is
well known that importance sampling strategies are subject to an exponential
growth in variance as the dimension of the integration increases. We solve this
problem by recognising that the EIS framework has an offline sequential Monte
Carlo interpretation. The particle EIS method is based on non-standard
resampling weights that take into account the look-ahead construction of the
importance sampler. We apply the method for a range of univariate and bivariate
stochastic volatility specifications. We also develop a new application of the
EIS approach to state space models with Student's t state innovations. Our
results show that the particle EIS method strongly outperforms both the
standard EIS method and particle filters for likelihood evaluation in high
dimensions. Moreover, the ratio between the variances of the particle EIS and
particle filter methods remains stable as the time series dimension increases.
We illustrate the efficiency of the method for Bayesian inference using the
particle marginal Metropolis-Hastings and importance sampling squared
algorithms
On particle filters applied to electricity load forecasting
We are interested in the online prediction of the electricity load, within
the Bayesian framework of dynamic models. We offer a review of sequential Monte
Carlo methods, and provide the calculations needed for the derivation of
so-called particles filters. We also discuss the practical issues arising from
their use, and some of the variants proposed in the literature to deal with
them, giving detailed algorithms whenever possible for an easy implementation.
We propose an additional step to help make basic particle filters more robust
with regard to outlying observations. Finally we use such a particle filter to
estimate a state-space model that includes exogenous variables in order to
forecast the electricity load for the customers of the French electricity
company \'Electricit\'e de France and discuss the various results obtained
- …