211 research outputs found
Jeffreys priors for mixture estimation: properties and alternatives
While Jeffreys priors usually are well-defined for the parameters of mixtures
of distributions, they are not available in closed form. Furthermore, they
often are improper priors. Hence, they have never been used to draw inference
on the mixture parameters. The implementation and the properties of Jeffreys
priors in several mixture settings are studied. It is shown that the associated
posterior distributions most often are improper. Nevertheless, the Jeffreys
prior for the mixture weights conditionally on the parameters of the mixture
components will be shown to have the property of conservativeness with respect
to the number of components, in case of overfitted mixture and it can be
therefore used as a default priors in this context.Comment: arXiv admin note: substantial text overlap with arXiv:1511.0314
Control Variates for Reversible MCMC Samplers
A general methodology is introduced for the construction and effective
application of control variates to estimation problems involving data from
reversible MCMC samplers. We propose the use of a specific class of functions
as control variates, and we introduce a new, consistent estimator for the
values of the coefficients of the optimal linear combination of these
functions. The form and proposed construction of the control variates is
derived from our solution of the Poisson equation associated with a specific
MCMC scenario. The new estimator, which can be applied to the same MCMC sample,
is derived from a novel, finite-dimensional, explicit representation for the
optimal coefficients. The resulting variance-reduction methodology is primarily
applicable when the simulated data are generated by a conjugate random-scan
Gibbs sampler. MCMC examples of Bayesian inference problems demonstrate that
the corresponding reduction in the estimation variance is significant, and that
in some cases it can be quite dramatic. Extensions of this methodology in
several directions are given, including certain families of Metropolis-Hastings
samplers and hybrid Metropolis-within-Gibbs algorithms. Corresponding
simulation examples are presented illustrating the utility of the proposed
methods. All methodological and asymptotic arguments are rigorously justified
under easily verifiable and essentially minimal conditions.Comment: 44 pages; 6 figures; 5 table
Likelihood-informed dimension reduction for nonlinear inverse problems
The intrinsic dimensionality of an inverse problem is affected by prior
information, the accuracy and number of observations, and the smoothing
properties of the forward operator. From a Bayesian perspective, changes from
the prior to the posterior may, in many problems, be confined to a relatively
low-dimensional subspace of the parameter space. We present a dimension
reduction approach that defines and identifies such a subspace, called the
"likelihood-informed subspace" (LIS), by characterizing the relative influences
of the prior and the likelihood over the support of the posterior distribution.
This identification enables new and more efficient computational methods for
Bayesian inference with nonlinear forward models and Gaussian priors. In
particular, we approximate the posterior distribution as the product of a
lower-dimensional posterior defined on the LIS and the prior distribution
marginalized onto the complementary subspace. Markov chain Monte Carlo sampling
can then proceed in lower dimensions, with significant gains in computational
efficiency. We also introduce a Rao-Blackwellization strategy that
de-randomizes Monte Carlo estimates of posterior expectations for additional
variance reduction. We demonstrate the efficiency of our methods using two
numerical examples: inference of permeability in a groundwater system governed
by an elliptic PDE, and an atmospheric remote sensing problem based on Global
Ozone Monitoring System (GOMOS) observations
Likelihood-Based Inference for Discretely Observed Birth-Death-Shift Processes, with Applications to Evolution of Mobile Genetic Elements
Continuous-time birth-death-shift (BDS) processes are frequently used in
stochastic modeling, with many applications in ecology and epidemiology. In
particular, such processes can model evolutionary dynamics of transposable
elements - important genetic markers in molecular epidemiology. Estimation of
the effects of individual covariates on the birth, death, and shift rates of
the process can be accomplished by analyzing patient data, but inferring these
rates in a discretely and unevenly observed setting presents computational
challenges. We propose a mutli-type branching process approximation to BDS
processes and develop a corresponding expectation maximization (EM) algorithm,
where we use spectral techniques to reduce calculation of expected sufficient
statistics to low dimensional integration. These techniques yield an efficient
and robust optimization routine for inferring the rates of the BDS process, and
apply more broadly to multi-type branching processes where rates can depend on
many covariates. After rigorously testing our methodology in simulation
studies, we apply our method to study intrapatient time evolution of IS6110
transposable element, a frequently used element during estimation of
epidemiological clusters of Mycobacterium tuberculosis infections.Comment: 31 pages, 7 figures, 1 tabl
A computational framework for infinite-dimensional Bayesian inverse problems: Part II. Stochastic Newton MCMC with application to ice sheet flow inverse problems
We address the numerical solution of infinite-dimensional inverse problems in
the framework of Bayesian inference. In the Part I companion to this paper
(arXiv.org:1308.1313), we considered the linearized infinite-dimensional
inverse problem. Here in Part II, we relax the linearization assumption and
consider the fully nonlinear infinite-dimensional inverse problem using a
Markov chain Monte Carlo (MCMC) sampling method. To address the challenges of
sampling high-dimensional pdfs arising from Bayesian inverse problems governed
by PDEs, we build on the stochastic Newton MCMC method. This method exploits
problem structure by taking as a proposal density a local Gaussian
approximation of the posterior pdf, whose construction is made tractable by
invoking a low-rank approximation of its data misfit component of the Hessian.
Here we introduce an approximation of the stochastic Newton proposal in which
we compute the low-rank-based Hessian at just the MAP point, and then reuse
this Hessian at each MCMC step. We compare the performance of the proposed
method to the original stochastic Newton MCMC method and to an independence
sampler. The comparison of the three methods is conducted on a synthetic ice
sheet inverse problem. For this problem, the stochastic Newton MCMC method with
a MAP-based Hessian converges at least as rapidly as the original stochastic
Newton MCMC method, but is far cheaper since it avoids recomputing the Hessian
at each step. On the other hand, it is more expensive per sample than the
independence sampler; however, its convergence is significantly more rapid, and
thus overall it is much cheaper. Finally, we present extensive analysis and
interpretation of the posterior distribution, and classify directions in
parameter space based on the extent to which they are informed by the prior or
the observations.Comment: 31 page
Non-Gaussian bivariate modelling with application to atmospheric trace-gas inversion
Atmospheric trace-gas inversion is the procedure by which the sources and
sinks of a trace gas are identified from observations of its mole fraction at
isolated locations in space and time. This is inherently a spatio-temporal
bivariate inversion problem, since the mole-fraction field evolves in space and
time and the flux is also spatio-temporally distributed. Further, the bivariate
model is likely to be non-Gaussian since the flux field is rarely Gaussian.
Here, we use conditioning to construct a non-Gaussian bivariate model, and we
describe some of its properties through auto- and cross-cumulant functions. A
bivariate non-Gaussian, specifically trans-Gaussian, model is then achieved
through the use of Box--Cox transformations, and we facilitate Bayesian
inference by approximating the likelihood in a hierarchical framework.
Trace-gas inversion, especially at high spatial resolution, is frequently
highly sensitive to prior specification. Therefore, unlike conventional
approaches, we assimilate trace-gas inventory information with the
observational data at the parameter layer, thus shifting prior sensitivity from
the inventory itself to its spatial characteristics (e.g., its spatial length
scale). We demonstrate the approach in controlled-experiment studies of methane
inversion, using fluxes extracted from inventories of the UK and Ireland and of
Northern Australia.Comment: 45 pages, 7 figure
Efficient real-time monitoring of an emerging influenza pandemic: How feasible?
A prompt public health response to a new epidemic relies on the ability to monitor and predict its evolution in real time as data accumulate. The 2009 A/H1N1 outbreak in the UK revealed pandemic data as noisy, contaminated, potentially biased and originating from multiple sources. This seriously challenges the capacity for real-time monitoring. Here, we assess the feasibility of real-time inference based on such data by constructing an analytic tool combining an age-stratified SEIR transmission model with various observation models describing the data generation mechanisms. As batches of data become available, a sequential Monte Carlo (SMC) algorithm is developed to synthesise multiple imperfect data streams, iterate epidemic inferences and assess model adequacy amidst a rapidly evolving epidemic environment, substantially reducing computation time in comparison to standard MCMC, to ensure timely delivery of real-time epidemic assessments. In application to simulated data designed to mimic the 2009 A/H1N1 epidemic, SMC is shown to have additional benefits in terms of assessing predictive performance and coping with parameter nonidentifiability
Efficient, concurrent Bayesian analysis of full waveform LaDAR data
Bayesian analysis of full waveform laser detection and ranging (LaDAR)
signals using reversible jump Markov chain Monte Carlo (RJMCMC) algorithms
have shown higher estimation accuracy, resolution and sensitivity to
detect weak signatures for 3D surface profiling, and construct multiple layer
images with varying number of surface returns. However, it is computational
expensive. Although parallel computing has the potential to reduce both the
processing time and the requirement for persistent memory storage, parallelizing
the serial sampling procedure in RJMCMC is a significant challenge
in both statistical and computing domains. While several strategies have been
developed for Markov chain Monte Carlo (MCMC) parallelization, these are
usually restricted to fixed dimensional parameter estimates, and not obviously
applicable to RJMCMC for varying dimensional signal analysis.
In the statistical domain, we propose an effective, concurrent RJMCMC algorithm,
state space decomposition RJMCMC (SSD-RJMCMC), which divides
the entire state space into groups and assign to each an independent
RJMCMC chain with restricted variation of model dimensions. It intrinsically
has a parallel structure, a form of model-level parallelization. Applying
the convergence diagnostic, we can adaptively assess the convergence of the
Markov chain on-the-fly and so dynamically terminate the chain generation.
Evaluations on both synthetic and real data demonstrate that the concurrent
chains have shorter convergence length and hence improved sampling efficiency.
Parallel exploration of the candidate models, in conjunction with an
error detection and correction scheme, improves the reliability of surface detection.
By adaptively generating a complimentary MCMC sequence for the
determined model, it enhances the accuracy for surface profiling.
In the computing domain, we develop a data parallel SSD-RJMCMC (DP
SSD-RJMCMCU) to achieve efficient parallel implementation on a distributed
computer cluster. Adding data-level parallelization on top of the model-level
parallelization, it formalizes a task queue and introduces an automatic scheduler
for dynamic task allocation. These two strategies successfully diminish
the load imbalance that occurred in SSD-RJMCMC. Thanks to the coarse
granularity, the processors communicate at a very low frequency. The MPIbased
implementation on a Beowulf cluster demonstrates that compared with
RJMCMC, DP SSD-RJMCMCU has further reduced problem size and computation
complexity. Therefore, it can achieve a super linear speedup if the
number of data segments and processors are chosen wisely
- …