6 research outputs found
Horseshoe priors for edge-preserving linear Bayesian inversion
In many large-scale inverse problems, such as computed tomography and image
deblurring, characterization of sharp edges in the solution is desired. Within
the Bayesian approach to inverse problems, edge-preservation is often achieved
using Markov random field priors based on heavy-tailed distributions. Another
strategy, popular in statistics, is the application of hierarchical shrinkage
priors. An advantage of this formulation lies in expressing the prior as a
conditionally Gaussian distribution depending of global and local
hyperparameters which are endowed with heavy-tailed hyperpriors. In this work,
we revisit the shrinkage horseshoe prior and introduce its formulation for
edge-preserving settings. We discuss a sampling framework based on the Gibbs
sampler to solve the resulting hierarchical formulation of the Bayesian inverse
problem. In particular, one of the conditional distributions is
high-dimensional Gaussian, and the rest are derived in closed form by using a
scale mixture representation of the heavy-tailed hyperpriors. Applications from
imaging science show that our computational procedure is able to compute sharp
edge-preserving posterior point estimates with reduced uncertainty
A Horseshoe Pit mixture model for Bayesian screening with an application to light sheet fluorescence microscopy in brain imaging
Finding parsimonious models through variable selection is a fundamental
problem in many areas of statistical inference. Here, we focus on Bayesian
regression models, where variable selection can be implemented through a
regularizing prior imposed on the distribution of the regression coefficients.
In the Bayesian literature, there are two main types of priors used to
accomplish this goal: the spike-and-slab and the continuous scale mixtures of
Gaussians. The former is a discrete mixture of two distributions characterized
by low and high variance. In the latter, a continuous prior is elicited on the
scale of a zero-mean Gaussian distribution. In contrast to these existing
methods, we propose a new class of priors based on discrete mixture of
continuous scale mixtures providing a more general framework for Bayesian
variable selection. To this end, we substitute the observation-specific local
shrinkage parameters (typical of continuous mixtures) with mixture component
shrinkage parameters. Our approach drastically reduces the number of parameters
needed and allows sharing information across the coefficients, improving the
shrinkage effect. By using half-Cauchy distributions, this approach leads to a
cluster-shrinkage version of the Horseshoe prior. We present the properties of
our model and showcase its estimation and prediction performance in a
simulation study. We then recast the model in a multiple hypothesis testing
framework and apply it to a neurological dataset obtained using a novel
whole-brain imaging technique
Group Inverse-Gamma Gamma Shrinkage for Sparse Regression with Block-Correlated Predictors
Heavy-tailed continuous shrinkage priors, such as the horseshoe prior, are
widely used for sparse estimation problems. However, there is limited work
extending these priors to predictors with grouping structures. Of particular
interest in this article, is regression coefficient estimation where pockets of
high collinearity in the covariate space are contained within known covariate
groupings. To assuage variance inflation due to multicollinearity we propose
the group inverse-gamma gamma (GIGG) prior, a heavy-tailed prior that can
trade-off between local and group shrinkage in a data adaptive fashion. A
special case of the GIGG prior is the group horseshoe prior, whose shrinkage
profile is correlated within-group such that the regression coefficients
marginally have exact horseshoe regularization. We show posterior consistency
for regression coefficients in linear regression models and posterior
concentration results for mean parameters in sparse normal means models. The
full conditional distributions corresponding to GIGG regression can be derived
in closed form, leading to straightforward posterior computation. We show that
GIGG regression results in low mean-squared error across a wide range of
correlation structures and within-group signal densities via simulation. We
apply GIGG regression to data from the National Health and Nutrition
Examination Survey for associating environmental exposures with liver
functionality.Comment: 44 pages, 4 figure
Dynamic Shrinkage Priors for Large Time-varying Parameter Regressions using Scalable Markov Chain Monte Carlo Methods
Time-varying parameter (TVP) regression models can involve a huge number of
coefficients. Careful prior elicitation is required to yield sensible posterior
and predictive inferences. In addition, the computational demands of Markov
Chain Monte Carlo (MCMC) methods mean their use is limited to the case where
the number of predictors is not too large. In light of these two concerns, this
paper proposes a new dynamic shrinkage prior which reflects the empirical
regularity that TVPs are typically sparse (i.e. time variation may occur only
episodically and only for some of the coefficients). A scalable MCMC algorithm
is developed which is capable of handling very high dimensional TVP regressions
or TVP Vector Autoregressions. In an exercise using artificial data we
demonstrate the accuracy and computational efficiency of our methods. In an
application involving the term structure of interest rates in the eurozone, we
find our dynamic shrinkage prior to effectively pick out small amounts of
parameter change and our methods to forecast well.Comment: Keywords: Time-varying parameter regression, dynamic shrinkage prior,
global-local shrinkage prior, Bayesian variable selection, scalable Markov
Chain Monte Carlo JEL Codes: C11, C30, E3, D3
Dimension-free Mixing for High-dimensional Bayesian Variable Selection
Yang et al. (2016) proved that the symmetric random walk Metropolis--Hastings
algorithm for Bayesian variable selection is rapidly mixing under mild
high-dimensional assumptions. We propose a novel MCMC sampler using an informed
proposal scheme, which we prove achieves a much faster mixing time that is
independent of the number of covariates, under the same assumptions. To the
best of our knowledge, this is the first high-dimensional result which
rigorously shows that the mixing rate of informed MCMC methods can be fast
enough to offset the computational cost of local posterior evaluation.
Motivated by the theoretical analysis of our sampler, we further propose a new
approach called "two-stage drift condition" to studying convergence rates of
Markov chains on general state spaces, which can be useful for obtaining tight
complexity bounds in high-dimensional settings. The practical advantages of our
algorithm are illustrated by both simulation studies and real data analysis