202,163 research outputs found
A Hierarchical Bayesian Framework for Constructing Sparsity-inducing Priors
Variable selection techniques have become increasingly popular amongst
statisticians due to an increased number of regression and classification
applications involving high-dimensional data where we expect some predictors to
be unimportant. In this context, Bayesian variable selection techniques
involving Markov chain Monte Carlo exploration of the posterior distribution
over models can be prohibitively computationally expensive and so there has
been attention paid to quasi-Bayesian approaches such as maximum a posteriori
(MAP) estimation using priors that induce sparsity in such estimates. We focus
on this latter approach, expanding on the hierarchies proposed to date to
provide a Bayesian interpretation and generalization of state-of-the-art
penalized optimization approaches and providing simultaneously a natural way to
include prior information about parameters within this framework. We give
examples of how to use this hierarchy to compute MAP estimates for linear and
logistic regression as well as sparse precision-matrix estimates in Gaussian
graphical models. In addition, an adaptive group lasso method is derived using
the framework.Comment: Submitted for publication; corrected typo
Bayesian inference in high-dimensional linear models using an empirical correlation-adaptive prior
In the context of a high-dimensional linear regression model, we propose the
use of an empirical correlation-adaptive prior that makes use of information in
the observed predictor variable matrix to adaptively address high collinearity,
determining if parameters associated with correlated predictors should be
shrunk together or kept apart. Under suitable conditions, we prove that this
empirical Bayes posterior concentrates around the true sparse parameter at the
optimal rate asymptotically. A simplified version of a shotgun stochastic
search algorithm is employed to implement the variable selection procedure, and
we show, via simulation experiments across different settings and a real-data
application, the favorable performance of the proposed method compared to
existing methods.Comment: 25 pages, 4 figures, 2 table
Function estimation with locally adaptive dynamic models
We present a nonparametric Bayesian method for fitting unsmooth and highly oscillating functions, which is based on a locally adaptive hierarchical extension of standard dynamic or state space models. The main idea is to introduce locally varying variances in the state equations and to add a further smoothness prior for this variance function. Estimation is fully Bayesian and carried out by recent MCMC techniques. The whole approach can be understood as an alternative to other nonparametric function estimators, such as local or penalized regression with variable bandwidth or smoothing parameter selection. Performance is illustrated with simulated data, including unsmooth examples constructed for wavelet shrinkage, and by an application to sales data. Although the approach is developed for classical Gaussian nonparametric regression, it can be extended to more complex regression problems
Bayesian Adaptive Selection of Variables for Function-on-Scalar Regression Models
Considering the field of functional data analysis, we developed a new
Bayesian method for variable selection in function-on-scalar regression (FOSR).
Our approach uses latent variables, allowing an adaptive selection since it can
determine the number of variables and which ones should be selected for a
function-on-scalar regression model. Simulation studies show the proposed
method's main properties, such as its accuracy in estimating the coefficients
and high capacity to select variables correctly. Furthermore, we conducted
comparative studies with the main competing methods, such as the BGLSS method
as well as the group LASSO, the group MCP and the group SCAD. We also used a
COVID-19 dataset and some socioeconomic data from Brazil for real data
application. In short, the proposed Bayesian variable selection model is
extremely competitive, showing significant predictive and selective quality
Comparison of Variable Selection Methods
Use of classic variable selection methods in public health research is quite common. Many criteria, and various strategies for applying them, now exist including forward selection, backward elimination, stepwise selection, best-subset selection and so on, but all suffer from similar drawbacks. Chief among them is a failure to account for the uncertainty contained in the model selection process. Ignoring model uncertainty can cause several serious problems. Variance estimates are generally underestimated, p-values are generally inflated, prediction ability is overestimated, and results are not reproducible in another dataset. Modern variable selection methods have become increasingly popular, especially in applications of high-dimensional or sparse data. Some of these methods were developed to address the short-comings of classic variable selection methods, such as backward elimination and stepwise selection methods. However, it remains unclear how modern variable selection methods behave in a classical, meaning non-high-dimensional, setting. A simulation study investigates the estimation, predictive performance and variable selection capabilities of three representative modern variable selection methods: Bayesian model averaging (BMA), stochastic search variable selection (SSVS), and the adaptive lasso. These three methods are considered in the setting of linear regression with a single variable of interest which is always included in the model. A second simulation study compares BMA to classical variable selection methods, including backward elimination, two-stage method, and change-in-effect method in the setting of logistic regression. Additionally, the data generated in both simulation studies closely mimic a real study and reflect a realistic correlation structure between potential covariates. Sample sizes ranging from 150 to 20000 are investigated. BMA is demonstrated in an example building a predictive model using data from the China Health and Nutrition Survey.Doctor of Public Healt
- …