3,877 research outputs found
A Noise-Robust Fast Sparse Bayesian Learning Model
This paper utilizes the hierarchical model structure from the Bayesian Lasso
in the Sparse Bayesian Learning process to develop a new type of probabilistic
supervised learning approach. The hierarchical model structure in this Bayesian
framework is designed such that the priors do not only penalize the unnecessary
complexity of the model but will also be conditioned on the variance of the
random noise in the data. The hyperparameters in the model are estimated by the
Fast Marginal Likelihood Maximization algorithm which can achieve sparsity, low
computational cost and faster learning process. We compare our methodology with
two other popular learning models; the Relevance Vector Machine and the
Bayesian Lasso. We test our model on examples involving both simulated and
empirical data, and the results show that this approach has several performance
advantages, such as being fast, sparse and also robust to the variance in
random noise. In addition, our method can give out a more stable estimation of
variance of random error, compared with the other methods in the study.Comment: 15 page
Sparse Probit Linear Mixed Model
Linear Mixed Models (LMMs) are important tools in statistical genetics. When
used for feature selection, they allow to find a sparse set of genetic traits
that best predict a continuous phenotype of interest, while simultaneously
correcting for various confounding factors such as age, ethnicity and
population structure. Formulated as models for linear regression, LMMs have
been restricted to continuous phenotypes. We introduce the Sparse Probit Linear
Mixed Model (Probit-LMM), where we generalize the LMM modeling paradigm to
binary phenotypes. As a technical challenge, the model no longer possesses a
closed-form likelihood function. In this paper, we present a scalable
approximate inference algorithm that lets us fit the model to high-dimensional
data sets. We show on three real-world examples from different domains that in
the setup of binary labels, our algorithm leads to better prediction accuracies
and also selects features which show less correlation with the confounding
factors.Comment: Published version, 21 pages, 6 figure
The Kernel Interaction Trick: Fast Bayesian Discovery of Pairwise Interactions in High Dimensions
Discovering interaction effects on a response of interest is a fundamental
problem faced in biology, medicine, economics, and many other scientific
disciplines. In theory, Bayesian methods for discovering pairwise interactions
enjoy many benefits such as coherent uncertainty quantification, the ability to
incorporate background knowledge, and desirable shrinkage properties. In
practice, however, Bayesian methods are often computationally intractable for
even moderate-dimensional problems. Our key insight is that many hierarchical
models of practical interest admit a particular Gaussian process (GP)
representation; the GP allows us to capture the posterior with a vector of O(p)
kernel hyper-parameters rather than O(p^2) interactions and main effects. With
the implicit representation, we can run Markov chain Monte Carlo (MCMC) over
model hyper-parameters in time and memory linear in p per iteration. We focus
on sparsity-inducing models and show on datasets with a variety of covariate
behaviors that our method: (1) reduces runtime by orders of magnitude over
naive applications of MCMC, (2) provides lower Type I and Type II error
relative to state-of-the-art LASSO-based approaches, and (3) offers improved
computational scaling in high dimensions relative to existing Bayesian and
LASSO-based approaches.Comment: Accepted at ICML 2019. 20 pages, 4 figures, 3 table
- …