19,827 research outputs found
Bayesian mixed-effects inference on classification performance in hierarchical data sets
Classification algorithms are frequently used on data with a natural hierarchical structure. For instance, classifiers are often trained and tested on trial-wise measurements, separately for each subject within a group. One important question is how classification outcomes observed in individual subjects can be generalized to the population from which the group was sampled. To address this question, this paper introduces novel statistical models that are guided by three desiderata. First, all models explicitly respect the hierarchical nature of the data, that is, they are mixed-effects models that simultaneously account for within-subjects (fixed-effects) and across-subjects (random-effects) variance components. Second, maximum-likelihood estimation is replaced by full Bayesian inference in order to enable natural regularization of the estimation problem and to afford conclusions in terms of posterior probability statements. Third, inference on classification accuracy is complemented by inference on the balanced accuracy, which avoids inflated accuracy estimates for imbalanced data sets. We introduce hierarchical models that satisfy these criteria and demonstrate their advantages over conventional methods usingMCMC implementations for model inversion and model selection on both synthetic and empirical data. We envisage that our approach will improve the sensitivity and validity of statistical inference in future hierarchical classification studies. © 2012
Sparse Bayesian variable selection for the identification of antigenic variability in the Foot-and-Mouth disease virus
Vaccines created from closely related viruses are vital for offering protection against newly emerging strains. For Foot-and-Mouth disease virus (FMDV), where multiple serotypes co-circulate, testing large numbers of vaccines can be infeasible. Therefore the development of an in silico predictor of cross-
protection between strains is important to help optimise vaccine choice. Here we describe a novel sparse Bayesian variable selection model using spike and slab priors which is able to predict antigenic variability and identify sites which are important for the neutralisation of the virus. We are able to iden-
tify multiple residues which are known to be key indicators of antigenic variability. Many of these were not identified previously using frequentist mixed-effects models and still cannot be found when an ℓ1 penalty is used. We further explore how the Markov chain Monte Carlo (MCMC) proposal method for the inclusion of variables can offer significant reductions in computational requirements, both for spike and slab priors in general, and
our hierarchical Bayesian model in particular
Sparse Probit Linear Mixed Model
Linear Mixed Models (LMMs) are important tools in statistical genetics. When
used for feature selection, they allow to find a sparse set of genetic traits
that best predict a continuous phenotype of interest, while simultaneously
correcting for various confounding factors such as age, ethnicity and
population structure. Formulated as models for linear regression, LMMs have
been restricted to continuous phenotypes. We introduce the Sparse Probit Linear
Mixed Model (Probit-LMM), where we generalize the LMM modeling paradigm to
binary phenotypes. As a technical challenge, the model no longer possesses a
closed-form likelihood function. In this paper, we present a scalable
approximate inference algorithm that lets us fit the model to high-dimensional
data sets. We show on three real-world examples from different domains that in
the setup of binary labels, our algorithm leads to better prediction accuracies
and also selects features which show less correlation with the confounding
factors.Comment: Published version, 21 pages, 6 figure
Spike-and-Slab Priors for Function Selection in Structured Additive Regression Models
Structured additive regression provides a general framework for complex
Gaussian and non-Gaussian regression models, with predictors comprising
arbitrary combinations of nonlinear functions and surfaces, spatial effects,
varying coefficients, random effects and further regression terms. The large
flexibility of structured additive regression makes function selection a
challenging and important task, aiming at (1) selecting the relevant
covariates, (2) choosing an appropriate and parsimonious representation of the
impact of covariates on the predictor and (3) determining the required
interactions. We propose a spike-and-slab prior structure for function
selection that allows to include or exclude single coefficients as well as
blocks of coefficients representing specific model terms. A novel
multiplicative parameter expansion is required to obtain good mixing and
convergence properties in a Markov chain Monte Carlo simulation approach and is
shown to induce desirable shrinkage properties. In simulation studies and with
(real) benchmark classification data, we investigate sensitivity to
hyperparameter settings and compare performance to competitors. The flexibility
and applicability of our approach are demonstrated in an additive piecewise
exponential model with time-varying effects for right-censored survival times
of intensive care patients with sepsis. Geoadditive and additive mixed logit
model applications are discussed in an extensive appendix
Generalized fiducial inference for normal linear mixed models
While linear mixed modeling methods are foundational concepts introduced in
any statistical education, adequate general methods for interval estimation
involving models with more than a few variance components are lacking,
especially in the unbalanced setting. Generalized fiducial inference provides a
possible framework that accommodates this absence of methodology. Under the
fabric of generalized fiducial inference along with sequential Monte Carlo
methods, we present an approach for interval estimation for both balanced and
unbalanced Gaussian linear mixed models. We compare the proposed method to
classical and Bayesian results in the literature in a simulation study of
two-fold nested models and two-factor crossed designs with an interaction term.
The proposed method is found to be competitive or better when evaluated based
on frequentist criteria of empirical coverage and average length of confidence
intervals for small sample sizes. A MATLAB implementation of the proposed
algorithm is available from the authors.Comment: Published in at http://dx.doi.org/10.1214/12-AOS1030 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
Efficient Bayesian hierarchical functional data analysis with basis function approximations using Gaussian-Wishart processes
Functional data are defined as realizations of random functions (mostly
smooth functions) varying over a continuum, which are usually collected with
measurement errors on discretized grids. In order to accurately smooth noisy
functional observations and deal with the issue of high-dimensional observation
grids, we propose a novel Bayesian method based on the Bayesian hierarchical
model with a Gaussian-Wishart process prior and basis function representations.
We first derive an induced model for the basis-function coefficients of the
functional data, and then use this model to conduct posterior inference through
Markov chain Monte Carlo. Compared to the standard Bayesian inference that
suffers serious computational burden and unstableness for analyzing
high-dimensional functional data, our method greatly improves the computational
scalability and stability, while inheriting the advantage of simultaneously
smoothing raw observations and estimating the mean-covariance functions in a
nonparametric way. In addition, our method can naturally handle functional data
observed on random or uncommon grids. Simulation and real studies demonstrate
that our method produces similar results as the standard Bayesian inference
with low-dimensional common grids, while efficiently smoothing and estimating
functional data with random and high-dimensional observation grids where the
standard Bayesian inference fails. In conclusion, our method can efficiently
smooth and estimate high-dimensional functional data, providing one way to
resolve the curse of dimensionality for Bayesian functional data analysis with
Gaussian-Wishart processes.Comment: Under revie
Bayesian Approximate Kernel Regression with Variable Selection
Nonlinear kernel regression models are often used in statistics and machine
learning because they are more accurate than linear models. Variable selection
for kernel regression models is a challenge partly because, unlike the linear
regression setting, there is no clear concept of an effect size for regression
coefficients. In this paper, we propose a novel framework that provides an
effect size analog of each explanatory variable for Bayesian kernel regression
models when the kernel is shift-invariant --- for example, the Gaussian kernel.
We use function analytic properties of shift-invariant reproducing kernel
Hilbert spaces (RKHS) to define a linear vector space that: (i) captures
nonlinear structure, and (ii) can be projected onto the original explanatory
variables. The projection onto the original explanatory variables serves as an
analog of effect sizes. The specific function analytic property we use is that
shift-invariant kernel functions can be approximated via random Fourier bases.
Based on the random Fourier expansion we propose a computationally efficient
class of Bayesian approximate kernel regression (BAKR) models for both
nonlinear regression and binary classification for which one can compute an
analog of effect sizes. We illustrate the utility of BAKR by examining two
important problems in statistical genetics: genomic selection (i.e. phenotypic
prediction) and association mapping (i.e. inference of significant variants or
loci). State-of-the-art methods for genomic selection and association mapping
are based on kernel regression and linear models, respectively. BAKR is the
first method that is competitive in both settings.Comment: 22 pages, 3 figures, 3 tables; theory added; new simulations
presented; references adde
Mean field variational Bayesian inference for support vector machine classification
A mean field variational Bayes approach to support vector machines (SVMs)
using the latent variable representation on Polson & Scott (2012) is presented.
This representation allows circumvention of many of the shortcomings associated
with classical SVMs including automatic penalty parameter selection, the
ability to handle dependent samples, missing data and variable selection. We
demonstrate on simulated and real datasets that our approach is easily
extendable to non-standard situations and outperforms the classical SVM
approach whilst remaining computationally efficient.Comment: 18 pages, 4 figure
- …