Location of Repository

The Bayesian Backfitting Relevance Vector Machine

By Aaron D'Souza, Sethu Vijayakumar and Stefan Schaal


Traditional non-parametric statistical learning\ud techniques are often computationally attractive,\ud but lack the same generalization and\ud model selection abilities as state-of-the-art\ud Bayesian algorithms which, however, are usually\ud computationally prohibitive. This paper\ud makes several important contributions that\ud allow Bayesian learning to scale to more complex,\ud real-world learning scenarios. Firstly,\ud we show that back tting | a traditional\ud non-parametric, yet highly e cient regression\ud tool | can be derived in a novel formulation\ud within an expectation maximization\ud (EM) framework and thus can nally\ud be given a probabilistic interpretation. Secondly,\ud we show that the general framework\ud of sparse Bayesian learning and in particular\ud the relevance vector machine (RVM), can\ud be derived as a highly e cient algorithm using\ud a Bayesian version of back tting at its\ud core. As we demonstrate on several regression\ud and classi cation benchmarks, Bayesian\ud back tting o ers a compelling alternative to\ud current regression methods, especially when\ud the size and dimensionality of the data challenge\ud computational resources

Topics: Backfitting
Publisher: ACM Press
Year: 2010
DOI identifier: 10.1145/1015330.1015358
OAI identifier: oai:www.era.lib.ed.ac.uk:1842/3694

Suggested articles



  1. (2000). An O(n) algorithm for incremental real time learning in high dimensional space.
  2. (1994). Bayesian learning for neural networks. doi
  3. (2000). Bayesian parameter estimation via variational methods.
  4. (1999). Comparison of approximate methods for handling hyperparameters. doi
  5. (2004). Dimensionality reduction for supervised learning using reproducing kernel Hilbert spaces.
  6. (2003). Fast marginal likelihood maximization for sparse Bayesian models.
  7. (1996). Gaussian processes for regression.
  8. (1990). Generalized additive models. doi
  9. (1998). Local dimensionality reduction. doi
  10. (1992). Numerical recipes in C: The art of scienti computing. doi
  11. (1965). Principal component regression in exploratory statistical research. doi
  12. (1975). Soft modeling by latent variables: The nonlinear iterative partial least squares approach. In doi
  13. (2001). Sparse Bayesian learning and the relevance vector machine. doi
  14. (2001). Sparse representation for Gaussian process models.
  15. (2001). Using the Nystr om method to speed up kernel machines.
  16. (2000). Variational inference for Bayesian mixtures of factor analysers.
  17. (2000). Variational relevance vector machine.

To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.