6 research outputs found
The Bayesian Backfitting Relevance Vector Machine
Traditional non-parametric statistical learning
techniques are often computationally attractive,
but lack the same generalization and
model selection abilities as state-of-the-art
Bayesian algorithms which, however, are usually
computationally prohibitive. This paper
makes several important contributions that
allow Bayesian learning to scale to more complex,
real-world learning scenarios. Firstly,
we show that back tting | a traditional
non-parametric, yet highly e cient regression
tool | can be derived in a novel formulation
within an expectation maximization
(EM) framework and thus can nally
be given a probabilistic interpretation. Secondly,
we show that the general framework
of sparse Bayesian learning and in particular
the relevance vector machine (RVM), can
be derived as a highly e cient algorithm using
a Bayesian version of back tting at its
core. As we demonstrate on several regression
and classi cation benchmarks, Bayesian
back tting o ers a compelling alternative to
current regression methods, especially when
the size and dimensionality of the data challenge
computational resources
Bayesian Kernel Shaping for Learning Control
In kernel-based regression learning, optimizing each kernel individually is useful
when the data density, curvature of regression surfaces (or decision boundaries)
or magnitude of output noise varies spatially. Previous work has suggested gradient
descent techniques or complex statistical hypothesis methods for local kernel
shaping, typically requiring some amount of manual tuning of meta parameters.
We introduce a Bayesian formulation of nonparametric regression that, with the
help of variational approximations, results in an EM-like algorithm for simultaneous
estimation of regression and kernel parameters. The algorithm is computationally
efficient, requires no sampling, automatically rejects outliers and has only
one prior to be specified. It can be used for nonparametric regression with local
polynomials or as a novel method to achieve nonstationary regression with Gaussian
processes. Our methods are particularly useful for learning control, where
reliable estimation of local tangent planes is essential for adaptive controllers and
reinforcement learning. We evaluate our methods on several synthetic data sets
and on an actual robot which learns a task-level control law
Bayesian Kernel Shaping for Learning Control
In kernel-based regression learning, optimizing each kernel individually is useful when the data density, curvature of regression surfaces (or decision boundaries) or magnitude of output noise varies spatially. Previous work has suggested gradient descent techniques or complex statistical hypothesis methods for local kernel shaping, typically requiring some amount of manual tuning of meta parameters. We introduce a Bayesian formulation of nonparametric regression that, with the help of variational approximations, results in an EM-like algorithm for simultaneous estimation of regression and kernel parameters. The algorithm is computationally efficient, requires no sampling, automatically rejects outliers and has only one prior to be specified. It can be used for nonparametric regression with local polynomials or as a novel method to achieve nonstationary regression with Gaussian processes. Our methods are particularly useful for learning control, where reliable estimation of local tangent planes is essential for adaptive controllers and reinforcement learning. We evaluate our methods on several synthetic data sets and on an actual robot which learns a task-level control law.
the Bayesian backfitting relevance vector machine
Traditional non-parametric statistical learning techniques are often computationally attractive, but lack the same generalization and model selection abilities as state-of-the-art Bayesian algorithms which, however, are usually computationally prohibitive. This paper makes several important contributions that allow Bayesian learning to scale to more complex, real-world learning scenarios. Firstly, we show that backfitting — a traditional non-parametric, yet highly efficient regression tool — can be derived in a novel formulation within an expectation maximization (EM) framework and thus can finally be given a probabilistic interpretation. Secondly, we show that the general framework of sparse Bayesian learning and in particular the relevance vector machine (RVM), can be derived as a highly efficient algorithm using a Bayesian version of backfitting at its core. As we demonstrate on several regression and classification benchmarks, Bayesian backfitting offers a compelling alternative to current regression methods, especially when the size and dimensionality of the data challenge computational resources. 1