10,829 research outputs found
Bayesian Nonparametric Adaptive Control using Gaussian Processes
This technical report is a preprint of an article submitted to a journal.Most current Model Reference Adaptive Control
(MRAC) methods rely on parametric adaptive elements, in
which the number of parameters of the adaptive element are
fixed a priori, often through expert judgment. An example of
such an adaptive element are Radial Basis Function Networks
(RBFNs), with RBF centers pre-allocated based on the expected
operating domain. If the system operates outside of the expected
operating domain, this adaptive element can become
non-effective in capturing and canceling the uncertainty, thus
rendering the adaptive controller only semi-global in nature.
This paper investigates a Gaussian Process (GP) based Bayesian
MRAC architecture (GP-MRAC), which leverages the power and
flexibility of GP Bayesian nonparametric models of uncertainty.
GP-MRAC does not require the centers to be preallocated, can
inherently handle measurement noise, and enables MRAC to
handle a broader set of uncertainties, including those that are
defined as distributions over functions. We use stochastic stability
arguments to show that GP-MRAC guarantees good closed loop
performance with no prior domain knowledge of the uncertainty.
Online implementable GP inference methods are compared in
numerical simulations against RBFN-MRAC with preallocated
centers and are shown to provide better tracking and improved
long-term learning.This research was supported in part by ONR MURI Grant
N000141110688 and NSF grant ECS #0846750
Locally adaptive smoothing with Markov random fields and shrinkage priors
We present a locally adaptive nonparametric curve fitting method that
operates within a fully Bayesian framework. This method uses shrinkage priors
to induce sparsity in order-k differences in the latent trend function,
providing a combination of local adaptation and global control. Using a scale
mixture of normals representation of shrinkage priors, we make explicit
connections between our method and kth order Gaussian Markov random field
smoothing. We call the resulting processes shrinkage prior Markov random fields
(SPMRFs). We use Hamiltonian Monte Carlo to approximate the posterior
distribution of model parameters because this method provides superior
performance in the presence of the high dimensionality and strong parameter
correlations exhibited by our models. We compare the performance of three prior
formulations using simulated data and find the horseshoe prior provides the
best compromise between bias and precision. We apply SPMRF models to two
benchmark data examples frequently used to test nonparametric methods. We find
that this method is flexible enough to accommodate a variety of data generating
models and offers the adaptive properties and computational tractability to
make it a useful addition to the Bayesian nonparametric toolbox.Comment: 38 pages, to appear in Bayesian Analysi
Stochastic expansions using continuous dictionaries: L\'{e}vy adaptive regression kernels
This article describes a new class of prior distributions for nonparametric
function estimation. The unknown function is modeled as a limit of weighted
sums of kernels or generator functions indexed by continuous parameters that
control local and global features such as their translation, dilation,
modulation and shape. L\'{e}vy random fields and their stochastic integrals are
employed to induce prior distributions for the unknown functions or,
equivalently, for the number of kernels and for the parameters governing their
features. Scaling, shape, and other features of the generating functions are
location-specific to allow quite different function properties in different
parts of the space, as with wavelet bases and other methods employing
overcomplete dictionaries. We provide conditions under which the stochastic
expansions converge in specified Besov or Sobolev norms. Under a Gaussian error
model, this may be viewed as a sparse regression problem, with regularization
induced via the L\'{e}vy random field prior distribution. Posterior inference
for the unknown functions is based on a reversible jump Markov chain Monte
Carlo algorithm. We compare the L\'{e}vy Adaptive Regression Kernel (LARK)
method to wavelet-based methods using some of the standard test functions, and
illustrate its flexibility and adaptability in nonstationary applications.Comment: Published in at http://dx.doi.org/10.1214/11-AOS889 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
Adaptive Bayesian estimation using a Gaussian random field with inverse Gamma bandwidth
We consider nonparametric Bayesian estimation inference using a rescaled
smooth Gaussian field as a prior for a multidimensional function. The rescaling
is achieved using a Gamma variable and the procedure can be viewed as choosing
an inverse Gamma bandwidth. The procedure is studied from a frequentist
perspective in three statistical settings involving replicated observations
(density estimation, regression and classification). We prove that the
resulting posterior distribution shrinks to the distribution that generates the
data at a speed which is minimax-optimal up to a logarithmic factor, whatever
the regularity level of the data-generating distribution. Thus the hierachical
Bayesian procedure, with a fixed prior, is shown to be fully adaptive.Comment: Published in at http://dx.doi.org/10.1214/08-AOS678 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
Posterior concentration rates for empirical Bayes procedures, with applications to Dirichlet Process mixtures
In this paper we provide general conditions to check on the model and the
prior to derive posterior concentration rates for data-dependent priors (or
empirical Bayes approaches). We aim at providing conditions that are close to
the conditions provided in the seminal paper by Ghosal and van der Vaart
(2007a). We then apply the general theorem to two different settings: the
estimation of a density using Dirichlet process mixtures of Gaussian random
variables with base measure depending on some empirical quantities and the
estimation of the intensity of a counting process under the Aalen model. A
simulation study for inhomogeneous Poisson processes also illustrates our
results. In the former case we also derive some results on the estimation of
the mixing density and on the deconvolution problem. In the latter, we provide
a general theorem on posterior concentration rates for counting processes with
Aalen multiplicative intensity with priors not depending on the data.Comment: With supplementary materia
Nonparametric statistical inference for drift vector fields of multi-dimensional diffusions
The problem of determining a periodic Lipschitz vector field from an observed trajectory of the solution of the
multi-dimensional stochastic differential equation \begin{equation*} dX_t =
b(X_t)dt + dW_t, \quad t \geq 0, \end{equation*} where is a standard
-dimensional Brownian motion, is considered. Convergence rates of a
penalised least squares estimator, which equals the maximum a posteriori (MAP)
estimate corresponding to a high-dimensional Gaussian product prior, are
derived. These results are deduced from corresponding contraction rates for the
associated posterior distributions. The rates obtained are optimal up to
log-factors in -loss in any dimension, and also for supremum norm loss
when . Further, when , nonparametric Bernstein-von Mises
theorems are proved for the posterior distributions of . From this we deduce
functional central limit theorems for the implied estimators of the invariant
measure . The limiting Gaussian process distributions have a covariance
structure that is asymptotically optimal from an information-theoretic point of
view.Comment: 55 pages, to appear in the Annals of Statistic
Functional Regression
Functional data analysis (FDA) involves the analysis of data whose ideal
units of observation are functions defined on some continuous domain, and the
observed data consist of a sample of functions taken from some population,
sampled on a discrete grid. Ramsay and Silverman's 1997 textbook sparked the
development of this field, which has accelerated in the past 10 years to become
one of the fastest growing areas of statistics, fueled by the growing number of
applications yielding this type of data. One unique characteristic of FDA is
the need to combine information both across and within functions, which Ramsay
and Silverman called replication and regularization, respectively. This article
will focus on functional regression, the area of FDA that has received the most
attention in applications and methodological development. First will be an
introduction to basis functions, key building blocks for regularization in
functional regression methods, followed by an overview of functional regression
methods, split into three types: [1] functional predictor regression
(scalar-on-function), [2] functional response regression (function-on-scalar)
and [3] function-on-function regression. For each, the role of replication and
regularization will be discussed and the methodological development described
in a roughly chronological manner, at times deviating from the historical
timeline to group together similar methods. The primary focus is on modeling
and methodology, highlighting the modeling structures that have been developed
and the various regularization approaches employed. At the end is a brief
discussion describing potential areas of future development in this field
- …