10,691 research outputs found
Bayesian nonparametric multivariate convex regression
In many applications, such as economics, operations research and
reinforcement learning, one often needs to estimate a multivariate regression
function f subject to a convexity constraint. For example, in sequential
decision processes the value of a state under optimal subsequent decisions may
be known to be convex or concave. We propose a new Bayesian nonparametric
multivariate approach based on characterizing the unknown regression function
as the max of a random collection of unknown hyperplanes. This specification
induces a prior with large support in a Kullback-Leibler sense on the space of
convex functions, while also leading to strong posterior consistency. Although
we assume that f is defined over R^p, we show that this model has a convergence
rate of log(n)^{-1} n^{-1/(d+2)} under the empirical L2 norm when f actually
maps a d dimensional linear subspace to R. We design an efficient reversible
jump MCMC algorithm for posterior computation and demonstrate the methods
through application to value function approximation
Variational Analysis of Constrained M-Estimators
We propose a unified framework for establishing existence of nonparametric
M-estimators, computing the corresponding estimates, and proving their strong
consistency when the class of functions is exceptionally rich. In particular,
the framework addresses situations where the class of functions is complex
involving information and assumptions about shape, pointwise bounds, location
of modes, height at modes, location of level-sets, values of moments, size of
subgradients, continuity, distance to a "prior" function, multivariate total
positivity, and any combination of the above. The class might be engineered to
perform well in a specific setting even in the presence of little data. The
framework views the class of functions as a subset of a particular metric space
of upper semicontinuous functions under the Attouch-Wets distance. In addition
to allowing a systematic treatment of numerous M-estimators, the framework
yields consistency of plug-in estimators of modes of densities, maximizers of
regression functions, level-sets of classifiers, and related quantities, and
also enables computation by means of approximating parametric classes. We
establish consistency through a one-sided law of large numbers, here extended
to sieves, that relaxes assumptions of uniform laws, while ensuring global
approximations even under model misspecification
Distributed Kernel Regression: An Algorithm for Training Collaboratively
This paper addresses the problem of distributed learning under communication
constraints, motivated by distributed signal processing in wireless sensor
networks and data mining with distributed databases. After formalizing a
general model for distributed learning, an algorithm for collaboratively
training regularized kernel least-squares regression estimators is derived.
Noting that the algorithm can be viewed as an application of successive
orthogonal projection algorithms, its convergence properties are investigated
and the statistical behavior of the estimator is discussed in a simplified
theoretical setting.Comment: To be presented at the 2006 IEEE Information Theory Workshop, Punta
del Este, Uruguay, March 13-17, 200
Additive isotone regression
This paper is about optimal estimation of the additive components of a
nonparametric, additive isotone regression model. It is shown that
asymptotically up to first order, each additive component can be estimated as
well as it could be by a least squares estimator if the other components were
known. The algorithm for the calculation of the estimator uses backfitting.
Convergence of the algorithm is shown. Finite sample properties are also
compared through simulation experiments.Comment: Published at http://dx.doi.org/10.1214/074921707000000355 in the IMS
Lecture Notes Monograph Series
(http://www.imstat.org/publications/lecnotes.htm) by the Institute of
Mathematical Statistics (http://www.imstat.org
Data-Driven Estimation in Equilibrium Using Inverse Optimization
Equilibrium modeling is common in a variety of fields such as game theory and
transportation science. The inputs for these models, however, are often
difficult to estimate, while their outputs, i.e., the equilibria they are meant
to describe, are often directly observable. By combining ideas from inverse
optimization with the theory of variational inequalities, we develop an
efficient, data-driven technique for estimating the parameters of these models
from observed equilibria. We use this technique to estimate the utility
functions of players in a game from their observed actions and to estimate the
congestion function on a road network from traffic count data. A distinguishing
feature of our approach is that it supports both parametric and
\emph{nonparametric} estimation by leveraging ideas from statistical learning
(kernel methods and regularization operators). In computational experiments
involving Nash and Wardrop equilibria in a nonparametric setting, we find that
a) we effectively estimate the unknown demand or congestion function,
respectively, and b) our proposed regularization technique substantially
improves the out-of-sample performance of our estimators.Comment: 36 pages, 5 figures Additional theorems for generalization guarantees
and statistical analysis adde
- …