31,443 research outputs found
Adaptivity to Noise Parameters in Nonparametric Active Learning
This work addresses various open questions in the theory of active learning
for nonparametric classification. Our contributions are both statistical and
algorithmic: -We establish new minimax-rates for active learning under common
\textit{noise conditions}. These rates display interesting transitions -- due
to the interaction between noise \textit{smoothness and margin} -- not present
in the passive setting. Some such transitions were previously conjectured, but
remained unconfirmed. -We present a generic algorithmic strategy for adaptivity
to unknown noise smoothness and margin; our strategy achieves optimal rates in
many general situations; furthermore, unlike in previous work, we avoid the
need for \textit{adaptive confidence sets}, resulting in strictly milder
distributional requirements
Fast global convergence of gradient methods for high-dimensional statistical recovery
Many statistical -estimators are based on convex optimization problems
formed by the combination of a data-dependent loss function with a norm-based
regularizer. We analyze the convergence rates of projected gradient and
composite gradient methods for solving such problems, working within a
high-dimensional framework that allows the data dimension \pdim to grow with
(and possibly exceed) the sample size \numobs. This high-dimensional
structure precludes the usual global assumptions---namely, strong convexity and
smoothness conditions---that underlie much of classical optimization analysis.
We define appropriately restricted versions of these conditions, and show that
they are satisfied with high probability for various statistical models. Under
these conditions, our theory guarantees that projected gradient descent has a
globally geometric rate of convergence up to the \emph{statistical precision}
of the model, meaning the typical distance between the true unknown parameter
and an optimal solution . This result is substantially
sharper than previous convergence results, which yielded sublinear convergence,
or linear convergence only up to the noise level. Our analysis applies to a
wide range of -estimators and statistical models, including sparse linear
regression using Lasso (-regularized regression); group Lasso for block
sparsity; log-linear models with regularization; low-rank matrix recovery using
nuclear norm regularization; and matrix decomposition. Overall, our analysis
reveals interesting connections between statistical precision and computational
efficiency in high-dimensional estimation
Nonparametric Regression, Confidence Regions and Regularization
In this paper we offer a unified approach to the problem of nonparametric
regression on the unit interval. It is based on a universal, honest and
non-asymptotic confidence region which is defined by a set of linear
inequalities involving the values of the functions at the design points.
Interest will typically centre on certain simplest functions in that region
where simplicity can be defined in terms of shape (number of local extremes,
intervals of convexity/concavity) or smoothness (bounds on derivatives) or a
combination of both. Once some form of regularization has been decided upon the
confidence region can be used to provide honest non-asymptotic confidence
bounds which are less informative but conceptually much simpler
Low Complexity Regularization of Linear Inverse Problems
Inverse problems and regularization theory is a central theme in contemporary
signal processing, where the goal is to reconstruct an unknown signal from
partial indirect, and possibly noisy, measurements of it. A now standard method
for recovering the unknown signal is to solve a convex optimization problem
that enforces some prior knowledge about its structure. This has proved
efficient in many problems routinely encountered in imaging sciences,
statistics and machine learning. This chapter delivers a review of recent
advances in the field where the regularization prior promotes solutions
conforming to some notion of simplicity/low-complexity. These priors encompass
as popular examples sparsity and group sparsity (to capture the compressibility
of natural signals and images), total variation and analysis sparsity (to
promote piecewise regularity), and low-rank (as natural extension of sparsity
to matrix-valued data). Our aim is to provide a unified treatment of all these
regularizations under a single umbrella, namely the theory of partial
smoothness. This framework is very general and accommodates all low-complexity
regularizers just mentioned, as well as many others. Partial smoothness turns
out to be the canonical way to encode low-dimensional models that can be linear
spaces or more general smooth manifolds. This review is intended to serve as a
one stop shop toward the understanding of the theoretical properties of the
so-regularized solutions. It covers a large spectrum including: (i) recovery
guarantees and stability to noise, both in terms of -stability and
model (manifold) identification; (ii) sensitivity analysis to perturbations of
the parameters involved (in particular the observations), with applications to
unbiased risk estimation ; (iii) convergence properties of the forward-backward
proximal splitting scheme, that is particularly well suited to solve the
corresponding large-scale regularized optimization problem
- …