3 research outputs found
M-Power Regularized Least Squares Regression
Regularization is used to find a solution that both fits the data and is
sufficiently smooth, and thereby is very effective for designing and refining
learning algorithms. But the influence of its exponent remains poorly
understood. In particular, it is unclear how the exponent of the reproducing
kernel Hilbert space~(RKHS) regularization term affects the accuracy and the
efficiency of kernel-based learning algorithms. Here we consider regularized
least squares regression (RLSR) with an RKHS regularization raised to the power
of m, where m is a variable real exponent. We design an efficient algorithm for
solving the associated minimization problem, we provide a theoretical analysis
of its stability, and we compare its advantage with respect to computational
complexity, speed of convergence and prediction accuracy to the classical
kernel ridge regression algorithm where the regularization exponent m is fixed
at 2. Our results show that the m-power RLSR problem can be solved efficiently,
and support the suggestion that one can use a regularization term that grows
significantly slower than the standard quadratic growth in the RKHS norm
An Analysis of Active Learning With Uniform Feature Noise
In active learning, the user sequentially chooses values for feature and
an oracle returns the corresponding label . In this paper, we consider the
effect of feature noise in active learning, which could arise either because
itself is being measured, or it is corrupted in transmission to the oracle,
or the oracle returns the label of a noisy version of the query point. In
statistics, feature noise is known as "errors in variables" and has been
studied extensively in non-active settings. However, the effect of feature
noise in active learning has not been studied before. We consider the
well-known Berkson errors-in-variables model with additive uniform noise of
width .
Our simple but revealing setting is that of one-dimensional binary
classification setting where the goal is to learn a threshold (point where the
probability of a label crosses half). We deal with regression functions
that are antisymmetric in a region of size around the threshold and
also satisfy Tsybakov's margin condition around the threshold. We prove minimax
lower and upper bounds which demonstrate that when is smaller than the
minimiax active/passive noiseless error derived in \cite{CN07}, then noise has
no effect on the rates and one achieves the same noiseless rates. For larger
, the \textit{unflattening} of the regression function on convolution
with uniform noise, along with its local antisymmetry around the threshold,
together yield a behaviour where noise \textit{appears} to be beneficial. Our
key result is that active learning can buy significant improvement over a
passive strategy even in the presence of feature noise.Comment: 24 pages, 2 figures, published in the proceedings of the 17th
International Conference on Artificial Intelligence and Statistics (AISTATS),
201