21,173 research outputs found
Explicit Learning Curves for Transduction and Application to Clustering and Compression Algorithms
Inductive learning is based on inferring a general rule from a finite data
set and using it to label new data. In transduction one attempts to solve the
problem of using a labeled training set to label a set of unlabeled points,
which are given to the learner prior to learning. Although transduction seems
at the outset to be an easier task than induction, there have not been many
provably useful algorithms for transduction. Moreover, the precise relation
between induction and transduction has not yet been determined. The main
theoretical developments related to transduction were presented by Vapnik more
than twenty years ago. One of Vapnik's basic results is a rather tight error
bound for transductive classification based on an exact computation of the
hypergeometric tail. While tight, this bound is given implicitly via a
computational routine. Our first contribution is a somewhat looser but explicit
characterization of a slightly extended PAC-Bayesian version of Vapnik's
transductive bound. This characterization is obtained using concentration
inequalities for the tail of sums of random variables obtained by sampling
without replacement. We then derive error bounds for compression schemes such
as (transductive) support vector machines and for transduction algorithms based
on clustering. The main observation used for deriving these new error bounds
and algorithms is that the unlabeled test points, which in the transductive
setting are known in advance, can be used in order to construct useful data
dependent prior distributions over the hypothesis space
Stable limits for empirical processes on vapnik-cervonenk is classes of functions
Alexander' s (1987) central limit theorem for empirical processes on Vapnik-Cervonenkis classes of functions is extended to the case with non-Gaussian stable limits. The corresponding weak laws of large numbers are also established
A complexity analysis of statistical learning algorithms
We apply information-based complexity analysis to support vector machine
(SVM) algorithms, with the goal of a comprehensive continuous algorithmic
analysis of such algorithms. This involves complexity measures in which some
higher order operations (e.g., certain optimizations) are considered primitive
for the purposes of measuring complexity. We consider classes of information
operators and algorithms made up of scaled families, and investigate the
utility of scaling the complexities to minimize error. We look at the division
of statistical learning into information and algorithmic components, at the
complexities of each, and at applications to support vector machine (SVM) and
more general machine learning algorithms. We give applications to SVM
algorithms graded into linear and higher order components, and give an example
in biomedical informatics
A Statistical Learning Theory Approach for Uncertain Linear and Bilinear Matrix Inequalities
In this paper, we consider the problem of minimizing a linear functional
subject to uncertain linear and bilinear matrix inequalities, which depend in a
possibly nonlinear way on a vector of uncertain parameters. Motivated by recent
results in statistical learning theory, we show that probabilistic guaranteed
solutions can be obtained by means of randomized algorithms. In particular, we
show that the Vapnik-Chervonenkis dimension (VC-dimension) of the two problems
is finite, and we compute upper bounds on it. In turn, these bounds allow us to
derive explicitly the sample complexity of these problems. Using these bounds,
in the second part of the paper, we derive a sequential scheme, based on a
sequence of optimization and validation steps. The algorithm is on the same
lines of recent schemes proposed for similar problems, but improves both in
terms of complexity and generality. The effectiveness of this approach is shown
using a linear model of a robot manipulator subject to uncertain parameters.Comment: 19 pages, 2 figures, Accepted for Publication in Automatic
- …