986 research outputs found
Generalization Guarantees for a Binary Classification Framework for Two-Stage Multiple Kernel Learning
We present generalization bounds for the TS-MKL framework for two stage
multiple kernel learning. We also present bounds for sparse kernel learning
formulations within the TS-MKL framework
Supervised Learning with Similarity Functions
We address the problem of general supervised learning when data can only be
accessed through an (indefinite) similarity function between data points.
Existing work on learning with indefinite kernels has concentrated solely on
binary/multi-class classification problems. We propose a model that is generic
enough to handle any supervised learning task and also subsumes the model
previously proposed for classification. We give a "goodness" criterion for
similarity functions w.r.t. a given supervised learning task and then adapt a
well-known landmarking technique to provide efficient algorithms for supervised
learning using "good" similarity functions. We demonstrate the effectiveness of
our model on three important super-vised learning problems: a) real-valued
regression, b) ordinal regression and c) ranking where we show that our method
guarantees bounded generalization error. Furthermore, for the case of
real-valued regression, we give a natural goodness definition that, when used
in conjunction with a recent result in sparse vector recovery, guarantees a
sparse predictor with bounded generalization error. Finally, we report results
of our learning algorithms on regression and ordinal regression tasks using
non-PSD similarity functions and demonstrate the effectiveness of our
algorithms, especially that of the sparse landmark selection algorithm that
achieves significantly higher accuracies than the baseline methods while
offering reduced computational costs.Comment: To appear in the proceedings of NIPS 2012, 30 page
On Estimating the First Frequency Moment of Data Streams
Estimating the first moment of a data stream defined as F_1 = \sum_{i \in
\{1, 2, \ldots, n\}} \abs{f_i} to within -relative error with
high probability is a basic and influential problem in data stream processing.
A tight space bound of is known from the work of
[Kane-Nelson-Woodruff-SODA10]. However, all known algorithms for this problem
require per-update stream processing time of , with the
only exception being the algorithm of [Ganguly-Cormode-RANDOM07] that requires
per-update processing time of albeit with sub-optimal
space . In this paper, we present an algorithm for
estimating that achieves near-optimality in both space and update
processing time. The space requirement is and the per-update processing time is .Comment: 12 page
Random Feature Maps for Dot Product Kernels
Approximating non-linear kernels using feature maps has gained a lot of
interest in recent years due to applications in reducing training and testing
times of SVM classifiers and other kernel based learning algorithms. We extend
this line of work and present low distortion embeddings for dot product kernels
into linear Euclidean spaces. We base our results on a classical result in
harmonic analysis characterizing all dot product kernels and use it to define
randomized feature maps into explicit low dimensional Euclidean spaces in which
the native dot product provides an approximation to the dot product kernel with
high confidence.Comment: To appear in the proceedings of the 15th International Conference on
Artificial Intelligence and Statistics (AISTATS 2012). This version corrects
a minor error with Lemma 10. Acknowledgements : Devanshu Bhimwa
On Translation Invariant Kernels and Screw Functions
We explore the connection between Hilbertian metrics and positive definite
kernels on the real line. In particular, we look at a well-known
characterization of translation invariant Hilbertian metrics on the real line
by von Neumann and Schoenberg (1941). Using this result we are able to give an
alternate proof of Bochner's theorem for translation invariant positive
definite kernels on the real line (Rudin, 1962)
- …