5 research outputs found
Sparse Semi-supervised Learning Using Conjugate Functions
In this paper, we propose a general framework for sparse semi-supervised learning, which concerns
using a small portion of unlabeled data and a few labeled data to represent target functions and thus
has the merit of accelerating function evaluations when predicting the output of a new example.
This framework makes use of Fenchel-Legendre conjugates to rewrite a convex insensitive loss
involving a regularization with unlabeled data, and is applicable to a family of semi-supervised
learning methods such as multi-view co-regularized least squares and single-view Laplacian support
vector machines (SVMs). As an instantiation of this framework, we propose sparse multi-view
SVMs which use a squared ε-insensitive loss. The resultant optimization is an inf-sup problem and
the optimal solutions have arguably saddle-point properties. We present a globally optimal iterative
algorithm to optimize the problem. We give the margin bound on the generalization error of the
sparse multi-view SVMs, and derive the empirical Rademacher complexity for the induced function
class. Experiments on artificial and real-world data show their effectiveness. We further give a
sequential training approach to show their possibility and potential for uses in large-scale problems
and provide encouraging experimental results indicating the efficacy of the margin bound and empirical
Rademacher complexity on characterizing the roles of unlabeled data for semi-supervised
learnin
PAC-Bayes Analysis of Multi-view Learning
This paper presents eight PAC-Bayes bounds to analyze the generalization
performance of multi-view classifiers. These bounds adopt data dependent
Gaussian priors which emphasize classifiers with high view agreements. The
center of the prior for the first two bounds is the origin, while the center of
the prior for the third and fourth bounds is given by a data dependent vector.
An important technique to obtain these bounds is two derived logarithmic
determinant inequalities whose difference lies in whether the dimensionality of
data is involved. The centers of the fifth and sixth bounds are calculated on a
separate subset of the training set. The last two bounds use unlabeled data to
represent view agreements and are thus applicable to semi-supervised multi-view
learning. We evaluate all the presented multi-view PAC-Bayes bounds on
benchmark data and compare them with previous single-view PAC-Bayes bounds. The
usefulness and performance of the multi-view bounds are discussed.Comment: 35 page
Top Rank Optimization in Linear Time
Bipartite ranking aims to learn a real-valued ranking function that orders
positive instances before negative instances. Recent efforts of bipartite
ranking are focused on optimizing ranking accuracy at the top of the ranked
list. Most existing approaches are either to optimize task specific metrics or
to extend the ranking loss by emphasizing more on the error associated with the
top ranked instances, leading to a high computational cost that is super-linear
in the number of training instances. We propose a highly efficient approach,
titled TopPush, for optimizing accuracy at the top that has computational
complexity linear in the number of training instances. We present a novel
analysis that bounds the generalization error for the top ranked instances for
the proposed approach. Empirical study shows that the proposed approach is
highly competitive to the state-of-the-art approaches and is 10-100 times
faster