1,687 research outputs found
Local Rademacher Complexity for Multi-Label Learning
© 1992-2012 IEEE. We analyze the local Rademacher complexity of empirical risk minimization-based multi-label learning algorithms, and in doing so propose a new algorithm for multi-label learning. Rather than using the trace norm to regularize the multi-label predictor, we instead minimize the tail sum of the singular values of the predictor in multi-label learning. Benefiting from the use of the local Rademacher complexity, our algorithm, therefore, has a sharper generalization error bound. Compared with methods that minimize over all singular values, concentrating on the tail singular values results in better recovery of the low-rank structure of the multi-label predictor, which plays an important role in exploiting label correlations. We propose a new conditional singular value thresholding algorithm to solve the resulting objective function. Moreover, a variance control strategy is employed to reduce the variance of variables in optimization. Empirical studies on real-world data sets validate our theoretical results and demonstrate the effectiveness of the proposed algorithm for multi-label learning
Semi-supervised Vector-valued Learning: From Theory to Algorithm
Vector-valued learning, where the output space admits a vector-valued
structure, is an important problem that covers a broad family of important
domains, e.g. multi-label learning and multi-class classification. Using local
Rademacher complexity and unlabeled data, we derive novel data-dependent excess
risk bounds for learning vector-valued functions in both the kernel space and
linear space. The derived bounds are much sharper than existing ones, where
convergence rates are improved from to
and in special cases. Motivated
by our theoretical analysis, we propose a unified framework for learning
vector-valued functions, incorporating both local Rademacher complexity and
Laplacian regularization. Empirical results on a wide number of benchmark
datasets show that the proposed algorithm significantly outperforms baseline
methods, which coincides with our theoretical findings
Optimistic Bounds for Multi-output Prediction
We investigate the challenge of multi-output learning, where the goal is to
learn a vector-valued function based on a supervised data set. This includes a
range of important problems in Machine Learning including multi-target
regression, multi-class classification and multi-label classification. We begin
our analysis by introducing the self-bounding Lipschitz condition for
multi-output loss functions, which interpolates continuously between a
classical Lipschitz condition and a multi-dimensional analogue of a smoothness
condition. We then show that the self-bounding Lipschitz condition gives rise
to optimistic bounds for multi-output learning, which are minimax optimal up to
logarithmic factors. The proof exploits local Rademacher complexity combined
with a powerful minoration inequality due to Srebro, Sridharan and Tewari. As
an application we derive a state-of-the-art generalization bound for
multi-class gradient boosting
- …