2 research outputs found
Normalized Hierarchical SVM
We present improved methods of using structured SVMs in a large-scale
hierarchical classification problem, that is when labels are leaves, or sets of
leaves, in a tree or a DAG. We examine the need to normalize both the
regularization and the margin and show how doing so significantly improves
performance, including allowing achieving state-of-the-art results where
unnormalized structured SVMs do not perform better than flat models. We also
describe a further extension of hierarchical SVMs that highlight the connection
between hierarchical SVMs and matrix factorization models
Efficient Structured Surrogate Loss and Regularization in Structured Prediction
In this dissertation, we focus on several important problems in structured
prediction. In structured prediction, the label has a rich intrinsic
substructure, and the loss varies with respect to the predicted label and the
true label pair. Structured SVM is an extension of binary SVM to adapt to such
structured tasks.
In the first part of the dissertation, we study the surrogate losses and its
efficient methods. To minimize the empirical risk, a surrogate loss which upper
bounds the loss, is used as a proxy to minimize the actual loss. Since the
objective function is written in terms of the surrogate loss, the choice of the
surrogate loss is important, and the performance depends on it. Another issue
regarding the surrogate loss is the efficiency of the argmax label inference
for the surrogate loss. Efficient inference is necessary for the optimization
since it is often the most time-consuming step. We present a new class of
surrogate losses named bi-criteria surrogate loss, which is a generalization of
the popular surrogate losses. We first investigate an efficient method for a
slack rescaling formulation as a starting point utilizing decomposability of
the model. Then, we extend the algorithm to the bi-criteria surrogate loss,
which is very efficient and also shows performance improvements.
In the second part of the dissertation, another important issue of
regularization is studied. Specifically, we investigate a problem of
regularization in hierarchical classification when a structural imbalance
exists in the label structure. We present a method to normalize the structure,
as well as a new norm, namely shared Frobenius norm. It is suitable for
hierarchical classification that adapts to the data in addition to the label
structure.Comment: PhD Thesi