3 research outputs found
Local Rademacher Complexity-based Learning Guarantees for Multi-Task Learning
We show a Talagrand-type concentration inequality for Multi-Task Learning
(MTL), using which we establish sharp excess risk bounds for MTL in terms of
distribution- and data-dependent versions of the Local Rademacher Complexity
(LRC). We also give a new bound on the LRC for norm regularized as well as
strongly convex hypothesis classes, which applies not only to MTL but also to
the standard i.i.d. setting. Combining both results, one can now easily derive
fast-rate bounds on the excess risk for many prominent MTL methods,
including---as we demonstrate---Schatten-norm, group-norm, and
graph-regularized MTL. The derived bounds reflect a relationship akeen to a
conservation law of asymptotic convergence rates. This very relationship allows
for trading off slower rates w.r.t. the number of tasks for faster rates with
respect to the number of available samples per task, when compared to the rates
obtained via a traditional, global Rademacher analysis.Comment: In this version, some arguments and results (of the previous version)
have been corrected, or modifie
Multi-Task Learning with Group-Specific Feature Space Sharing
When faced with learning a set of inter-related tasks from a limited amount
of usable data, learning each task independently may lead to poor
generalization performance. Multi-Task Learning (MTL) exploits the latent
relations between tasks and overcomes data scarcity limitations by co-learning
all these tasks simultaneously to offer improved performance. We propose a
novel Multi-Task Multiple Kernel Learning framework based on Support Vector
Machines for binary classification tasks. By considering pair-wise task
affinity in terms of similarity between a pair's respective feature spaces, the
new framework, compared to other similar MTL approaches, offers a high degree
of flexibility in determining how similar feature spaces should be, as well as
which pairs of tasks should share a common feature space in order to benefit
overall performance. The associated optimization problem is solved via a block
coordinate descent, which employs a consensus-form Alternating Direction Method
of Multipliers algorithm to optimize the Multiple Kernel Learning weights and,
hence, to determine task affinities. Empirical evaluation on seven data sets
exhibits a statistically significant improvement of our framework's results
compared to the ones of several other Clustered Multi-Task Learning methods