92 research outputs found
Trace Norm Regularised Deep Multi-Task Learning
We propose a framework for training multiple neural networks simultaneously.
The parameters from all models are regularised by the tensor trace norm, so
that each neural network is encouraged to reuse others' parameters if possible
-- this is the main motivation behind multi-task learning. In contrast to many
deep multi-task learning models, we do not predefine a parameter sharing
strategy by specifying which layers have tied parameters. Instead, our
framework considers sharing for all shareable layers, and the sharing strategy
is learned in a data-driven way.Comment: Submission to Workshop track - ICLR 201
Clustered Multi-Task Learning: A Convex Formulation
In multi-task learning several related tasks are considered simultaneously,
with the hope that by an appropriate sharing of information across tasks, each
task may benefit from the others. In the context of learning linear functions
for supervised classification or regression, this can be achieved by including
a priori information about the weight vectors associated with the tasks, and
how they are expected to be related to each other. In this paper, we assume
that tasks are clustered into groups, which are unknown beforehand, and that
tasks within a group have similar weight vectors. We design a new spectral norm
that encodes this a priori assumption, without the prior knowledge of the
partition of tasks into groups, resulting in a new convex optimization
formulation for multi-task learning. We show in simulations on synthetic
examples and on the IEDB MHC-I binding dataset, that our approach outperforms
well-known convex methods for multi-task learning, as well as related non
convex methods dedicated to the same problem
Many Task Learning with Task Routing
Typical multi-task learning (MTL) methods rely on architectural adjustments
and a large trainable parameter set to jointly optimize over several tasks.
However, when the number of tasks increases so do the complexity of the
architectural adjustments and resource requirements. In this paper, we
introduce a method which applies a conditional feature-wise transformation over
the convolutional activations that enables a model to successfully perform a
large number of tasks. To distinguish from regular MTL, we introduce Many Task
Learning (MaTL) as a special case of MTL where more than 20 tasks are performed
by a single model. Our method dubbed Task Routing (TR) is encapsulated in a
layer we call the Task Routing Layer (TRL), which applied in an MaTL scenario
successfully fits hundreds of classification tasks in one model. We evaluate
our method on 5 datasets against strong baselines and state-of-the-art
approaches.Comment: 8 Pages, 5 Figures, 2 Table
- …