12 research outputs found
Distributed Multi-Task Relationship Learning
Multi-task learning aims to learn multiple tasks jointly by exploiting their
relatedness to improve the generalization performance for each task.
Traditionally, to perform multi-task learning, one needs to centralize data
from all the tasks to a single machine. However, in many real-world
applications, data of different tasks may be geo-distributed over different
local machines. Due to heavy communication caused by transmitting the data and
the issue of data privacy and security, it is impossible to send data of
different task to a master machine to perform multi-task learning. Therefore,
in this paper, we propose a distributed multi-task learning framework that
simultaneously learns predictive models for each task as well as task
relationships between tasks alternatingly in the parameter server paradigm. In
our framework, we first offer a general dual form for a family of regularized
multi-task relationship learning methods. Subsequently, we propose a
communication-efficient primal-dual distributed optimization algorithm to solve
the dual problem by carefully designing local subproblems to make the dual
problem decomposable. Moreover, we provide a theoretical convergence analysis
for the proposed algorithm, which is specific for distributed multi-task
relationship learning. We conduct extensive experiments on both synthetic and
real-world datasets to evaluate our proposed framework in terms of
effectiveness and convergence.Comment: To appear in KDD 201
Distributed Primal-Dual Optimization for Online Multi-Task Learning
Conventional online multi-task learning algorithms suffer from two critical
limitations: 1) Heavy communication caused by delivering high velocity of
sequential data to a central machine; 2) Expensive runtime complexity for
building task relatedness. To address these issues, in this paper we consider a
setting where multiple tasks are geographically located in different places,
where one task can synchronize data with others to leverage knowledge of
related tasks. Specifically, we propose an adaptive primal-dual algorithm,
which not only captures task-specific noise in adversarial learning but also
carries out a projection-free update with runtime efficiency. Moreover, our
model is well-suited to decentralized periodic-connected tasks as it allows the
energy-starved or bandwidth-constraint tasks to postpone the update.
Theoretical results demonstrate the convergence guarantee of our distributed
algorithm with an optimal regret. Empirical results confirm that the proposed
model is highly effective on various real-world datasets