1 research outputs found
Byzantine Resilient Distributed Multi-Task Learning
Distributed multi-task learning provides significant advantages in
multi-agent networks with heterogeneous data sources where agents aim to learn
distinct but correlated models simultaneously.However, distributed algorithms
for learning relatedness among tasks are not resilient in the presence of
Byzantine agents. In this paper, we present an approach for Byzantine resilient
distributed multi-task learning. We propose an efficient online weight
assignment rule by measuring the accumulated loss using an agent's data and its
neighbors' models. A small accumulated loss indicates a large similarity
between the two tasks. In order to ensure the Byzantine resilience of the
aggregation at a normal agent, we introduce a step for filtering out larger
losses. We analyze the approach for convex models and show that normal agents
converge resiliently towards the global minimum.Further, aggregation with the
proposed weight assignment rule always results in an improved expected regret
than the non-cooperative case. Finally, we demonstrate the approach using three
case studies, including regression and classification problems, and show that
our method exhibits good empirical performance for non-convex models, such as
convolutional neural networks