6 research outputs found
Towards Byzantine-resilient Learning in Decentralized Systems
With the proliferation of IoT and edge computing, decentralized learning is
becoming more promising. When designing a distributed learning system, one
major challenge to consider is Byzantine Fault Tolerance (BFT). Past works have
researched Byzantine-resilient solutions for centralized distributed learning.
However, there are currently no satisfactory solutions with strong efficiency
and security in decentralized systems. In this paper, we propose a novel
algorithm, Mozi, to achieve BFT in decentralized learning systems.
Specifically, Mozi provides a uniform Byzantine-resilient aggregation rule for
benign nodes to select the useful parameter updates and filter out the
malicious ones in each training iteration. It guarantees that each benign node
in a decentralized system can train a correct model under very strong Byzantine
attacks with an arbitrary number of faulty nodes. We perform the theoretical
analysis to prove the uniform convergence of our proposed algorithm.
Experimental evaluations demonstrate the high security and efficiency of Mozi
compared to all existing solutions.Comment: We would like to extensively revise the paper and submit it to a
conferenc