2,877 research outputs found
Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning
Federated learning is a distributed framework for training machine learning
models over the data residing at mobile devices, while protecting the privacy
of individual users. A major bottleneck in scaling federated learning to a
large number of users is the overhead of secure model aggregation across many
users. In particular, the overhead of the state-of-the-art protocols for secure
model aggregation grows quadratically with the number of users. In this paper,
we propose the first secure aggregation framework, named Turbo-Aggregate, that
in a network with users achieves a secure aggregation overhead of
, as opposed to , while tolerating up to a user dropout
rate of . Turbo-Aggregate employs a multi-group circular strategy for
efficient model aggregation, and leverages additive secret sharing and novel
coding techniques for injecting aggregation redundancy in order to handle user
dropouts while guaranteeing user privacy. We experimentally demonstrate that
Turbo-Aggregate achieves a total running time that grows almost linear in the
number of users, and provides up to speedup over the
state-of-the-art protocols with up to users. Our experiments also
demonstrate the impact of model size and bandwidth on the performance of
Turbo-Aggregate
The Role of Interactivity in Local Differential Privacy
We study the power of interactivity in local differential privacy. First, we
focus on the difference between fully interactive and sequentially interactive
protocols. Sequentially interactive protocols may query users adaptively in
sequence, but they cannot return to previously queried users. The vast majority
of existing lower bounds for local differential privacy apply only to
sequentially interactive protocols, and before this paper it was not known
whether fully interactive protocols were more powerful. We resolve this
question. First, we classify locally private protocols by their
compositionality, the multiplicative factor by which the sum of a
protocol's single-round privacy parameters exceeds its overall privacy
guarantee. We then show how to efficiently transform any fully interactive
-compositional protocol into an equivalent sequentially interactive protocol
with an blowup in sample complexity. Next, we show that our reduction is
tight by exhibiting a family of problems such that for any , there is a
fully interactive -compositional protocol which solves the problem, while no
sequentially interactive protocol can solve the problem without at least an
factor more examples. We then turn our attention to
hypothesis testing problems. We show that for a large class of compound
hypothesis testing problems --- which include all simple hypothesis testing
problems as a special case --- a simple noninteractive test is optimal among
the class of all (possibly fully interactive) tests
- …