Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning

Federated learning is a distributed framework for training machine learning models over the data residing at mobile devices, while protecting the privacy of individual users. A major bottleneck in scaling federated learning to a large number of users is the overhead of secure model aggregation across many users. In particular, the overhead of the state-of-the-art protocols for secure model aggregation grows quadratically with the number of users. In this paper, we propose the first secure aggregation framework, named Turbo-Aggregate, that in a network with $N$ users achieves a secure aggregation overhead of $O(N\log{N})$, as opposed to $O(N^2)$, while tolerating up to a user dropout rate of $50\%$. Turbo-Aggregate employs a multi-group circular strategy for efficient model aggregation, and leverages additive secret sharing and novel coding techniques for injecting aggregation redundancy in order to handle user dropouts while guaranteeing user privacy. We experimentally demonstrate that Turbo-Aggregate achieves a total running time that grows almost linear in the number of users, and provides up to $40\times$ speedup over the state-of-the-art protocols with up to $N=200$ users. Our experiments also demonstrate the impact of model size and bandwidth on the performance of Turbo-Aggregate

Market partitioning and the geometry of the resource space

Operations such as integration or modularization of databases can be considered as operations on database universes. This paper describes some operations on database universes. Formally, a database universe is a special kind of table. It turns out that various operations on tables constitute interesting operations on database universes as well.

Fast Local Computation Algorithms

For input $x$, let $F(x)$ denote the set of outputs that are the "legal" answers for a computational problem $F$. Suppose $x$ and members of $F(x)$ are so large that there is not time to read them in their entirety. We propose a model of {\em local computation algorithms} which for a given input $x$, support queries by a user to values of specified locations $y_i$ in a legal output $y \in F(x)$. When more than one legal output $y$ exists for a given $x$, the local computation algorithm should output in a way that is consistent with at least one such $y$. Local computation algorithms are intended to distill the common features of several concepts that have appeared in various algorithmic subfields, including local distributed computation, local algorithms, locally decodable codes, and local reconstruction. We develop a technique, based on known constructions of small sample spaces of $k$-wise independent random variables and Beck's analysis in his algorithmic approach to the Lov{\'{a}}sz Local Lemma, which under certain conditions can be applied to construct local computation algorithms that run in {\em polylogarithmic} time and space. We apply this technique to maximal independent set computations, scheduling radio network broadcasts, hypergraph coloring and satisfying $k$-SAT formulas.Comment: A preliminary version of this paper appeared in ICS 2011, pp. 223-23

Method of up-front load balancing for local memory parallel processors

In a parallel processing computer system with multiple processing units and shared memory, a method is disclosed for uniformly balancing the aggregate computational load in, and utilizing minimal memory by, a network having identical computations to be executed at each connection therein. Read-only and read-write memory are subdivided into a plurality of process sets, which function like artificial processing units. Said plurality of process sets is iteratively merged and reduced to the number of processing units without exceeding the balance load. Said merger is based upon the value of a partition threshold, which is a measure of the memory utilization. The turnaround time and memory savings of the instant method are functions of the number of processing units available and the number of partitions into which the memory is subdivided. Typical results of the preferred embodiment yielded memory savings of from sixty to seventy five percent

Sonet Network Design Problems

This paper presents a new method and a constraint-based objective function to solve two problems related to the design of optical telecommunication networks, namely the Synchronous Optical Network Ring Assignment Problem (SRAP) and the Intra-ring Synchronous Optical Network Design Problem (IDP). These network topology problems can be represented as a graph partitioning with capacity constraints as shown in previous works. We present here a new objective function and a new local search algorithm to solve these problems. Experiments conducted in Comet allow us to compare our method to previous ones and show that we obtain better results

Reserve Size And Fragmentation Alter Community Assembly, Diversity, And Dynamics

Researchers have disputed whether a single large habitat reserve will support more species than many small reserves. However, relatively little is known from a theoretical perspective about how reserve size affects competitive communities structured by spatial abiotic gradients. We investigate how reserve size affects theoretical communities whose assembly is governed by dispersal limitation, abiotic niche differentiation, and source-sink dynamics. Simulations were conducted with varying scales of dispersal across landscapes with variable environmental spatial autocorrelation. Landscapes were inhabited by simulated trees with seedling and adult stages. For a fixed total area in reserves, we found that small reserve systems increased the distance between environments dominated by different species, diminishing the effects of source-sink dynamics. As reserve size decreased, environmental limitations to community assembly became stronger, species richness decreased, and richness increased. When dispersal occurred across short distances, a large reserve strategy caused greater stochastic community variation, greater richness, and lower richness than in small reserve systems. We found that reserve size variation trades off between preserving different aspects of natural communities, including diversity versus diversity. Optimal reserve size will depend on the importance of source-sink dynamics and the value placed on different characteristics of natural communities. Anthropogenic changes to the size and separation of remnant habitats can have far-reaching effects on community structure and assembly.Integrative Biolog

The impact of a network split on cascading failure processes

Cascading failure models are typically used to capture the phenomenon where failures possibly trigger further failures in succession, causing knock-on effects. In many networks this ultimately leads to a disintegrated network where the failure propagation continues independently across the various components. In order to gain insight in the impact of network splitting on cascading failure processes, we extend a well-established cascading failure model for which the number of failures obeys a power-law distribution. We assume that a single line failure immediately splits the network in two components, and examine its effect on the power-law exponent. The results provide valuable qualitative insights that are crucial first steps towards understanding more complex network splitting scenarios