17 research outputs found
2P-BFT-Log: 2-Phase Single-Author Append-Only Log for Adversarial Environments
Replicated append-only logs sequentially order messages from the same author
such that their ordering can be eventually recovered even with out-of-order and
unreliable dissemination of individual messages. They are widely used for
implementing replicated services in both clouds and peer-to-peer environments
because they provide simple and efficient incremental reconciliation. However,
existing designs of replicated append-only logs assume replicas faithfully
maintain the sequential properties of logs and do not provide eventual
consistency when malicious participants fork their logs by disseminating
different messages to different replicas for the same index, which may result
in partitioning of replicas according to which branch was first replicated.
In this paper, we present 2P-BFT-Log, a two-phase replicated append-only log
that provides eventual consistency in the presence of forks from malicious
participants such that all correct replicas will eventually agree either on the
most recent message of a valid log (first phase) or on the earliest point at
which a fork occurred as well as on an irrefutable proof that it happened
(second phase). We provide definitions, algorithms, and proofs of the key
properties of the design, and explain one way to implement the design onto Git,
an eventually consistent replicated database originally designed for
distributed version control.
Our design enables correct replicas to faithfully implement the
happens-before relationship first introduced by Lamport that underpins most
existing distributed algorithms, with eventual detection of forks from
malicious participants to exclude the latter from further progress. This opens
the door to adaptations of existing distributed algorithms to a cheaper detect
and repair paradigm, rather than the more common and expensive systematic
prevention of incorrect behaviour.Comment: Fixed 'two-phase' typ
Genet: A Quickly Scalable Fat-Tree Overlay for Personal Volunteer Computing using WebRTC
WebRTC enables browsers to exchange data directly but the number of possible
concurrent connections to a single source is limited. We overcome the
limitation by organizing participants in a fat-tree overlay: when the maximum
number of connections of a tree node is reached, the new participants connect
to the node's children. Our design quickly scales when a large number of
participants join in a short amount of time, by relying on a novel scheme that
only requires local information to route connection messages: the destination
is derived from the hash value of the combined identifiers of the message's
source and of the node that is holding the message. The scheme provides
deterministic routing of a sequence of connection messages from a single source
and probabilistic balancing of newer connections among the leaves. We show that
this design puts at least 83% of nodes at the same depth as a deterministic
algorithm, can connect a thousand browser windows in 21-55 seconds in a local
network, and can be deployed for volunteer computing to tap into 320 cores in
less than 30 seconds on a local network to increase the total throughput on the
Collatz application by two orders of magnitude compared to a single core
A layered JavaScript virtual machine supporting dynamic instrumentation
L’observation de l’exécution d’applications JavaScript est habituellement réalisée en instrumentant une machine virtuelle (MV) industrielle ou en effectuant une traduction source-à -source ad hoc et complexe. Ce mémoire présente une alternative basée sur la superposition de machines virtuelles. Notre approche consiste à faire une traduction source-à -source d’un programme pendant son exécution pour exposer ses opérations de bas niveau au travers d’un modèle objet flexible. Ces opérations de bas niveau peuvent ensuite être redéfinies pendant l’exécution pour pouvoir en faire l’observation. Pour limiter la pénalité en performance introduite, notre approche exploite les opérations rapides originales de la MV sous-jacente, lorsque cela est possible, et applique les techniques de compilation à -la-volée dans la MV superposée. Notre implémentation, Photon, est en moyenne 19% plus rapide qu’un interprète moderne, et entre 19× et 56× plus lente en moyenne que les compilateurs à -la-volée utilisés dans les navigateurs web populaires. Ce mémoire montre donc que la superposition de machines virtuelles est une technique alternative compétitive à la modification d’un interprète moderne pour JavaScript lorsqu’appliqué à l’observation à l’exécution des opérations sur les objets et des appels de fonction.Run-time monitoring of JavaScript applications is typically achieved by instrumenting a production virtual machine or through ad-hoc, complex source-to-source transformations. This dissertation presents an alternative based on virtual machine layering. Our approach performs a dynamic translation of the client program to expose low-level operations through a flexible object model. These low-level operations can then be redefined at run time to monitor the execution. In order to limit the incurred performance overhead, our approach leverages fast operations from the underlying host VM implementation whenever possible, and applies Just-In-Time compilation (JIT) techniques within the added virtual machine layer. Our implementation, Photon, is on average 19% faster than a state-of-the-art interpreter, and between 19× and 56× slower on average than the commercial JIT compilers found in popular web browsers. This dissertation therefore shows that virtual machine layering is a competitive alternative approach to the modification of a production JavaScript interpreter when applied to run-time monitoring of object operations and function calls
D-Cliques: Compensating for Data Heterogeneity with Topology in Decentralized Federated Learning
International audienceThe convergence speed of machine learning models trained with Federated Learning is significantly affected by heterogeneous data partitions, even more so in a fully decentralized setting without a central server. In this paper, we show that the impact of label distribution skew, an important type of data heterogeneity, can be significantly reduced by carefully designing the underlying communication topology. We present D-Cliques, a novel topology that reduces gradient bias by grouping nodes in sparsely interconnected cliques such that the label distribution in a clique is representative of the global label distribution. We also show how to adapt the updates of decentralized SGD to obtain unbiased gradients and implement an effective momentum with D-Cliques. Our extensive empirical evaluation on MNIST and CIFAR10 demonstrates that our approach provides similar convergence speed as a fully-connected topology, which provides the best convergence in a data heterogeneous setting, with a significant reduction in the number of edges and messages. In a 1000-node topology, D-Cliques require 98% less edges and 96% less total messages, with further possible gains using a small-world topology across cliques
Pando: Personal Volunteer Computing in Browsers
The large penetration and continued growth in ownership of personal
electronic devices represents a freely available and largely untapped source of
computing power. To leverage those, we present Pando, a new volunteer computing
tool based on a declarative concurrent programming model and implemented using
JavaScript, WebRTC, and WebSockets. This tool enables a dynamically varying
number of failure-prone personal devices contributed by volunteers to
parallelize the application of a function on a stream of values, by using the
devices' browsers. We show that Pando can provide throughput improvements
compared to a single personal device, on a variety of compute-bound
applications including animation rendering and image processing. We also show
the flexibility of our approach by deploying Pando on personal devices
connected over a local network, on Grid5000, a French-wide computing grid in a
virtual private network, and seven PlanetLab nodes distributed in a wide area
network over Europe.Comment: 14 pages, 12 figures, 2 table
Refined Convergence and Topology Learning for Decentralized SGD with Heterogeneous Data
One of the key challenges in decentralized and federated learning is to design algorithms that efficiently deal with highly heterogeneous data distributions across agents. In this paper, we revisit the analysis of the popular Decentralized Stochastic Gradient Descent algorithm (D-SGD) under data heterogeneity. We exhibit the key role played by a new quantity, called neighborhood heterogeneity, on the convergence rate of D-SGD. By coupling the communication topology and the heterogeneity, our analysis sheds light on the poorly understood interplay between these two concepts. We then argue that neighborhood heterogeneity provides a natural criterion to learn data-dependent topologies that reduce (and can even eliminate) the otherwise detrimental effect of data heterogeneity on the convergence time of D-SGD. For the important case of classification with label skew, we formulate the problem of learning such a good topology as a tractable optimization problem that we solve with a Frank-Wolfe algorithm. As illustrated over a set of simulated and real-world experiments, our approach provides a principled way to design a sparse topology that balances the convergence speed and the per-iteration communication costs of D-SGD under data heterogeneity
D-Cliques: Compensating for Data Heterogeneity with Topology in Decentralized Federated Learning
The convergence speed of machine learning models trained with Federated
Learning is significantly affected by heterogeneous data partitions, even more
so in a fully decentralized setting without a central server. In this paper, we
show that the impact of label distribution skew, an important type of data
heterogeneity, can be significantly reduced by carefully designing the
underlying communication topology. We present D-Cliques, a novel topology that
reduces gradient bias by grouping nodes in sparsely interconnected cliques such
that the label distribution in a clique is representative of the global label
distribution. We also show how to adapt the updates of decentralized SGD to
obtain unbiased gradients and implement an effective momentum with D-Cliques.
Our extensive empirical evaluation on MNIST and CIFAR10 demonstrates that our
approach provides similar convergence speed as a fully-connected topology,
which provides the best convergence in a data heterogeneous setting, with a
significant reduction in the number of edges and messages. In a 1000-node
topology, D-Cliques require 98% less edges and 96% less total messages, with
further possible gains using a small-world topology across cliques.Comment: 18 pages, 26 figures. Revision v4: Made title and abstract more
specific (data heterogeneity), added generalized clique construction
algorithm (Greedy Swap), updated main text experiments to use generalized
partitioning scheme, reorganized paper structure, re-ran and added previous
experiments in appendix using updated simulato
D-Cliques: Compensating for Data Heterogeneity with Topology in Decentralized Federated Learning
The convergence speed of machine learning models trained with Federated Learning is significantly affected by heterogeneous data partitions, even more so in a fully decentralized setting without a central server. In this paper, we show that the impact of label distribution skew, an important type of data heterogeneity, can be significantly reduced by carefully designing the underlying communication topology. We present D-Cliques, a novel topology that reduces gradient bias by grouping nodes in sparsely interconnected cliques such that the label distribution in a clique is representative of the global label distribution. We also show how to adapt the updates of decentralized SGD to obtain unbiased gradients and implement an effective momentum with D-Cliques. Our extensive empirical evaluation on MNIST and CIFAR10 demonstrates that our approach provides similar convergence speed as a fully-connected topology, which provides the best convergence in a data heterogeneous setting, with a significant reduction in the number of edges and messages. In a 1000-node topology, D-Cliques require 98% less edges and 96% less total messages, with further possible gains using a small-world topology across cliques