Some Simple Distributed Algorithms for Sparse Networks

We give simple, deterministic, distributed algorithms for computing maximal matchings, maximal independent sets and colourings. We show that edge colourings with at most 2D-1 colours, and maximal matchings can be computed within O(log^* n + D) deterministic rounds, where D is the maximum degree of the network. We also show how to find maximal independent sets and (D+1)-vertex colourings within O(log^* n + D^2) deterministic rounds. All hidden constants are very small and the algorithms are very simple

Information dissemination via random walks

Information dissemination is a fundamental task in distributed computing: How to deliver a piece of information from a node of a network to some or all other nodes? In the face of large and still growing modern networks, it is imperative that dissemination algorithms are decentralised and can operate under unreliable conditions. In the past decades, randomised rumour spreading algorithms have addressed these challenges. In these algorithms, a message is initially placed at a source node of a network, and, at regular intervals, each node contacts a randomly selected neighbour. A message may be transmitted in one or both directions during each of these communications, depending on the exact protocol. The main measure of performance for these algorithms is their broadcast time, which is the time until a message originating from a source node is disseminated to all nodes of the network. Apart from being extremely simple and robust to failures, randomised rumour spreading achieves theoretically optimal broadcast time in many common network topologies. In this thesis, we propose an agent-based information dissemination algorithm, called Visit-Exchange. In our protocol, a number of agents perform independent random walks in the network. An agent becomes informed when it visits a node that has a message, and later informs all future nodes it visits. Visit-Exchange shares many of the properties of randomised rumour spreading, namely, it is very simple and uses the same amount of communication in a unit of time. Moreover, the protocol can be used as a simple model of non-recoverable epidemic processes. We investigate the broadcast time of Visit-Exchange on a variety of network topologies, and compare it to traditional rumour spreading. On dense regular networks we show that the two types of protocols are equivalent, which means that in this setting the vast literature on randomised rumour spreading applies in our model as well. Since many networks of interest, including real-world ones, are very sparse, we also study agent-based broadcast for sparse networks. Our results include almost optimal or optimal bounds for sparse regular graphs, expanders, random regular graphs, balanced trees and grids. We establish that depending on the network topology, Visit-Exchange may be either slower or faster than traditional rumour spreading. In particular, in graphs consisting of hubs that are not well connected, broadcast using agents can be significantly faster. Our conclusion is that a combined broadcasting protocol that simultaneously uses both traditional rumour spreading and agent-based dissemination can be fast on a larger range of topologies than each of its components separately.Gates Cambridge Trust, St John's College Benefactors' Scholarshi

Sparse Allreduce: Efficient Scalable Communication for Power-Law Data

Many large datasets exhibit power-law statistics: The web graph, social networks, text data, click through data etc. Their adjacency graphs are termed natural graphs, and are known to be difficult to partition. As a consequence most distributed algorithms on these graphs are communication intensive. Many algorithms on natural graphs involve an Allreduce: a sum or average of partitioned data which is then shared back to the cluster nodes. Examples include PageRank, spectral partitioning, and many machine learning algorithms including regression, factor (topic) models, and clustering. In this paper we describe an efficient and scalable Allreduce primitive for power-law data. We point out scaling problems with existing butterfly and round-robin networks for Sparse Allreduce, and show that a hybrid approach improves on both. Furthermore, we show that Sparse Allreduce stages should be nested instead of cascaded (as in the dense case). And that the optimum throughput Allreduce network should be a butterfly of heterogeneous degree where degree decreases with depth into the network. Finally, a simple replication scheme is introduced to deal with node failures. We present experiments showing significant improvements over existing systems such as PowerGraph and Hadoop