A powerful heuristic for telephone gossiping

A refined heuristic for computing schedules for gossiping in the telephone model is presented. The heuristic is fast: for a network with n nodes and m edges, requiring R rounds for gossiping, the running time is O(R n log(n) m) for all tested classes of graphs. This moderate time consumption allows to compute gossiping schedules for networks with more than 10,000 PUs and 100,000 connections. The heuristic is good: in practice the computed schedules never exceed the optimum by more than a few rounds. The heuristic is versatile: it can also be used for broadcasting and more general information dispersion patterns. It can handle both the unit-cost and the linear-cost model. Actually, the heuristic is so good, that for CCC, shuffle-exchange, butterfly de Bruijn, star and pancake networks the constructed gossiping schedules are better than the best theoretically derived ones. For example, for gossiping on a shuffle-exchange network with 2^{13} PUs, the former upper bound was 49 rounds, while our heuristic finds a schedule requiring 31 rounds. Also for broadcasting the heuristic improves on many formerly known results. A second heuristic, works even better for CCC, butterfly, star and pancake networks. For example, with this heuristic we found that gossiping on a pancake network with 7! PUs can be performed in 15 rounds, 2 fewer than achieved by the best theoretical construction. This second heuristic is less versatile than the first, but by refined search techniques it can tackle even larger problems, the main limitation being the storage capacity. Another advantage is that the constructed schedules can be represented concisely

On distributed scheduling in wireless networks exploiting broadcast and network coding

In this paper, we consider cross-layer optimization in wireless networks with wireless broadcast advantage, focusing on the problem of distributed scheduling of broadcast links. The wireless broadcast advantage is most useful in multicast scenarios. As such, we include network coding in our design to exploit the throughput gain brought in by network coding for multicasting. We derive a subgradient algorithm for joint rate control, network coding and scheduling, which however requires centralized link scheduling. Under the primary interference model, link scheduling problem is equivalent to a maximum weighted hypergraph matching problem that is NP-complete. To solve the scheduling problem distributedly, locally greedy and randomized approximation algorithms are proposed and shown to have bounded worst-case performance. With random network coding, we obtain a fully distributed cross-layer design. Numerical results show promising throughput gain using the proposed algorithms, and surprisingly, in some cases even with less complexity than cross-layer design without broadcast advantage

Lifeguard: Local Health Awareness for More Accurate Failure Detection

SWIM is a peer-to-peer group membership protocol with attractive scaling and robustness properties. However, slow message processing can cause SWIM to mark healthy members as failed (so called false positive failure detection), despite inclusion of a mechanism to avoid this. We identify the properties of SWIM that lead to the problem, and propose Lifeguard, a set of extensions to SWIM which consider that the local failure detector module may be at fault, via the concept of local health. We evaluate this approach in a precisely controlled environment and validate it in a real-world scenario, showing that it drastically reduces the rate of false positives. The false positive rate and detection time for true failures can be reduced simultaneously, compared to the baseline levels of SWIM

Breaking (Global) Barriers in Parallel Stochastic Optimization with Wait-Avoiding Group Averaging

Deep learning at scale is dominated by communication time. Distributing samples across nodes usually yields the best performance, but poses scaling challenges due to global information dissemination and load imbalance across uneven sample lengths. State-of-the-art decentralized optimizers mitigate the problem, but require more iterations to achieve the same accuracy as their globally-communicating counterparts. We present Wait-Avoiding Group Model Averaging (WAGMA) SGD, a wait-avoiding stochastic optimizer that reduces global communication via subgroup weight exchange. The key insight is a combination of algorithmic changes to the averaging scheme and the use of a group allreduce operation. We prove the convergence of WAGMA-SGD, and empirically show that it retains convergence rates similar to Allreduce-SGD. For evaluation, we train ResNet-50 on ImageNet; Transformer for machine translation; and deep reinforcement learning for navigation at scale. Compared with state-of-the-art decentralized SGD variants, WAGMA-SGD significantly improves training throughput (e.g., 2.1x on 1,024 GPUs for reinforcement learning), and achieves the fastest time-to-solution (e.g., the highest score using the shortest training time for Transformer).Comment: Published in IEEE Transactions on Parallel and Distributed Systems (IEEE TPDS), vol. 32, no. 7, pp. 1725-1739, 1 July 202

Content Distribution in P2P Systems

The report provides a literature review of the state-of-the-art for content distribution. The report's contributions are of threefold. First, it gives more insight into traditional Content Distribution Networks (CDN), their requirements and open issues. Second, it discusses Peer-to-Peer (P2P) systems as a cheap and scalable alternative for CDN and extracts their design challenges. Finally, it evaluates the existing P2P systems dedicated for content distribution according to the identied requirements and challenges

Distributed Communication in Swarms of Autonomous Underwater Vehicles

Effective communication mechanisms are a key requirement for schools of submersible robots and their meaningful deployment. Large schools of identical submersibles require a fully distributed communication system which scales well and optimises for ”many-to-many” communication (omnicast, also known as gossiping). As an additional constraint, communication channels under water are typically very low bandwidth and short range. This thesis discusses possible electric and electro-magnetic wireless communication channels suitable for underwater environments. Theoretical findings on the omnicast communication problem are presented, as well as the implementation of a distributed time division multiple access (TDMA) scheduling algorithm in simulation and in hardware. It is shown theoretically and in simulation that short range links in a robotic swarm are actually an advantage, compared to links that cover large parts of the network. Experiments were carried out on custom-developed digital long-wave radio and optical link modules. The results of the experiments are used to revisit the initial assumptions on communication in multi-hop wireless networks

Distributed CSMA with pairwise coding

We consider distributed strategies for joint routing, scheduling, and network coding to maximize throughput in wireless networks. Network coding allows for an increase in network throughput under certain routing conditions. We previously developed a centralized control policy to jointly optimize for routing and scheduling combined with a simple network coding strategy using max-weight scheduling (MWS) [9]. In this work we focus on pairwise network coding and develop a distributed carrier sense multiple access (CSMA) policy that supports all arrival rates allowed by the network subject to the pairwise coding constraint. We extend our scheme to optimize for packet overhearing to increase the number of beneficial coding opportunities. Simulation results show that the CSMA strategy yields the same throughput as the optimal centralized policy of [9], but at the cost of increased delay. Moreover, overhearing provides up to an additional 25% increase in throughput on random topologies.United States. Dept. of Defense. Assistant Secretary of Defense for Research & EngineeringUnited States. Air Force (Air Force Contract FA8721-05-C-0002

Digital storytelling: Creating participatory space, addressing stigma, and enabling agency

This article explores using digital storytelling as community-based participatory research methodology with twelve secondary school learners in a rural community in South Africa who had  experienced, witnessed, or heard about HIV- and AIDS-related stigma. It explores the question of how digital storytelling can enable secondary school learners in a rural community to identify, describe and address HIV- and AIDS-related stigma. The learners produced digital stories and written reflections, and also engaged in focus group discussions. My focus is on the way in which digital storytelling created a critical space of participation and, in so doing, enabled the learner participants to identify and address issues related to HIV- and AIDS-related stigma as well as enabling them to take charge of effecting change in their community. My fieldwork experience encouraged me to think more critically about using digital storytelling in community-based research