Greedy Gossip with Eavesdropping

This paper presents greedy gossip with eavesdropping (GGE), a novel randomized gossip algorithm for distributed computation of the average consensus problem. In gossip algorithms, nodes in the network randomly communicate with their neighbors and exchange information iteratively. The algorithms are simple and decentralized, making them attractive for wireless network applications. In general, gossip algorithms are robust to unreliable wireless conditions and time varying network topologies. In this paper we introduce GGE and demonstrate that greedy updates lead to rapid convergence. We do not require nodes to have any location information. Instead, greedy updates are made possible by exploiting the broadcast nature of wireless communications. During the operation of GGE, when a node decides to gossip, instead of choosing one of its neighbors at random, it makes a greedy selection, choosing the node which has the value most different from its own. In order to make this selection, nodes need to know their neighbors' values. Therefore, we assume that all transmissions are wireless broadcasts and nodes keep track of their neighbors' values by eavesdropping on their communications. We show that the convergence of GGE is guaranteed for connected network topologies. We also study the rates of convergence and illustrate, through theoretical bounds and numerical simulations, that GGE consistently outperforms randomized gossip and performs comparably to geographic gossip on moderate-sized random geometric graph topologies.Comment: 25 pages, 7 figure

A Chemistry-Inspired Framework for Achieving Consensus in Wireless Sensor Networks

The aim of this paper is to show how simple interaction mechanisms, inspired by chemical systems, can provide the basic tools to design and analyze a mathematical model for achieving consensus in wireless sensor networks, characterized by balanced directed graphs. The convergence and stability of the model are first proven by using new mathematical tools, which are borrowed directly from chemical theory, and then validated by means of simulation results, for different network topologies and number of sensors. The underlying chemical theory is also used to derive simple interaction rules that may account for practical issues, such as the estimation of the number of neighbors and the robustness against perturbations. Finally, the proposed chemical solution is validated under real-world conditions by means of a four-node hardware implementation where the exchange of information among nodes takes place in a distributed manner (with no need for any admission control and synchronism procedure), simply relying on the transmission of a pulse whose rate is proportional to the state of each sensor.Comment: 12 pages, 10 figures, submitted to IEEE Sensors Journa

Fault-Tolerant Aggregation: Flow-Updating Meets Mass-Distribution

Flow-Updating (FU) is a fault-tolerant technique that has proved to be efficient in practice for the distributed computation of aggregate functions in communication networks where individual processors do not have access to global information. Previous distributed aggregation protocols, based on repeated sharing of input values (or mass) among processors, sometimes called Mass-Distribution (MD) protocols, are not resilient to communication failures (or message loss) because such failures yield a loss of mass. In this paper, we present a protocol which we call Mass-Distribution with Flow-Updating (MDFU). We obtain MDFU by applying FU techniques to classic MD. We analyze the convergence time of MDFU showing that stochastic message loss produces low overhead. This is the first convergence proof of an FU-based algorithm. We evaluate MDFU experimentally, comparing it with previous MD and FU protocols, and verifying the behavior predicted by the analysis. Finally, given that MDFU incurs a fixed deviation proportional to the message-loss rate, we adjust the accuracy of MDFU heuristically in a new protocol called MDFU with Linear Prediction (MDFU-LP). The evaluation shows that both MDFU and MDFU-LP behave very well in practice, even under high rates of message loss and even changing the input values dynamically.Comment: 18 pages, 5 figures, To appear in OPODIS 201

Push sum with transmission failures

The push-sum algorithm allows distributed computing of the average on a directed graph, and is particularly relevant when one is restricted to one-way and/or asynchronous communications. We investigate its behavior in the presence of unreliable communication channels where messages can be lost. We show that exponential convergence still holds and deduce fundamental properties that implicitly describe the distribution of the final value obtained. We analyze the error of the final common value we get for the essential case of two nodes, both theoretically and numerically. We provide performance comparison with a standard consensus algorithm

D2^2: Decentralized Training over Decentralized Data

While training a machine learning model using multiple workers, each of which collects data from their own data sources, it would be most useful when the data collected from different workers can be {\em unique} and {\em different}. Ironically, recent analysis of decentralized parallel stochastic gradient descent (D-PSGD) relies on the assumption that the data hosted on different workers are {\em not too different}. In this paper, we ask the question: {\em Can we design a decentralized parallel stochastic gradient descent algorithm that is less sensitive to the data variance across workers?} In this paper, we present D$^2$, a novel decentralized parallel stochastic gradient descent algorithm designed for large data variance \xr{among workers} (imprecisely, "decentralized" data). The core of D$^2$ is a variance blackuction extension of the standard D-PSGD algorithm, which improves the convergence rate from $O\left({\sigma \over \sqrt{nT}} + {(n\zeta^2)^{\frac{1}{3}} \over T^{2/3}}\right)$ to $O\left({\sigma \over \sqrt{nT}}\right)$ where $\zeta^{2}$ denotes the variance among data on different workers. As a result, D$^2$ is robust to data variance among workers. We empirically evaluated D$^2$ on image classification tasks where each worker has access to only the data of a limited set of labels, and find that D$^2$ significantly outperforms D-PSGD

ODIN: Obfuscation-based privacy-preserving consensus algorithm for Decentralized Information fusion in smart device Networks

The large spread of sensors and smart devices in urban infrastructures are motivating research in the area of the Internet of Things (IoT) to develop new services and improve citizens’ quality of life. Sensors and smart devices generate large amounts of measurement data from sensing the environment, which is used to enable services such as control of power consumption or traffic density. To deal with such a large amount of information and provide accurate measurements, service providers can adopt information fusion, which given the decentralized nature of urban deployments can be performed by means of consensus algorithms. These algorithms allow distributed agents to (iteratively) compute linear functions on the exchanged data, and take decisions based on the outcome, without the need for the support of a central entity. However, the use of consensus algorithms raises several security concerns, especially when private or security critical information is involved in the computation. In this article we propose ODIN, a novel algorithm allowing information fusion over encrypted data. ODIN is a privacy-preserving extension of the popular consensus gossip algorithm, which prevents distributed agents from having direct access to the data while they iteratively reach consensus; agents cannot access even the final consensus value but can only retrieve partial information (e.g., a binary decision). ODIN uses efficient additive obfuscation and proxy re-encryption during the update steps and garbled circuits to make final decisions on the obfuscated consensus. We discuss the security of our proposal and show its practicability and efficiency on real-world resource-constrained devices, developing a prototype implementation for Raspberry Pi devices

ODIN: Obfuscation-based privacy-preserving consensus algorithm for Decentralized Information fusion in smart device Networks

The large spread of sensors and smart devices in urban infrastructures are motivating research in the area of the Internet of Things (IoT) to develop new services and improve citizens’ quality of life. Sensors and smart devices generate large amounts of measurement data from sensing the environment, which is used to enable services such as control of power consumption or traffic density. To deal with such a large amount of information and provide accurate measurements, service providers can adopt information fusion, which given the decentralized nature of urban deployments can be performed by means of consensus algorithms. These algorithms allow distributed agents to (iteratively) compute linear functions on the exchanged data, and take decisions based on the outcome, without the need for the support of a central entity. However, the use of consensus algorithms raises several security concerns, especially when private or security critical information is involved in the computation. In this article we propose ODIN, a novel algorithm allowing information fusion over encrypted data. ODIN is a privacy-preserving extension of the popular consensus gossip algorithm, which prevents distributed agents from having direct access to the data while they iteratively reach consensus; agents cannot access even the final consensus value but can only retrieve partial information (e.g., a binary decision). ODIN uses efficient additive obfuscation and proxy re-encryption during the update steps and garbled circuits to make final decisions on the obfuscated consensus. We discuss the security of our proposal and show its practicability and efficiency on real-world resource-constrained devices, developing a prototype implementation for Raspberry Pi devices