Gossip Algorithms for Distributed Signal Processing

Gossip algorithms are attractive for in-network processing in sensor networks because they do not require any specialized routing, there is no bottleneck or single point of failure, and they are robust to unreliable wireless network conditions. Recently, there has been a surge of activity in the computer science, control, signal processing, and information theory communities, developing faster and more robust gossip algorithms and deriving theoretical performance guarantees. This article presents an overview of recent work in the area. We describe convergence rate results, which are related to the number of transmitted messages and thus the amount of energy consumed in the network for gossiping. We discuss issues related to gossiping over wireless links, including the effects of quantization and noise, and we illustrate the use of gossip algorithms for canonical signal processing tasks including distributed estimation, source localization, and compression.Comment: Submitted to Proceedings of the IEEE, 29 page

Network Information Flow with Correlated Sources

In this paper, we consider a network communications problem in which multiple correlated sources must be delivered to a single data collector node, over a network of noisy independent point-to-point channels. We prove that perfect reconstruction of all the sources at the sink is possible if and only if, for all partitions of the network nodes into two subsets S and S^c such that the sink is always in S^c, we have that H(U_S|U_{S^c}) < \sum_{i\in S,j\in S^c} C_{ij}. Our main finding is that in this setup a general source/channel separation theorem holds, and that Shannon information behaves as a classical network flow, identical in nature to the flow of water in pipes. At first glance, it might seem surprising that separation holds in a fairly general network situation like the one we study. A closer look, however, reveals that the reason for this is that our model allows only for independent point-to-point channels between pairs of nodes, and not multiple-access and/or broadcast channels, for which separation is well known not to hold. This ``information as flow'' view provides an algorithmic interpretation for our results, among which perhaps the most important one is the optimality of implementing codes using a layered protocol stack.Comment: Final version, to appear in the IEEE Transactions on Information Theory -- contains (very) minor changes based on the last round of review

A Graph-based Framework for Transmission of Correlated Sources over Broadcast Channels

In this paper we consider the communication problem that involves transmission of correlated sources over broadcast channels. We consider a graph-based framework for this information transmission problem. The system involves a source coding module and a channel coding module. In the source coding module, the sources are efficiently mapped into a nearly semi-regular bipartite graph, and in the channel coding module, the edges of this graph are reliably transmitted over a broadcast channel. We consider nearly semi-regular bipartite graphs as discrete interface between source coding and channel coding in this multiterminal setting. We provide an information-theoretic characterization of (1) the rate of exponential growth (as a function of the number of channel uses) of the size of the bipartite graphs whose edges can be reliably transmitted over a broadcast channel and (2) the rate of exponential growth (as a function of the number of source samples) of the size of the bipartite graphs which can reliably represent a pair of correlated sources to be transmitted over a broadcast channel.Comment: 36 pages, 9 figure

Orthogonal Multiple Access with Correlated Sources: Feasible Region and Pragmatic Schemes

In this paper, we consider orthogonal multiple access coding schemes, where correlated sources are encoded in a distributed fashion and transmitted, through additive white Gaussian noise (AWGN) channels, to an access point (AP). At the AP, component decoders, associated with the source encoders, iteratively exchange soft information by taking into account the source correlation. The first goal of this paper is to investigate the ultimate achievable performance limits in terms of a multi-dimensional feasible region in the space of channel parameters, deriving insights on the impact of the number of sources. The second goal is the design of pragmatic schemes, where the sources use "off-the-shelf" channel codes. In order to analyze the performance of given coding schemes, we propose an extrinsic information transfer (EXIT)-based approach, which allows to determine the corresponding multi-dimensional feasible regions. On the basis of the proposed analytical framework, the performance of pragmatic coded schemes, based on serially concatenated convolutional codes (SCCCs), is discussed