Cooperative Convex Optimization in Networked Systems: Augmented Lagrangian Algorithms with Directed Gossip Communication

We study distributed optimization in networked systems, where nodes cooperate to find the optimal quantity of common interest, x=x^\star. The objective function of the corresponding optimization problem is the sum of private (known only by a node,) convex, nodes' objectives and each node imposes a private convex constraint on the allowed values of x. We solve this problem for generic connected network topologies with asymmetric random link failures with a novel distributed, decentralized algorithm. We refer to this algorithm as AL-G (augmented Lagrangian gossiping,) and to its variants as AL-MG (augmented Lagrangian multi neighbor gossiping) and AL-BG (augmented Lagrangian broadcast gossiping.) The AL-G algorithm is based on the augmented Lagrangian dual function. Dual variables are updated by the standard method of multipliers, at a slow time scale. To update the primal variables, we propose a novel, Gauss-Seidel type, randomized algorithm, at a fast time scale. AL-G uses unidirectional gossip communication, only between immediate neighbors in the network and is resilient to random link failures. For networks with reliable communication (i.e., no failures,) the simplified, AL-BG (augmented Lagrangian broadcast gossiping) algorithm reduces communication, computation and data storage cost. We prove convergence for all proposed algorithms and demonstrate by simulations the effectiveness on two applications: l_1-regularized logistic regression for classification and cooperative spectrum sensing for cognitive radio networks.Comment: 28 pages, journal; revise

Fast Structuring of Radio Networks for Multi-Message Communications

We introduce collision free layerings as a powerful way to structure radio networks. These layerings can replace hard-to-compute BFS-trees in many contexts while having an efficient randomized distributed construction. We demonstrate their versatility by using them to provide near optimal distributed algorithms for several multi-message communication primitives. Designing efficient communication primitives for radio networks has a rich history that began 25 years ago when Bar-Yehuda et al. introduced fast randomized algorithms for broadcasting and for constructing BFS-trees. Their BFS-tree construction time was $O(D \log^2 n)$ rounds, where $D$ is the network diameter and $n$ is the number of nodes. Since then, the complexity of a broadcast has been resolved to be $T_{BC} = \Theta(D \log \frac{n}{D} + \log^2 n)$ rounds. On the other hand, BFS-trees have been used as a crucial building block for many communication primitives and their construction time remained a bottleneck for these primitives. We introduce collision free layerings that can be used in place of BFS-trees and we give a randomized construction of these layerings that runs in nearly broadcast time, that is, w.h.p. in $T_{Lay} = O(D \log \frac{n}{D} + \log^{2+\epsilon} n)$ rounds for any constant $\epsilon>0$. We then use these layerings to obtain: (1) A randomized algorithm for gathering $k$ messages running w.h.p. in $O(T_{Lay} + k)$ rounds. (2) A randomized $k$-message broadcast algorithm running w.h.p. in $O(T_{Lay} + k \log n)$ rounds. These algorithms are optimal up to the small difference in the additive poly-logarithmic term between $T_{BC}$ and $T_{Lay}$. Moreover, they imply the first optimal $O(n \log n)$ round randomized gossip algorithm

Faster Gossiping in Bidirectional Radio Networks with Large Labels

We consider unknown ad-hoc radio networks, when the underlying network is bidirectional and nodes can have polynomially large labels. For this model, we present a deterministic protocol for gossiping which takes $O(n \lg^2 n \lg \lg n)$ rounds. This improves upon the previous best result for deterministic gossiping for this model by [Gasienec, Potapov, Pagourtizis, Deterministic Gossiping in Radio Networks with Large labels, ESA (2002)], who present a protocol of round complexity $O(n \lg^3 n \lg \lg n)$ for this problem. This resolves open problem posed in [Gasienec, Efficient gossiping in radio networks, SIROCCO (2009)], who cite bridging gap between lower and upper bounds for this problem as an important objective. We emphasize that a salient feature of our protocol is its simplicity, especially with respect to the previous best known protocol for this problem

Information Gathering in Ad-Hoc Radio Networks with Tree Topology

We study the problem of information gathering in ad-hoc radio networks without collision detection, focussing on the case when the network forms a tree, with edges directed towards the root. Initially, each node has a piece of information that we refer to as a rumor. Our goal is to design protocols that deliver all rumors to the root of the tree as quickly as possible. The protocol must complete this task within its allotted time even though the actual tree topology is unknown when the computation starts. In the deterministic case, assuming that the nodes are labeled with small integers, we give an O(n)-time protocol that uses unbounded messages, and an O(n log n)-time protocol using bounded messages, where any message can include only one rumor. We also consider fire-and-forward protocols, in which a node can only transmit its own rumor or the rumor received in the previous step. We give a deterministic fire-and- forward protocol with running time O(n^1.5), and we show that it is asymptotically optimal. We then study randomized algorithms where the nodes are not labelled. In this model, we give an O(n log n)-time protocol and we prove that this bound is asymptotically optimal

Latency Optimal Broadcasting in Noisy Wireless Mesh Networks

In this paper, we adopt a new noisy wireless network model introduced very recently by Censor-Hillel et al. in [ACM PODC 2017, CHHZ17]. More specifically, for a given noise parameter $p\in [0,1],$ any sender has a probability of $p$ of transmitting noise or any receiver of a single transmission in its neighborhood has a probability $p$ of receiving noise. In this paper, we first propose a new asymptotically latency-optimal approximation algorithm (under faultless model) that can complete single-message broadcasting task in $D+O(\log^2 n)$ time units/rounds in any WMN of size $n,$ and diameter $D$. We then show this diameter-linear broadcasting algorithm remains robust under the noisy wireless network model and also improves the currently best known result in CHHZ17 by a $\Theta(\log\log n)$ factor. In this paper, we also further extend our robust single-message broadcasting algorithm to $k$ multi-message broadcasting scenario and show it can broadcast $k$ messages in $O(D+k\log n+\log^2 n)$ time rounds. This new robust multi-message broadcasting scheme is not only asymptotically optimal but also answers affirmatively the problem left open in CHHZ17 on the existence of an algorithm that is robust to sender and receiver faults and can broadcast $k$ messages in $O(D+k\log n + polylog(n))$ time rounds.Comment: arXiv admin note: text overlap with arXiv:1705.07369 by other author

Geographic Gossip: Efficient Averaging for Sensor Networks

Gossip algorithms for distributed computation are attractive due to their simplicity, distributed nature, and robustness in noisy and uncertain environments. However, using standard gossip algorithms can lead to a significant waste in energy by repeatedly recirculating redundant information. For realistic sensor network model topologies like grids and random geometric graphs, the inefficiency of gossip schemes is related to the slow mixing times of random walks on the communication graph. We propose and analyze an alternative gossiping scheme that exploits geographic information. By utilizing geographic routing combined with a simple resampling method, we demonstrate substantial gains over previously proposed gossip protocols. For regular graphs such as the ring or grid, our algorithm improves standard gossip by factors of $n$ and $\sqrt{n}$ respectively. For the more challenging case of random geometric graphs, our algorithm computes the true average to accuracy $\epsilon$ using $O(\frac{n^{1.5}}{\sqrt{\log n}} \log \epsilon^{-1})$ radio transmissions, which yields a $\sqrt{\frac{n}{\log n}}$ factor improvement over standard gossip algorithms. We illustrate these theoretical results with experimental comparisons between our algorithm and standard methods as applied to various classes of random fields.Comment: To appear, IEEE Transactions on Signal Processin

Message and time efficient multi-broadcast schemes

We consider message and time efficient broadcasting and multi-broadcasting in wireless ad-hoc networks, where a subset of nodes, each with a unique rumor, wish to broadcast their rumors to all destinations while minimizing the total number of transmissions and total time until all rumors arrive to their destination. Under centralized settings, we introduce a novel approximation algorithm that provides almost optimal results with respect to the number of transmissions and total time, separately. Later on, we show how to efficiently implement this algorithm under distributed settings, where the nodes have only local information about their surroundings. In addition, we show multiple approximation techniques based on the network collision detection capabilities and explain how to calibrate the algorithms' parameters to produce optimal results for time and messages.Comment: In Proceedings FOMC 2013, arXiv:1310.459