Gossip Algorithms for Distributed Signal Processing

Gossip algorithms are attractive for in-network processing in sensor networks because they do not require any specialized routing, there is no bottleneck or single point of failure, and they are robust to unreliable wireless network conditions. Recently, there has been a surge of activity in the computer science, control, signal processing, and information theory communities, developing faster and more robust gossip algorithms and deriving theoretical performance guarantees. This article presents an overview of recent work in the area. We describe convergence rate results, which are related to the number of transmitted messages and thus the amount of energy consumed in the network for gossiping. We discuss issues related to gossiping over wireless links, including the effects of quantization and noise, and we illustrate the use of gossip algorithms for canonical signal processing tasks including distributed estimation, source localization, and compression.Comment: Submitted to Proceedings of the IEEE, 29 page

Randomized Algorithms for Distributed Nonlinear Optimization Under Sparsity Constraints

Distributed optimization in multi-agent systems under sparsity constraints has recently received a lot of attention. In this paper, we consider the in-network minimization of a continuously differentiable nonlinear function which is a combination of local agent objective functions subject to sparsity constraints on the variables. A crucial issue of in-network optimization is the handling of the communications, which may be expensive. This calls for efficient algorithms, that are able to reduce the number of required communication links and transmitted messages. To this end, we focus on asynchronous and randomized distributed techniques. Based on consensus techniques and iterative hard thresholding methods, we propose three methods that attempt to minimize the given function, promoting sparsity of the solution: asynchronous hard thresholding (AHT), broadcast hard thresholding (BHT), and gossip hard thresholding (GHT). Although similar in many aspects, it is difficult to obtain a unified analysis for the proposed algorithms. Specifically, we theoretically prove the convergence and characterize the limit points of AHT in regular networks under some proper assumptions on the functions to be minimized. For BHT and GHT, instead, we characterize the fixed points of the maps that rule their dynamics in terms of stationary points of original problem. Finally, we illustrate the implementation of our techniques in compressed sensing and present several numerical results on performance and number of transmissions required for convergence

Energy-aware Gossip Protocol for Wireless Sensor Networks

Dissertação de mestrado em Engenharia InformáticaIn Wireless Sensor Networks (WSNs), typically composed of nodes with resource constraints, leveraging efficient processes is crucial to enhance the network longevity and consequently the sustainability in ultra-dense and heterogeneous environments, such as smart cities. Epidemic algorithms are usually efficient in delivering packets to a sink or to all it’s peers but have poor energy efficiency due to the amount of packet redundancy. Directional algorithms, such as Minimum Cost Forward Algorithm (MCFA) or Directed Diffusion, yield high energy efficiency but fail to handle mobile environments, and have poor network coverage. This work proposes a new epidemic algorithm that uses the current energy state of the network to create a topology that is cyclically updated, fault tolerant, whilst being able to handle the challenges of a static or mobile heterogeneous network. Depending on the application, tuning in the protocol settings can be made to prioritise desired characteristics. The proposed protocol has a small computational footprint and the required memory is proportional not to the size of the network, but to the number of neighbours of a node, enabling high scalability. The proposed protocol was tested, using a ESP8266 as an energy model reference, in a simulated environment with ad-hoc wireless nodes. It was implemented at the application level with UDP sockets, and resulted in a highly energy efficient protocol, capable of leveraging extended network longevity with different static or mobile topologies, with results comparable to a static directional algorithm in delivery efficiency.Em Redes de Sensores sem Fios (RSF), tipicamente compostas por nós com recursos lim-itados, alavancar processos eficientes é crucial para aumentar o tempo de vida da rede e consequentemente a sustentabilidade em ambientes heterogéneos e ultra densos, como cidades inteligentes por exemplo. Algoritmos epidêmicos são geralmente eficientes em en-tregar pacotes para um sink ou para todos os nós da rede, no entanto têm baixa eficiência energética devido a alta taxa de duplicação de pacotes. Algoritmos direcionais, como o MCFA ou de Difusão Direta, rendem alta eficiência energética mas não conseguem lidar com ambientes móveis, e alcançam baixa cobertura da rede. Este trabalho propõe um novo protocolo epidêmico que faz uso do estado energético atual da rede para criar uma topologia que por sua vez atualizada ciclicamente, tolerante a falhas, ao mesmo tempo que é capaz de lidar com os desafios de uma rede heterogênea estática ou móvel. A depender da aplicação, ajustes podem ser feitos às configurações do protocolo para que o mesmo priorize determinadas características. O protocolo proposto tem um pequeno impacto computacional e a memória requerida é proporcional somente à quantidade de vizinhos do nó, não ao tamanho da rede inteira, permitindo assim alta escalabilidade. O algoritmo proposto foi testado fazendo uso do modelo energético de uma ESP8266, em um ambiente simulado com uma rede sem fios ad-hoc. Foi implementado à nível aplicacional com sockets UDP, e resultou em um protocol energeticamente eficiente, capaz de disponibilizar alta longevidade da rede mesmo com diferentes topologias estáticas ou móveis com resultados comparáveis à um protocolo direcional em termos de eficiência na entrega de pacotes

Over-the-Air Computation for Distributed Systems: Something Old and Something New

Facing the upcoming era of Internet-of-Things and connected intelligence, efficient information processing, computation and communication design becomes a key challenge in large-scale intelligent systems. Recently, Over-the-Air (OtA) computation has been proposed for data aggregation and distributed function computation over a large set of network nodes. Theoretical foundations for this concept exist for a long time, but it was mainly investigated within the context of wireless sensor networks. There are still many open questions when applying OtA computation in different types of distributed systems where modern wireless communication technology is applied. In this article, we provide a comprehensive overview of the OtA computation principle and its applications in distributed learning, control, and inference systems, for both server-coordinated and fully decentralized architectures. Particularly, we highlight the importance of the statistical heterogeneity of data and wireless channels, the temporal evolution of model updates, and the choice of performance metrics, for the communication design in OtA federated learning (FL) systems. Several key challenges in privacy, security and robustness aspects of OtA FL are also identified for further investigation.Comment: 7 pages, 3 figures, submitted for possible publicatio

Distributed Recovery of Jointly Sparse Signals Under Communication Constraints

The problem of the distributed recovery of jointly sparse signals has attracted much attention recently. Let us assume that the nodes of a network observe different sparse signals with common support; starting from linear, compressed measurements, and exploiting network communication, each node aims at reconstructing the support and the non-zero values of its observed signal. In the literature, distributed greedy algorithms have been proposed to tackle this problem, among which the most reliable ones require a large amount of transmitted data, which barely adapts to realistic network communication constraints. In this work, we address the problem through a reweighted l1 soft thresholding technique, in which the threshold is iteratively tuned based on the current estimate of the support. The proposed method adapts to constrained networks, as it requires only local communication among neighbors, and the transmitted messages are indices from a finite set. We analytically prove the convergence of the proposed algorithm and we show that it outperforms the state-of-the-art greedy methods in terms of balance between recovery accuracy and communication load

Distributed Estimation and Performance Limits in Resource-constrained Wireless Sensor Networks

Distributed inference arising in sensor networks has been an interesting and promising discipline in recent years. The goal of this dissertation is to investigate several issues related to distributed inference in sensor networks, emphasizing parameter estimation and target tracking with resource-constrainted networks. To reduce the transmissions between sensors and the fusion center thereby saving bandwidth and energy consumption in sensor networks, a novel methodology, where each local sensor performs a censoring procedure based on the normalized innovation square (NIS), is proposed for the sequential Bayesian estimation problem in this dissertation. In this methodology, each sensor sends only the informative measurements and the fusion center fuses both missing measurements and received ones to yield more accurate inference. The new methodology is derived for both linear and nonlinear dynamic systems, and both scalar and vector measurements. The relationship between the censoring rule based on NIS and the one based on Kullback-Leibler (KL) divergence is investigated. A probabilistic transmission model over multiple access channels (MACs) is investigated. With this model, a relationship between the sensor management and compressive sensing problems is established, based on which, the sensor management problem becomes a constrained optimization problem, where the goal is to determine the optimal values of probabilities that each sensor should transmit with such that the determinant of the Fisher information matrix (FIM) at any given time step is maximized. The performance of the proposed compressive sensing based sensor management methodology in terms of accuracy of inference is investigated. For the Bayesian parameter estimation problem, a framework is proposed where quantized observations from local sensors are not directly fused at the fusion center, instead, an additive noise is injected independently to each quantized observation. The injected noise performs as a low-pass filter in the characteristic function (CF) domain, and therefore, is capable of recoverving the original analog data if certain conditions are satisfied. The optimal estimator based on the new framework is derived, so is the performance bound in terms of Fisher information. Moreover, a sub-optimal estimator, namely, linear minimum mean square error estimator (LMMSE) is derived, due to the fact that the proposed framework theoretically justifies the additive noise modeling of the quantization process. The bit allocation problem based on the framework is also investigated. A source localization problem in a large-scale sensor network is explored. The maximum-likelihood (ML) estimator based on the quantized data from local sensors and its performance bound in terms of Cram\\u27{e}r-Rao lower bound (CRLB) are derived. Since the number of sensors is large, the law of large numbers (LLN) is utilized to obtain a closed-form version of the performance bound, which clearly shows the dependence of the bound on the sensor density, $i.e.,$ the Fisher information is a linearly increasing function of the sensor density. Error incurred by the LLN approximation is also theoretically analyzed. Furthermore, the design of sub-optimal local sensor quantizers based on the closed-form solution is proposed. The problem of on-line performance evaluation for state estimation of a moving target is studied. In particular, a compact and efficient recursive conditional Posterior Cram\\u27{e}r-Rao lower bound (PCRLB) is proposed. This bound provides theoretical justification for a heuristic one proposed by other researchers in this area. Theoretical complexity analysis is provided to show the efficiency of the proposed bound, compared to the existing bound

Architectures and algorithms for dynamic overlay networks

Most of today’s Internet of Things (IoT) applications assume that data will be moved offdevices into centralized cloud platforms. While existing IoT systems leverage cloud-based analytics for meaningful data reasoning, the assumption that data should always be moved off the devices is problematic. The amount of data to be moved from devices over Internet gateways to cloud platforms is huge which potentially make it cost inefficient. In other scenarios, privacy concerns of customers or organizational rules complicate the process of transferring data to third-party data centers.This dissertation proposes architectures and dynamic overlay network algorithms for in-networkand edge processing of data offered by the globally available IoT devices and provides a global platform for meaningful and responsive data analysis and decision making. The proposed techniques shift IoT analytics from a ”collect data now and analyze it later” scenario to directlyproviding meaningful information from the in-network processing of devices data at or near thedevices. The techniques serve future IoT use cases including distributed context awareness, on-demand data analysis, and in-network decision making. The dissertation comprises three main components.The first component is a device management protocol for cloning devices’ data in proximateEdge Computing platforms. Unlike existing application-layer IoT management protocols theproposed protocol uses the LTE LTE-A radio frame structure, device-to-device communication,and IoT data properties to avoid excessive network access latency in existing technologies.The second component realizes distributed IoT analytics as overlay networks of devices clones. By means of virtual network embedding, it selects and interconnects devices’ clones to efficiently realize applications’ virtual topologies to achieve goals such as minimum latency, minimum infrastructure cost, or maximum infrastructure utilization.Finally, the dissertation presents a communication middleware that allows autonomous discovery, self-deployment, and online migration of devices’ clones across heterogeneous Edge computing platforms. The middleware ensures that communication latency between clones is kept minimum despite the uncontrolled variability of the network and hosting platforms conditions.We evaluate the proposed architectures and algorithms through simulations and prototypeimplementation of various components in controlled testbed environments, which we evaluateusing real user applications. We explore the feasibility of the proposed techniques from boththeoretical and practical perspectives.Keywords: Cloud Computing, Internet of Things, Algorithmic Game Theory, Compressive Sensin