Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling

The goal of decentralized optimization over a network is to optimize a global objective formed by a sum of local (possibly nonsmooth) convex functions using only local computation and communication. It arises in various application domains, including distributed tracking and localization, multi-agent co-ordination, estimation in sensor networks, and large-scale optimization in machine learning. We develop and analyze distributed algorithms based on dual averaging of subgradients, and we provide sharp bounds on their convergence rates as a function of the network size and topology. Our method of analysis allows for a clear separation between the convergence of the optimization algorithm itself and the effects of communication constraints arising from the network structure. In particular, we show that the number of iterations required by our algorithm scales inversely in the spectral gap of the network. The sharpness of this prediction is confirmed both by theoretical lower bounds and simulations for various networks. Our approach includes both the cases of deterministic optimization and communication, as well as problems with stochastic optimization and/or communication.Comment: 40 pages, 4 figure

Epidemic Spreading with External Agents

We study epidemic spreading processes in large networks, when the spread is assisted by a small number of external agents: infection sources with bounded spreading power, but whose movement is unrestricted vis-\`a-vis the underlying network topology. For networks which are `spatially constrained', we show that the spread of infection can be significantly speeded up even by a few such external agents infecting randomly. Moreover, for general networks, we derive upper-bounds on the order of the spreading time achieved by certain simple (random/greedy) external-spreading policies. Conversely, for certain common classes of networks such as line graphs, grids and random geometric graphs, we also derive lower bounds on the order of the spreading time over all (potentially network-state aware and adversarial) external-spreading policies; these adversarial lower bounds match (up to logarithmic factors) the spreading time achieved by an external agent with a random spreading policy. This demonstrates that random, state-oblivious infection-spreading by an external agent is in fact order-wise optimal for spreading in such spatially constrained networks

Directed cooperation in multihop wireless sensors network

This paper proposes a relational abstraction for Wireless Sensors Network where node can identify its neighbors around it. Because of limited radio link range only some of nodes have a direct contact with the base station and transmission is carried out in a multihop way so information is send from one node to another towards the BS. We propose a relation π that represents cooperation between nodes and takes advantages of topological properties of the network. Based on the hop-distance from the BS any node k can determine a set N<(k) that consists of nodes to which/k should send messages in order to retain a data-flow direction towards the BS

Amorphous Placement and Informed Diffusion for Timely Monitoring by Autonomous, Resource-Constrained, Mobile Sensors

Personal communication devices are increasingly equipped with sensors for passive monitoring of encounters and surroundings. We envision the emergence of services that enable a community of mobile users carrying such resource-limited devices to query such information at remote locations in the field in which they collectively roam. One approach to implement such a service is directed placement and retrieval (DPR), whereby readings/queries about a specific location are routed to a node responsible for that location. In a mobile, potentially sparse setting, where end-to-end paths are unavailable, DPR is not an attractive solution as it would require the use of delay-tolerant (flooding-based store-carry-forward) routing of both readings and queries, which is inappropriate for applications with data freshness constraints, and which is incompatible with stringent device power/memory constraints. Alternatively, we propose the use of amorphous placement and retrieval (APR), in which routing and field monitoring are integrated through the use of a cache management scheme coupled with an informed exchange of cached samples to diffuse sensory data throughout the network, in such a way that a query answer is likely to be found close to the query origin. We argue that knowledge of the distribution of query targets could be used effectively by an informed cache management policy to maximize the utility of collective storage of all devices. Using a simple analytical model, we show that the use of informed cache management is particularly important when the mobility model results in a non-uniform distribution of users over the field. We present results from extensive simulations which show that in sparsely-connected networks, APR is more cost-effective than DPR, that it provides extra resilience to node failure and packet losses, and that its use of informed cache management yields superior performance

Nothing can compare with a population, besides agents

15 pagesLeveraging the resemblances between two areas explored so far independently enables to provide a theoretical framework for dis- tributed systems where global behaviors emerge from a set of local in- teractions. The contribution of this paper arise from the observation that population protocols and multi-agent systems (MAS) bear many resemblances. Particularly, some subclasses of MAS seem to fit the same computational power than population protocols. Population protocols provide theoretical foundations for mobile tiny device networks. On the other hand, from long-standing research study in distributed artificial in- telligence, MAS forms an interesting model for society and owns a broad spectrum of application field, from simple reactive system to social sci- ences. Linking the both model should offers several extremely interesting outcomes

Modelling and Analysis Using GROOVE

In this paper we present case studies that describe how the graph transformation tool GROOVE has been used to model problems from a wide variety of domains. These case studies highlight the wide applicability of GROOVE in particular, and of graph transformation in general. They also give concrete templates for using GROOVE in practice. Furthermore, we use the case studies to analyse the main strong and weak points of GROOVE

A survey of flooding, gossip routing, and related schemes for wireless multi- hop networks

Flooding is an essential and critical service in computer networks that is used by many routing protocols to send packets from a source to all nodes in the network. As the packets are forwarded once by each receiving node, many copies of the same packet traverse the network which leads to high redundancy and unnecessary usage of the sparse capacity of the transmission medium. Gossip routing is a well-known approach to improve the flooding in wireless multi-hop networks. Each node has a forwarding probability p that is either statically per-configured or determined by information that is available at runtime, e.g, the node degree. When a packet is received, the node selects a random number r. If the number r is below p, the packet is forwarded and otherwise, in the most simple gossip routing protocol, dropped. With this approach the redundancy can be reduced while at the same time the reachability is preserved if the value of the parameter p (and others) is chosen with consideration of the network topology. This technical report gives an overview of the relevant publications in the research domain of gossip routing and gives an insight in the improvements that can be achieved. We discuss the simulation setups and results of gossip routing protocols as well as further improved flooding schemes. The three most important metrics in this application domain are elaborated: reachability, redundancy, and management overhead. The published studies used simulation environments for their research and thus the assumptions, models, and parameters of the simulations are discussed and the feasibility of an application for real world wireless networks are highlighted. Wireless mesh networks based on IEEE 802.11 are the focus of this survey but publications about other network types and technologies are also included. As percolation theory, epidemiological models, and delay tolerant networks are often referred as foundation, inspiration, or application of gossip routing in wireless networks, a brief introduction to each research domain is included and the applicability of the particular models for the gossip routing is discussed

Sensing physical fields: Inverse problems for the diffusion equation and beyond

Due to significant advances made over the last few decades in the areas of (wireless) networking, communications and microprocessor fabrication, the use of sensor networks to observe physical phenomena is rapidly becoming commonplace. Over this period, many aspects of sensor networks have been explored, yet a thorough understanding of how to analyse and process the vast amounts of sensor data collected, remains an open area of research. This work therefore, aims to provide theoretical, as well as practical, advances this area. In particular, we consider the problem of inferring certain underlying properties of the monitored phenomena, from our sensor measurements. Within mathematics, this is commonly formulated as an inverse problem; whereas in signal processing it appears as a (multidimensional) sampling and reconstruction problem. Indeed it is well known that inverse problems are notoriously ill-posed and very demanding to solve; meanwhile viewing it as the latter also presents several technical challenges. In particular, the monitored field is usually nonbandlimited, the sensor placement is typically non-regular and the space-time dimensions of the field are generally non-homogeneous. Furthermore, although sensor production is a very advanced domain, it is near impossible and/or extremely costly to design sensors with no measurement noise. These challenges therefore motivate the need for a stable, noise robust, yet simple sampling theory for the problem at hand. In our work, we narrow the gap between the domains of inverse problems and modern sampling theory, and in so doing, extend existing results by introducing a framework for solving the inverse source problems for a class of some well-known physical phenomena. Some examples include: the reconstruction of plume sources, thermal monitoring of multi-core processors and acoustic source estimation, to name a few. We assume these phenomena and their sources can be described using partial differential equation (PDE) and parametric source models, respectively. Under this assumption, we obtain a well-posed inverse problem. Initially, we consider a phenomena governed by the two-dimensional diffusion equation -- i.e. 2-D diffusion fields, and assume that we have access to its continuous field measurements. In this setup, we derive novel exact closed-form inverse formulae that solve the inverse diffusion source problem, for a class of localized and non-localized source models. In our derivation, we prove that a particular 1-D sequence of, so called, generalized measurements of the field is governed by a power-sum series, hence it can be efficiently solved using existing algebraic methods such as Prony's method. Next, we show how to obtain these generalized measurements, by using Green's second identity to combine the continuous diffusion field with a family of well-chosen sensing functions. From these new inverse formulae, we therefore develop novel noise robust centralized and distributed reconstruction methods for diffusion fields. Specifically, we extend these inverse formulae to centralized sensor networks using numerical quadrature; conversely for distributed networks, we propose a new physics-driven consensus scheme to approximate the generalized measurements through localized interactions between the sensor nodes. Finally we provide numerical results using both synthetic and real data to validate the proposed algorithms. Given the insights gained, we eventually turn to the more general problem. That is, the two- and three-dimensional inverse source problems for any linear PDE with constant coefficients. Extending the previous framework, we solve the new class of inverse problems by establishing an otherwise subtle link with modern sampling theory. We achieved this by showing that, the desired generalized measurements can be computed by taking linear weighted-sums of the sensor measurements. The advantage of this is two-fold. First, we obtain a more flexible framework that permits the use of more general sensing functions, this freedom is important for solving the 3-D problem. Second, and remarkably, we are able to analyse many more physical phenomena beyond diffusion fields. We prove that computing the proper sequence of generalized measurements for any such field, via linear sums, reduces to approximating (a family of) exponentials with translates of a particular prototype function. We show that this prototype function depends on the Green's function of the field, and then derive an explicit formula to evaluate the proper weights. Furthermore, since we now have more freedom in selecting the sensing functions, we discuss how to make the correct choice whilst emphasizing how to retrieve the unknown source parameters from the resulting (multidimensional) Prony-like systems. Based on this new theory we develop practical, noise robust, sensor network strategies for solving the inverse source problem, and then present numerical simulation results to verify the performance of our proposed schemes.Open Acces