A gossip algorithm for convex consensus optimization over networks

In many applications, nodes in a network wish to achieve not only a consensus, but an optimal one. To date, a family of subgradient algorithms have been proposed to solve this problem under general convexity assumptions. This paper shows that, with a few additional mild assumptions, a fundamentally different, non-gradient-based algorithm with appealing features can be constructed. Specifically, we develop Pairwise Equalizing (PE), a gossip-style, distributed asynchronous iterative algorithm for achieving unconstrained, separable, convex consensus optimization over undirected networks with time-varying topologies, where each component function is strictly convex, continuously differentiable, and has a minimizer. We show that PE is easy to implement, bypasses limitations facing the subgradient algorithms, and produces a switched, nonlinear, networked dynamical system that is deterministically and stochastically asymptotically convergent. Moreover, we show that PE admits a common Lyapunov function and reduces to the well-studied Pairwise Averaging and Randomized Gossip Algorithm in a special case

A Consensus Approach to Distributed Convex Optimization in Multi-Agent Systems

In this thesis we address the problem of distributed unconstrained convex optimization under separability assumptions, i.e., the framework where a network of agents, each endowed with local private convex cost and subject to communication constraints, wants to collaborate to compute the minimizer of the sum of the local costs. We propose a design methodology that combines average consensus algorithms and separation of time-scales ideas. This strategy is proven, under suitable hypotheses, to be globally convergent to the true minimizer. Intuitively, the procedure lets the agents distributedly compute and sequentially update an approximated Newton-Raphson direction by means of suitable average consensus ratios. We consider both a scalar and a multidimensional scenario of the Synchronous Newton-Raphson Consensus, proposing some alternative strategies which trade-off communication and computational requirements with convergence speed. We provide analytical proofs of convergence and we show with numerical simulations that the speed of convergence of this strategy is comparable with alternative optimization strategies such as the Alternating Direction Method of Multipliers, the Distributed Subgradient Method and Distributed Control Method. Moreover, we consider the convergence rates of the Synchronous Newton-Raphson Consensus and the Gradient Descent Consensus under the simplificative assumption of quadratic local cost functions. We derive sufficient conditions which guarantee the convergence of the algorithms. From these conditions we then obtain closed form expressions that can be used to tune the parameters for maximizing the rate of convergence. Despite these formulas have been derived under quadratic local cost functions assumptions, they can be used as rules-of-thumb for tuning the parameters of the algorithms. Finally, we propose an asynchronous version of the Newton-Raphson Consensus. Beside having low computational complexity, low communication requirements and being interpretable as a distributed Newton-Raphson algorithm, the technique has also the beneficial properties of requiring very little coordination and naturally supporting time-varying topologies. Again, we analytically prove that under some assumptions it shows either local or global convergence properties. Through numerical simulations we corroborate these results and we compare the performance of the Asynchronous Newton-Raphson Consensus with other distributed optimization methods

Wireless Sensor Data Transport, Aggregation and Security

abstract: Wireless sensor networks (WSN) and the communication and the security therein have been gaining further prominence in the tech-industry recently, with the emergence of the so called Internet of Things (IoT). The steps from acquiring data and making a reactive decision base on the acquired sensor measurements are complex and requires careful execution of several steps. In many of these steps there are still technological gaps to fill that are due to the fact that several primitives that are desirable in a sensor network environment are bolt on the networks as application layer functionalities, rather than built in them. For several important functionalities that are at the core of IoT architectures we have developed a solution that is analyzed and discussed in the following chapters. The chain of steps from the acquisition of sensor samples until these samples reach a control center or the cloud where the data analytics are performed, starts with the acquisition of the sensor measurements at the correct time and, importantly, synchronously among all sensors deployed. This synchronization has to be network wide, including both the wired core network as well as the wireless edge devices. This thesis studies a decentralized and lightweight solution to synchronize and schedule IoT devices over wireless and wired networks adaptively, with very simple local signaling. Furthermore, measurement results have to be transported and aggregated over the same interface, requiring clever coordination among all nodes, as network resources are shared, keeping scalability and fail-safe operation in mind. Furthermore ensuring the integrity of measurements is a complicated task. On the one hand Cryptography can shield the network from outside attackers and therefore is the first step to take, but due to the volume of sensors must rely on an automated key distribution mechanism. On the other hand cryptography does not protect against exposed keys or inside attackers. One however can exploit statistical properties to detect and identify nodes that send false information and exclude these attacker nodes from the network to avoid data manipulation. Furthermore, if data is supplied by a third party, one can apply automated trust metric for each individual data source to define which data to accept and consider for mentioned statistical tests in the first place. Monitoring the cyber and physical activities of an IoT infrastructure in concert is another topic that is investigated in this thesis.Dissertation/ThesisDoctoral Dissertation Electrical Engineering 201

Distributed Computation and Optimization over Networks

This dissertation is devoted to the development of efficient, robust, and scalable distributed algorithms, which enable agents in a large-scale, multi-hop network to cooperatively compute a global quantity, or solve an optimization problem, with only local interactions and without any centralized coordination. Algorithms of this nature are attracting growing interest from a number of scientific communities due to their broad application, for example, to autonomous agent coordination and control in mobile ad hoc networks, distributed signal processing and data fusion in wireless sensor networks, and studies of opinion dynamics in social networks.In this dissertation, we address three fundamental problems in the area, namely: averaging, solving of positive definite linear equations, and unconstrained separable convex optimization. Based on a blend of tools and ideas from system, optimization, and graph theories, we construct a novel set of distributed algorithms---including continuous- and discrete-time, gossip and asynchronous---which solve these problems over undirected networks with arbitrary (and, in some cases, time-varying) topologies and agent memberships. We also analyze the properties of these algorithms, including their convergence rates and complexity characteristics, and compare them with existing schemes, showing analytically and numerically that our algorithms possess several appealing features.The major contributions of this dissertation are as follows: first, we show that Lyapunov stability theory may be used to shape the behavior of asynchronous distributed algorithms. This finding allows us to introduce the notion of greedy, decentralized, feedback iteration control, leading to a class of Controlled Hopwise algorithms, which are highly bandwidth/energy efficient in wireless networks. The finding also creates a new paradigm in the design of asynchronous distributed algorithms, where iterations are opportunistically controlled, as opposed to being randomized.Second, we show that the Bregman divergence of the Lagrangian of a separable convex optimization problem may be used to form a common Lyapunov function. This result enables us to derive a family of Zero-Gradient-Sum algorithms, which yield nonlinear networked dynamical systems on an invariant manifold, and which differ fundamentally from, and have pros and cons over, the existing subgradient algorithms. The derivation also shows that a gossip variant within the family generalizes the classic Pairwise Averaging, and the family itself is a natural generalization of several well-known algorithms for distributed consensus, to distributed convex optimization.Finally, we provide a series of analysis of the properties of our algorithms (e.g., boundedness, asymptotic and exponential convergence, lower and upper bounds on convergence rates, scalability) on various networks (e.g., path, cycle, regular, complete, and general graphs), describing explicitly the dependency of such properties on network topologies, problem characteristics, and algorithm parameters, including the algebraic connectivity, Laplacian spectral radius, and function curvatures

Compressive Privacy for a Linear Dynamical System

We consider a linear dynamical system in which the state vector consists of both public and private states. One or more sensors make measurements of the state vector and sends information to a fusion center, which performs the final state estimation. To achieve an optimal tradeoff between the utility of estimating the public states and protection of the private states, the measurements at each time step are linearly compressed into a lower dimensional space. Under the centralized setting where all measurements are collected by a single sensor, we propose an optimization problem and an algorithm to find the best compression matrix. Under the decentralized setting where measurements are made separately at multiple sensors, each sensor optimizes its own local compression matrix. We propose methods to separate the overall optimization problem into multiple sub-problems that can be solved locally at each sensor. We consider the cases where there is no message exchange between the sensors; and where each sensor takes turns to transmit messages to the other sensors. Simulations and empirical experiments demonstrate the efficiency of our proposed approach in allowing the fusion center to estimate the public states with good accuracy while preventing it from estimating the private states accurately