Distributed Weight Selection in Consensus Protocols by Schatten Norm Minimization

In average consensus protocols, nodes in a network perform an iterative weighted average of their estimates and those of their neighbors. The protocol converges to the average of initial estimates of all nodes found in the network. The speed of convergence of average consensus protocols depends on the weights selected on links (to neighbors). We address in this paper how to select the weights in a given network in order to have a fast speed of convergence for these protocols. We approximate the problem of optimal weight selection by the minimization of the Schatten p-norm of a matrix with some constraints related to the connectivity of the underlying network. We then provide a totally distributed gradient method to solve the Schatten norm optimization problem. By tuning the parameter p in our proposed minimization, we can simply trade-off the quality of the solution (i.e. the speed of convergence) for communication/computation requirements (in terms of number of messages exchanged and volume of data processed). Simulation results show that our approach provides very good performance already for values of p that only needs limited information exchange. The weight optimization iterative procedure can also run in parallel with the consensus protocol and form a joint consensus-optimization procedure.Comment: N° RR-8078 (2012

Design and Analysis of Distributed Averaging with Quantized Communication

Consider a network whose nodes have some initial values, and it is desired to design an algorithm that builds on neighbor to neighbor interactions with the ultimate goal of convergence to the average of all initial node values or to some value close to that average. Such an algorithm is called generically "distributed averaging," and our goal in this paper is to study the performance of a subclass of deterministic distributed averaging algorithms where the information exchange between neighboring nodes (agents) is subject to uniform quantization. With such quantization, convergence to the precise average cannot be achieved in general, but the convergence would be to some value close to it, called quantized consensus. Using Lyapunov stability analysis, we characterize the convergence properties of the resulting nonlinear quantized system. We show that in finite time and depending on initial conditions, the algorithm will either cause all agents to reach a quantized consensus where the consensus value is the largest quantized value not greater than the average of their initial values, or will lead all variables to cycle in a small neighborhood around the average. In the latter case, we identify tight bounds for the size of the neighborhood and we further show that the error can be made arbitrarily small by adjusting the algorithm's parameters in a distributed manner

Optimal Strategies for Dynamic Weight Selection in Consensus Protocols in the Presence of an Adversary

Abstract-In this paper, we consider optimal design strategies in consensus protocols for networks vulnerable to adversarial attacks. First we study dynamic (multi-stage) weight selection optimal control for consensus protocols. For the general (multi-stage) case, the solution exists but can rarely be expressed in closed-form. In view of this, we apply optimization techniques to obtain a locally (and possibly globally) optimizing feasible control path. For the one-stage case, however, we obtain a closed-form solution for the optimal control and provide sufficient conditions for the existence of a control that makes the system reach consensus in only one iteration. We then consider a game theoretical model for the problem of a network with an adversary corrupting the control signal with noise. We derive the optimal strategies for both players (the adversary and the network designer) of the resulting game using a saddle point equilibrium (SPE) solution in mixed strategies

Distributed Weight Selection in Consensus Protocols by Schatten Norm Minimization

In average consensus protocols, nodes in a network perform an iterative weighted average of their estimates and those of their neighbors. The protocol converges to the average of initial estimates of all nodes found in the network. The speed of convergence of average consensus protocols depends on the weights selected on links (to neighbors). We address in this paper how to select the weights in a given network in order to have a fast speed of convergence for these protocols. We approximate the problem of optimal weight selection by the minimization of the Schatten p-norm of a matrix with some constraints related to the connectivity of the underlying network. We then provide a totally distributed gradient method to solve the Schatten norm optimization problem. By tuning the parameter p in our proposed minimization, we can simply trade-off the quality of the solution (i.e. the speed of convergence) for communication/computation requirements (in terms of number of messages exchanged and volume of data processed). Simulation results show that our approach provides very good performance already for values of p that only needs limited information exchange. The weight optimization iterative procedure can also run in parallel with the consensus protocol and form a joint consensus-optimization procedure.Dans les protocoles de consensus, les nœuds d'un réseau calculent itérativement une moyenne pondérée de leurs mesures et celles de leurs voisins. Le protocole converge vers la moyenne des mesures initiales de tous les nœuds présents dans le réseau. La vitesse de convergence des protocoles de consensus dépend des poids sélectionnés sur les liens entre voisins. Nous abordons dans cet article la question suivante : comment choisir les poids dans un réseau donné afin d'avoir une plus grande vitesse de convergence du protocole? Nous approchons le problème de la sélection optimale de poids avec un problème de minimisation de la p-norme de Schatten. Ce dernier est résolu de manière totalement distribuée grâce à une méthode du gradient. Selon la valeur du paramètre p, nous pouvons trouver un compromis entre la qualité de la solution (c'est-à-dire la vitesse de convergence) et les coût en termes de communication et calcul (e.g. nombre de messages échangés et volume de données traitées). Les résultats des simulations montrent que notre approche fournit une très bonne performance même avec un échange d'informations limité. La procédure d'optimisation des poids peut également se dérouler en simultané avec le protocole de consensus

Graph Clustering Based on Mixing Time of Random Walks

International audienceClustering of a graph is the task of grouping its nodes in such a way that the nodes within the same cluster are well connected, but they are less connected to nodes in different clusters. In this paper we propose a clustering metric based on the random walks' properties to evaluate the quality of a graph clustering. We also propose a randomized algorithm that identifies a locally optimal clustering of the graph according to the metric defined. The algorithm is intrinsically distributed and asynchronous. If the graph represents an actual network where nodes have computing capabilities, each node can determine its own cluster relying only on local communications. We show that the size of clusters can be adapted to the available processing capabilities to reduce the algorithm's complexity

Distributed Weight Selection in Consensus Protocols by Schatten Norm Minimization

In average consensus protocols, nodes in a network perform an iterative weighted average of their estimates and those of their neighbors. The protocol converges to the average of initial estimates of all nodes found in the network. The speed of convergence of average consensus protocols depends on the weights selected on links (to neighbors). We address in this paper how to select the weights in a given network in order to have a fast speed of convergence for these protocols. We approximate the problem of optimal weight selection by the minimization of the Schatten p-norm of a matrix with some constraints related to the connectivity of the underlying network. We then provide a totally distributed gradient method to solve the Schatten norm optimization problem. By tuning the parameter p in our proposed minimization, we can simply trade-off the quality of the solution (i.e. the speed of convergence) for communication/computation requirements (in terms of number of messages exchanged and volume of data processed). Simulation results show that our approach provides very good performance already for values of p that only needs limited information exchange. The weight optimization iterative procedure can also run in parallel with the consensus protocol and form a joint consensus-optimization procedure.Dans les protocoles de consensus, les nœuds d'un réseau calculent itérativement une moyenne pondérée de leurs mesures et celles de leurs voisins. Le protocole converge vers la moyenne des mesures initiales de tous les nœuds présents dans le réseau. La vitesse de convergence des protocoles de consensus dépend des poids sélectionnés sur les liens entre voisins. Nous abordons dans cet article la question suivante : comment choisir les poids dans un réseau donné afin d'avoir une plus grande vitesse de convergence du protocole? Nous approchons le problème de la sélection optimale de poids avec un problème de minimisation de la p-norme de Schatten. Ce dernier est résolu de manière totalement distribuée grâce à une méthode du gradient. Selon la valeur du paramètre p, nous pouvons trouver un compromis entre la qualité de la solution (c'est-à-dire la vitesse de convergence) et les coût en termes de communication et calcul (e.g. nombre de messages échangés et volume de données traitées). Les résultats des simulations montrent que notre approche fournit une très bonne performance même avec un échange d'informations limité. La procédure d'optimisation des poids peut également se dérouler en simultané avec le protocole de consensus

Topology versus Link Strength for Information Dissemination in Networks

International audienceInformation can flow in a network through communication links connecting the nodes. The topology of connections and the strength of the links are two factors that effect the speed of spread of information in the network. In this paper we show that the topology can have stronger effect on the information spread than the strength of the links. In particular, we consider an iterative belief propagation process as in average consensus protocols where each node in the network has a certain belief (a real number), and with every iteration each node updates its own belief with the weighted average of its belief and the ones of it is connected to. The speed of spread of beliefs in the network is governed by the speed of convergence of the average consensus protocol. We show by simulations that a topological optimization can have a significant faster convergence than any weight selection optimization techniques. We also give a 2-hop message averaging that perform faster convergence than standard algorithms. The simulations are done on different graph topologies: static graphs (Rings, Grids), random graphs (Erdos Renyi, Random Geometric), and a real world network (Enron internal email exchange network).L'information peut circuler dans un réseau de communication par les liens reliant les nœuds. La topologie du réseau et la force des liens sont deux facteurs qui influent sur la vitesse de propagation de l'information dans le réseau. Dans cet article, nous montrons que la topologie peut avoir un rôle plus important que la force des liens pour la vitesse de propagation de l'information. En particulier, nous considérons un processus itératif de propagation de croyance comme dans les protocoles de consensus moyen où chaque nœud dans le réseau a une certaine croyance (exprimée par un nombre réel), et à chaque itération il met à jour sa croyance en calculant une moyenne pondérée de sa croyance et de celles des ses voisins. Nous montrons que l'ajout de liens peut conduire à une augmentation de la vitesse de convergence du protocole de consensus plus significative que les techniques d'optimisation des poids. Les simulations sont effectuées sur différentes topologies: anneaux, grilles, graphes aléatoires (Erdos Renyi, graphes géométriques aléatoires) et le graphe d'échange de courriels chez Enron

Reducing Communication Overhead for Average Consensus

International audienceAn average consensus protocol is an iterative distributed algorithm to calculate the average of local values stored at the nodes of a network. Each node maintains a local estimate of the average and, at every iteration, it sends its estimate to all its neighbors and then updates the estimate by performing a weighted average of the estimates received. The average consensus protocol is guaranteed to converge only asymptotically and implementing a termination algorithm is challenging when nodes are not aware of some global information (e.g. the diameter of the network or the total number of nodes). In this paper, we are interested in decreasing the rate of the messages sent in the network as nodes estimates become closer to the average. We propose a totally distributed algorithm for average consensus where nodes send more messages when they have large differences in their estimates, and reduce their message sending rate when the consensus is almost reached. The convergence of the system is guaranteed to be within a predefined margin. Tuning the parameter provides a trade-off between the precision of consensus and communication overhead of the protocol. The proposed algorithm is robust against nodes changing their initial values and can also be applied in dynamic networks with faulty links