Distributed Weight Selection in Consensus Protocols by Schatten Norm Minimization

In average consensus protocols, nodes in a network perform an iterative weighted average of their estimates and those of their neighbors. The protocol converges to the average of initial estimates of all nodes found in the network. The speed of convergence of average consensus protocols depends on the weights selected on links (to neighbors). We address in this paper how to select the weights in a given network in order to have a fast speed of convergence for these protocols. We approximate the problem of optimal weight selection by the minimization of the Schatten p-norm of a matrix with some constraints related to the connectivity of the underlying network. We then provide a totally distributed gradient method to solve the Schatten norm optimization problem. By tuning the parameter p in our proposed minimization, we can simply trade-off the quality of the solution (i.e. the speed of convergence) for communication/computation requirements (in terms of number of messages exchanged and volume of data processed). Simulation results show that our approach provides very good performance already for values of p that only needs limited information exchange. The weight optimization iterative procedure can also run in parallel with the consensus protocol and form a joint consensus-optimization procedure.Comment: N° RR-8078 (2012

Distributed Weight Selection in Consensus Protocols by Schatten Norm Minimization

In average consensus protocols, nodes in a network perform an iterative weighted average of their estimates and those of their neighbors. The protocol converges to the average of initial estimates of all nodes found in the network. The speed of convergence of average consensus protocols depends on the weights selected on links (to neighbors). We address in this paper how to select the weights in a given network in order to have a fast speed of convergence for these protocols. We approximate the problem of optimal weight selection by the minimization of the Schatten p-norm of a matrix with some constraints related to the connectivity of the underlying network. We then provide a totally distributed gradient method to solve the Schatten norm optimization problem. By tuning the parameter p in our proposed minimization, we can simply trade-off the quality of the solution (i.e. the speed of convergence) for communication/computation requirements (in terms of number of messages exchanged and volume of data processed). Simulation results show that our approach provides very good performance already for values of p that only needs limited information exchange. The weight optimization iterative procedure can also run in parallel with the consensus protocol and form a joint consensus-optimization procedure.Dans les protocoles de consensus, les nœuds d'un réseau calculent itérativement une moyenne pondérée de leurs mesures et celles de leurs voisins. Le protocole converge vers la moyenne des mesures initiales de tous les nœuds présents dans le réseau. La vitesse de convergence des protocoles de consensus dépend des poids sélectionnés sur les liens entre voisins. Nous abordons dans cet article la question suivante : comment choisir les poids dans un réseau donné afin d'avoir une plus grande vitesse de convergence du protocole? Nous approchons le problème de la sélection optimale de poids avec un problème de minimisation de la p-norme de Schatten. Ce dernier est résolu de manière totalement distribuée grâce à une méthode du gradient. Selon la valeur du paramètre p, nous pouvons trouver un compromis entre la qualité de la solution (c'est-à-dire la vitesse de convergence) et les coût en termes de communication et calcul (e.g. nombre de messages échangés et volume de données traitées). Les résultats des simulations montrent que notre approche fournit une très bonne performance même avec un échange d'informations limité. La procédure d'optimisation des poids peut également se dérouler en simultané avec le protocole de consensus

Distributed Weight Selection in Consensus Protocols by Schatten Norm Minimization

In average consensus protocols, nodes in a network perform an iterative weighted average of their estimates and those of their neighbors. The protocol converges to the average of initial estimates of all nodes found in the network. The speed of convergence of average consensus protocols depends on the weights selected on links (to neighbors). We address in this paper how to select the weights in a given network in order to have a fast speed of convergence for these protocols. We approximate the problem of optimal weight selection by the minimization of the Schatten p-norm of a matrix with some constraints related to the connectivity of the underlying network. We then provide a totally distributed gradient method to solve the Schatten norm optimization problem. By tuning the parameter p in our proposed minimization, we can simply trade-off the quality of the solution (i.e. the speed of convergence) for communication/computation requirements (in terms of number of messages exchanged and volume of data processed). Simulation results show that our approach provides very good performance already for values of p that only needs limited information exchange. The weight optimization iterative procedure can also run in parallel with the consensus protocol and form a joint consensus-optimization procedure.Dans les protocoles de consensus, les nœuds d'un réseau calculent itérativement une moyenne pondérée de leurs mesures et celles de leurs voisins. Le protocole converge vers la moyenne des mesures initiales de tous les nœuds présents dans le réseau. La vitesse de convergence des protocoles de consensus dépend des poids sélectionnés sur les liens entre voisins. Nous abordons dans cet article la question suivante : comment choisir les poids dans un réseau donné afin d'avoir une plus grande vitesse de convergence du protocole? Nous approchons le problème de la sélection optimale de poids avec un problème de minimisation de la p-norme de Schatten. Ce dernier est résolu de manière totalement distribuée grâce à une méthode du gradient. Selon la valeur du paramètre p, nous pouvons trouver un compromis entre la qualité de la solution (c'est-à-dire la vitesse de convergence) et les coût en termes de communication et calcul (e.g. nombre de messages échangés et volume de données traitées). Les résultats des simulations montrent que notre approche fournit une très bonne performance même avec un échange d'informations limité. La procédure d'optimisation des poids peut également se dérouler en simultané avec le protocole de consensus

Topology versus Link Strength for Information Dissemination in Networks

International audienceInformation can flow in a network through communication links connecting the nodes. The topology of connections and the strength of the links are two factors that effect the speed of spread of information in the network. In this paper we show that the topology can have stronger effect on the information spread than the strength of the links. In particular, we consider an iterative belief propagation process as in average consensus protocols where each node in the network has a certain belief (a real number), and with every iteration each node updates its own belief with the weighted average of its belief and the ones of it is connected to. The speed of spread of beliefs in the network is governed by the speed of convergence of the average consensus protocol. We show by simulations that a topological optimization can have a significant faster convergence than any weight selection optimization techniques. We also give a 2-hop message averaging that perform faster convergence than standard algorithms. The simulations are done on different graph topologies: static graphs (Rings, Grids), random graphs (Erdos Renyi, Random Geometric), and a real world network (Enron internal email exchange network).L'information peut circuler dans un réseau de communication par les liens reliant les nœuds. La topologie du réseau et la force des liens sont deux facteurs qui influent sur la vitesse de propagation de l'information dans le réseau. Dans cet article, nous montrons que la topologie peut avoir un rôle plus important que la force des liens pour la vitesse de propagation de l'information. En particulier, nous considérons un processus itératif de propagation de croyance comme dans les protocoles de consensus moyen où chaque nœud dans le réseau a une certaine croyance (exprimée par un nombre réel), et à chaque itération il met à jour sa croyance en calculant une moyenne pondérée de sa croyance et de celles des ses voisins. Nous montrons que l'ajout de liens peut conduire à une augmentation de la vitesse de convergence du protocole de consensus plus significative que les techniques d'optimisation des poids. Les simulations sont effectuées sur différentes topologies: anneaux, grilles, graphes aléatoires (Erdos Renyi, graphes géométriques aléatoires) et le graphe d'échange de courriels chez Enron

Optimal Strategies for Dynamic Weight Selection in Consensus Protocols in the Presence of an Adversary

Abstract-In this paper, we consider optimal design strategies in consensus protocols for networks vulnerable to adversarial attacks. First we study dynamic (multi-stage) weight selection optimal control for consensus protocols. For the general (multi-stage) case, the solution exists but can rarely be expressed in closed-form. In view of this, we apply optimization techniques to obtain a locally (and possibly globally) optimizing feasible control path. For the one-stage case, however, we obtain a closed-form solution for the optimal control and provide sufficient conditions for the existence of a control that makes the system reach consensus in only one iteration. We then consider a game theoretical model for the problem of a network with an adversary corrupting the control signal with noise. We derive the optimal strategies for both players (the adversary and the network designer) of the resulting game using a saddle point equilibrium (SPE) solution in mixed strategies

Design and Analysis of Distributed Averaging with Quantized Communication

Consider a network whose nodes have some initial values, and it is desired to design an algorithm that builds on neighbor to neighbor interactions with the ultimate goal of convergence to the average of all initial node values or to some value close to that average. Such an algorithm is called generically "distributed averaging," and our goal in this paper is to study the performance of a subclass of deterministic distributed averaging algorithms where the information exchange between neighboring nodes (agents) is subject to uniform quantization. With such quantization, convergence to the precise average cannot be achieved in general, but the convergence would be to some value close to it, called quantized consensus. Using Lyapunov stability analysis, we characterize the convergence properties of the resulting nonlinear quantized system. We show that in finite time and depending on initial conditions, the algorithm will either cause all agents to reach a quantized consensus where the consensus value is the largest quantized value not greater than the average of their initial values, or will lead all variables to cycle in a small neighborhood around the average. In the latter case, we identify tight bounds for the size of the neighborhood and we further show that the error can be made arbitrarily small by adjusting the algorithm's parameters in a distributed manner

Robust subspace learning for static and dynamic affect and behaviour modelling

Machine analysis of human affect and behavior in naturalistic contexts has witnessed a growing attention in the last decade from various disciplines ranging from social and cognitive sciences to machine learning and computer vision. Endowing machines with the ability to seamlessly detect, analyze, model, predict as well as simulate and synthesize manifestations of internal emotional and behavioral states in real-world data is deemed essential for the deployment of next-generation, emotionally- and socially-competent human-centered interfaces. In this thesis, we are primarily motivated by the problem of modeling, recognizing and predicting spontaneous expressions of non-verbal human affect and behavior manifested through either low-level facial attributes in static images or high-level semantic events in image sequences. Both visual data and annotations of naturalistic affect and behavior naturally contain noisy measurements of unbounded magnitude at random locations, commonly referred to as ‘outliers’. We present here machine learning methods that are robust to such gross, sparse noise. First, we deal with static analysis of face images, viewing the latter as a superposition of mutually-incoherent, low-complexity components corresponding to facial attributes, such as facial identity, expressions and activation of atomic facial muscle actions. We develop a robust, discriminant dictionary learning framework to extract these components from grossly corrupted training data and combine it with sparse representation to recognize the associated attributes. We demonstrate that our framework can jointly address interrelated classification tasks such as face and facial expression recognition. Inspired by the well-documented importance of the temporal aspect in perceiving affect and behavior, we direct the bulk of our research efforts into continuous-time modeling of dimensional affect and social behavior. Having identified a gap in the literature which is the lack of data containing annotations of social attitudes in continuous time and scale, we first curate a new audio-visual database of multi-party conversations from political debates annotated frame-by-frame in terms of real-valued conflict intensity and use it to conduct the first study on continuous-time conflict intensity estimation. Our experimental findings corroborate previous evidence indicating the inability of existing classifiers in capturing the hidden temporal structures of affective and behavioral displays. We present here a novel dynamic behavior analysis framework which models temporal dynamics in an explicit way, based on the natural assumption that continuous- time annotations of smoothly-varying affect or behavior can be viewed as outputs of a low-complexity linear dynamical system when behavioral cues (features) act as system inputs. A novel robust structured rank minimization framework is proposed to estimate the system parameters in the presence of gross corruptions and partially missing data. Experiments on prediction of dimensional conflict and affect as well as multi-object tracking from detection validate the effectiveness of our predictive framework and demonstrate that for the first time that complex human behavior and affect can be learned and predicted based on small training sets of person(s)-specific observations.Open Acces

New and Provable Results for Network Inference Problems and Multi-agent Optimization Algorithms

abstract: Our ability to understand networks is important to many applications, from the analysis and modeling of biological networks to analyzing social networks. Unveiling network dynamics allows us to make predictions and decisions. Moreover, network dynamics models have inspired new ideas for computational methods involving multi-agent cooperation, offering effective solutions for optimization tasks. This dissertation presents new theoretical results on network inference and multi-agent optimization, split into two parts - The first part deals with modeling and identification of network dynamics. I study two types of network dynamics arising from social and gene networks. Based on the network dynamics, the proposed network identification method works like a `network RADAR', meaning that interaction strengths between agents are inferred by injecting `signal' into the network and observing the resultant reverberation. In social networks, this is accomplished by stubborn agents whose opinions do not change throughout a discussion. In gene networks, genes are suppressed to create desired perturbations. The steady-states under these perturbations are characterized. In contrast to the common assumption of full rank input, I take a laxer assumption where low-rank input is used, to better model the empirical network data. Importantly, a network is proven to be identifiable from low rank data of rank that grows proportional to the network's sparsity. The proposed method is applied to synthetic and empirical data, and is shown to offer superior performance compared to prior work. The second part is concerned with algorithms on networks. I develop three consensus-based algorithms for multi-agent optimization. The first method is a decentralized Frank-Wolfe (DeFW) algorithm. The main advantage of DeFW lies on its projection-free nature, where we can replace the costly projection step in traditional algorithms by a low-cost linear optimization step. I prove the convergence rates of DeFW for convex and non-convex problems. I also develop two consensus-based alternating optimization algorithms --- one for least square problems and one for non-convex problems. These algorithms exploit the problem structure for faster convergence and their efficacy is demonstrated by numerical simulations. I conclude this dissertation by describing future research directions.Dissertation/ThesisDoctoral Dissertation Electrical Engineering 201

Knowledge Accumulation of Microbial Data Aiming at a Dynamic Taxonomic Framework

Deze thesis is een poging om precies dit onderzoeksgebied te overbruggen dat ligt tussen ruw gegeven en abstract concept, tussen praktijk en theorie, binnen het kader van de hedendaagse bacteriële taxonomie. Als gevolg hiervan is het een kruisbestuiving geworden tussen microbiologie, wiskunde en computerwetenschappen. De kunst om het landschap van de bacteriële diversiteit uit te tekenen, gebruikt als een metafoor voor het modelleren van de taxonomie, vereist het bepalen van een representatieve waaier aan reproduceerbare en vergelijkbare experimentele kenmerken van een verzameling bacteriën (microbiologie/taxonomie), het ontwerpen en implementeren van objectieve classificatiemethodes voor het groeperen van gegevens op een niet gecoördineerde manier (wiskunde/classificatie) en het consolideren van experimentele gegevens en hun verschillende onderverdelingen via een uniforme en weldoordachte aanpak (computerwetenschappen/kennisbeheer). Men kan zich gemakkelijk een globaal kennissysteem voor de geest halen dat de vellen vol experimentele gegevens die voortspruiten uit de microbiologische onderzoeksverrichtingen op een gestructureerde en geüniformiseerde manier kan absorberen. Een dergelijk kennisbeheersysteem zou een ongelofelijke vooruitgang betekenen voor de mogelijke toepassing van intelligente en goed gefundeerde methodes voor het ontginnen van de gegevens, ingezet als hulpmiddel om het afbakenen van objectieve en universele taxonomische consensusmodellen op een betere manier te stroomlijnen en te automatiseren. Bovendien kunnen dergelijke inferentiesystemen in staat worden geacht om ogenblikkelijk te reageren op een toevloed van nieuwe gegevens en interactief te communiceren met de buitenwereld indien noodzakelijke stukken voor het vervolledigen van de taxonomische puzzel zouden ontbreken. De geldigheid van nieuwe inzichten of hypothesen omtrent het leven en de evolutie van bacteriën zou onmiddellijk kunnen getoetst worden aan deze vergaarbakken vol kennis, mogelijks met een directe aanpassing van bestaande taxonomische modellen tot gevolg. Vooraleer de betrachtingen van een autodidactisch inferentiesysteem voor het uittekenen van het landschap van de bacteriële diversiteit kunnen gerealiseerd worden, moeten belangrijke technische en organisatorische hindernissen overwonnen worden. Dit vraagt het verleggen van de grenzen van een mondiale uitwisseling van gegevens, het nasporen en invullen van de hiaten in de waarnemingen, en het verkennen van de mogelijkheden van nieuwe technieken voor het ontginnen van gegevens, ten voordele van een beter inzicht in het leven en de evolutie van bacteriën. Spijts de nog vele onopgeloste kwesties, kunnen de ideeën die worden aangebracht in deze verhandeling als stimulans en leidraad dienen bij het integreren en exploiteren van microbiële gegevens, in plaats van het blijvend koesteren van een ijdele hoo