Pooling or sampling: Collective dynamics for electrical flow estimation

The computation of electrical flows is a crucial primitive for many recently proposed optimization algorithms on weighted networks. While typically implemented as a centralized subroutine, the ability to perform this task in a fully decentralized way is implicit in a number of biological systems. Thus, a natural question is whether this task can provably be accomplished in an efficient way by a network of agents executing a simple protocol. We provide a positive answer, proposing two distributed approaches to electrical flow computation on a weighted network: a deterministic process mimicking Jacobi's iterative method for solving linear systems, and a randomized token diffusion process, based on revisiting a classical random walk process on a graph with an absorbing node. We show that both processes converge to a solution of Kirchhoff's node potential equations, derive bounds on their convergence rates in terms of the weights of the network, and analyze their time and message complexity

Diff-DAC: Distributed Actor-Critic for Average Multitask Deep Reinforcement Learning

We propose a fully distributed actor-critic algorithm approximated by deep neural networks, named \textit{Diff-DAC}, with application to single-task and to average multitask reinforcement learning (MRL). Each agent has access to data from its local task only, but it aims to learn a policy that performs well on average for the whole set of tasks. During the learning process, agents communicate their value-policy parameters to their neighbors, diffusing the information across the network, so that they converge to a common policy, with no need for a central node. The method is scalable, since the computational and communication costs per agent grow with its number of neighbors. We derive Diff-DAC's from duality theory and provide novel insights into the standard actor-critic framework, showing that it is actually an instance of the dual ascent method that approximates the solution of a linear program. Experiments suggest that Diff-DAC can outperform the single previous distributed MRL approach (i.e., Dist-MTLPS) and even the centralized architecture

Cooperative learning in multi-agent systems from intermittent measurements

Motivated by the problem of tracking a direction in a decentralized way, we consider the general problem of cooperative learning in multi-agent systems with time-varying connectivity and intermittent measurements. We propose a distributed learning protocol capable of learning an unknown vector $\mu$ from noisy measurements made independently by autonomous nodes. Our protocol is completely distributed and able to cope with the time-varying, unpredictable, and noisy nature of inter-agent communication, and intermittent noisy measurements of $\mu$. Our main result bounds the learning speed of our protocol in terms of the size and combinatorial features of the (time-varying) networks connecting the nodes

Autonomic Management for Multi-agent Systems

Autonomic computing is a computing system that can manage itself by self-configuration, self-healing, self-optimizing and self-protection. Researchers have been emphasizing the strong role that multi agent systems can play progressively towards the design and implementation of complex autonomic systems. The important of autonomic computing is to create computing systems capable of managing themselves to a far greater extent than they do today. With the nature of autonomy, reactivity, sociality and pro-activity, software agents are promising to make autonomic computing system a reality. This paper mixed multi-agent system with autonomic feature that completely hides its complexity from users/services. Mentioned Java Application Development Framework as platform example of this environment, could applied to web services as front end to users. With multi agent support it also provides adaptability, intelligence, collaboration, goal oriented interactions, flexibility, mobility and persistence in software systemsComment: 4page,2figur

Distributed Policy Evaluation Under Multiple Behavior Strategies

We apply diffusion strategies to develop a fully-distributed cooperative reinforcement learning algorithm in which agents in a network communicate only with their immediate neighbors to improve predictions about their environment. The algorithm can also be applied to off-policy learning, meaning that the agents can predict the response to a behavior different from the actual policies they are following. The proposed distributed strategy is efficient, with linear complexity in both computation time and memory footprint. We provide a mean-square-error performance analysis and establish convergence under constant step-size updates, which endow the network with continuous learning capabilities. The results show a clear gain from cooperation: when the individual agents can estimate the solution, cooperation increases stability and reduces bias and variance of the prediction error; but, more importantly, the network is able to approach the optimal solution even when none of the individual agents can (e.g., when the individual behavior policies restrict each agent to sample a small portion of the state space).Comment: 36 pages, 4 figures, accepted for publication on IEEE Transactions on Automatic Contro

Using Social Networks to Aid Homeless Shelters: Dynamic Influence Maximization under Uncertainty - An Extended Version

This paper presents HEALER, a software agent that recommends sequential intervention plans for use by homeless shelters, who organize these interventions to raise awareness about HIV among homeless youth. HEALER's sequential plans (built using knowledge of social networks of homeless youth) choose intervention participants strategically to maximize influence spread, while reasoning about uncertainties in the network. While previous work presents influence maximizing techniques to choose intervention participants, they do not address three real-world issues: (i) they completely fail to scale up to real-world sizes; (ii) they do not handle deviations in execution of intervention plans; (iii) constructing real-world social networks is an expensive process. HEALER handles these issues via four major contributions: (i) HEALER casts this influence maximization problem as a POMDP and solves it using a novel planner which scales up to previously unsolvable real-world sizes; (ii) HEALER allows shelter officials to modify its recommendations, and updates its future plans in a deviation-tolerant manner; (iii) HEALER constructs social networks of homeless youth at low cost, using a Facebook application. Finally, (iv) we show hardness results for the problem that HEALER solves. HEALER will be deployed in the real world in early Spring 2016 and is currently undergoing testing at a homeless shelter.Comment: This is an extended version of our AAMAS 2016 paper (with the same name) with full proofs of all our theorems include

Cyber-Physical Modeling and Control of Crowd of Pedestrians: A Review and New Framework

Recent advances in modeling and control of crowds of pedestrians are briefly surveyed in this paper. Possibilities of applying fractional calculus in the modeling of crowd of pedestrians have been shortly reviewed and discussed from different aspects such as descriptions of motion, interactions of long range and effects of memory. Control of the crowd of pedestrians have also been formulated using the framework of Cyber-Physical Systems and been realized using networked Segways with onboard emergency response personnels to regulate the velocity and flux of the crowd. Platform for verification of the theoretical results are also provided in this paper.Comment: 16 pages, 3 figure

Lyapunov Approach to Consensus Problems

This paper investigates the weighted-averaging dynamic for unconstrained and constrained consensus problems. Through the use of a suitably defined adjoint dynamic, quadratic Lyapunov comparison functions are constructed to analyze the behavior of weighted-averaging dynamic. As a result, new convergence rate results are obtained that capture the graph structure in a novel way. In particular, the exponential convergence rate is established for unconstrained consensus with the exponent of the order of $1-O(1/(m\log_2m))$. Also, the exponential convergence rate is established for constrained consensus, which extends the existing results limited to the use of doubly stochastic weight matrices

Online Distributed Optimization on Dynamic Networks

This paper presents a distributed optimization scheme over a network of agents in the presence of cost uncertainties and over switching communication topologies. Inspired by recent advances in distributed convex optimization, we propose a distributed algorithm based on a dual sub-gradient averaging. The objective of this algorithm is to minimize a cost function cooperatively. Furthermore, the algorithm changes the weights on the communication links in the network to adapt to varying reliability of neighboring agents. A convergence rate analysis as a function of the underlying network topology is then presented, followed by simulation results for representative classes of sensor networks.Comment: Submitted to The IEEE Transactions on Automatic Control, 201

Wide-Area Time-Synchronized Closed-Loop Control of Power Systems And Decentralized Active Distribution Networks

The rapidly expanding power system grid infrastructure and the need to reduce the occurrence of major blackouts and prevention or hardening of systems against cyber-attacks, have led to increased interest in the improved resilience of the electrical grid. Distributed and decentralized control have been widely applied to computer science research. However, for power system applications, the real-time application of decentralized and distributed control algorithms introduce several challenges. In this dissertation, new algorithms and methods for decentralized control, protection and energy management of Wide Area Monitoring, Protection and Control (WAMPAC) and the Active Distribution Network (ADN) are developed to improve the resiliency of the power system. To evaluate the findings of this dissertation, a laboratory-scale integrated Wide WAMPAC and ADN control platform was designed and implemented. The developed platform consists of phasor measurement units (PMU), intelligent electronic devices (IED) and programmable logic controllers (PLC). On top of the designed hardware control platform, a multi-agent cyber-physical interoperability viii framework was developed for real-time verification of the developed decentralized and distributed algorithms using local wireless and Internet-based cloud communication. A novel real-time multiagent system interoperability testbed was developed to enable utility independent private microgrids standardized interoperability framework and define behavioral models for expandability and plug-and-play operation. The state-of-theart power system multiagent framework is improved by providing specific attributes and a deliberative behavior modeling capability. The proposed multi-agent framework is validated in a laboratory based testbed involving developed intelligent electronic device prototypes and actual microgrid setups. Experimental results are demonstrated for both decentralized and distributed control approaches. A new adaptive real-time protection and remedial action scheme (RAS) method using agent-based distributed communication was developed for autonomous hybrid AC/DC microgrids to increase resiliency and continuous operability after fault conditions. Unlike the conventional consecutive time delay-based overcurrent protection schemes, the developed technique defines a selectivity mechanism considering the RAS of the microgrid after fault instant based on feeder characteristics and the location of the IEDs. The experimental results showed a significant improvement in terms of resiliency of microgrids through protection using agent-based distributed communication