Deep Reinforcement Learning for Swarm Systems

Recently, deep reinforcement learning (RL) methods have been applied successfully to multi-agent scenarios. Typically, these methods rely on a concatenation of agent states to represent the information content required for decentralized decision making. However, concatenation scales poorly to swarm systems with a large number of homogeneous agents as it does not exploit the fundamental properties inherent to these systems: (i) the agents in the swarm are interchangeable and (ii) the exact number of agents in the swarm is irrelevant. Therefore, we propose a new state representation for deep multi-agent RL based on mean embeddings of distributions. We treat the agents as samples of a distribution and use the empirical mean embedding as input for a decentralized policy. We define different feature spaces of the mean embedding using histograms, radial basis functions and a neural network learned end-to-end. We evaluate the representation on two well known problems from the swarm literature (rendezvous and pursuit evasion), in a globally and locally observable setup. For the local setup we furthermore introduce simple communication protocols. Of all approaches, the mean embedding representation using neural network features enables the richest information exchange between neighboring agents facilitating the development of more complex collective strategies.Comment: 31 pages, 12 figures, version 3 (published in JMLR Volume 20

A principled information valuation for communications during multi-agent coordination

Decentralised coordination in multi-agent systems is typically achieved using communication. However, in many cases, communication is expensive to utilise because there is limited bandwidth, it may be dangerous to communicate, or communication may simply be unavailable at times. In this context, we argue for a rational approach to communication --- if it has a cost, the agents should be able to calculate a value of communicating. By doing this, the agents can balance the need to communicate with the cost of doing so. In this research, we present a novel model of rational communication that uses information theory to value communications, and employ this valuation in a decision theoretic coordination mechanism. A preliminary empirical evaluation of the benefits of this approach is presented in the context of the RoboCupRescue simulator

Multiagent autonomous energy management

The objective of this thesis is to design distributed software agents for reliable operation of integrated electric power systems of modern electric warships. The automatic reconfiguration of electric shipboard power systems is an important step toward improved fight-through and self-healing capabilities of naval warships. The improvements are conceptualized by redesigning the electric power system and its controls. This research focuses on a new scheme for an energy management system in the form of distributed control/software agents. Multiagent systems provide an ideal level of abstraction for modeling complex applications where distributed and heterogeneous entities need to cooperate to achieve a common goal. The agents\u27 task is to ensure supply of the various load demands while taking into consideration system constraints and load and supply path priorities. A self-stabilizing maximum flow algorithm is investigated to allow implementation of the agents\u27 strategies and find a global solution by only considering local information and a minimum amount of communication. (Abstract shortened by UMI.)

Asynchronous Communication under Reliable and Unreliable Network Topologies in Distributed Multiagent Systems: A Robust Technique for Computing Average Consensus

Nearly all applications in multiagent systems demand precision, robustness, consistency, and rapid convergence in designing of distributed consensus algorithms. Keeping this thing in our sight, this research suggests a robust consensus protocol for distributed multiagent networks, continuing asynchronous communications, where agent’s states values are updated at diverse interval of time. This paper presents an asynchronous communication for both reliable and unreliable network topologies. The primary goal is to delineate local control inputs to attain time synchronization by processing the update information received by the agents associated in a communication topology. Additionally in order to accomplish the robust convergence, modelling of convergence analysis is conceded by commissioning the basic principles of graph and matrix theory alongside the suitable lemmas. Moreover, statistical examples presenting four diverse scenarios are provided in the end; produced results are the recognisable indicator to authenticate the robust effectiveness of the proposed algorithm. Likewise, a simulation comparison of the projected algorithm with the other existing approaches is conducted, considering different performance parameters are being carried out to support our claim

Reallocation Problems in Agent Societies: A Local Mechanism to Maximize Social Welfare

Resource reallocation problems are common in real life and therefore gain an increasing interest in Computer Science and Economics. Such problems consider agents living in a society and negotiating their resources with each other in order to improve the welfare of the population. In many studies however, the unrealistic context considered, where agents have a flawless knowledge and unlimited interaction abilities, impedes the application of these techniques in real life problematics. In this paper, we study how agents should behave in order to maximize the welfare of the society. We propose a multi-agent method based on autonomous agents endowed with a local knowledge and local interactions. Our approach features a more realistic environment based on social networks, inside which we provide the behavior for the agents and the negotiation settings required for them to lead the negotiation processes towards socially optimal allocations. We prove that bilateral transactions of restricted cardinality are sufficient in practice to converge towards an optimal solution for different social objectives. An experimental study supports our claims and highlights the impact of a realistic environment on the efficiency of the techniques utilized.Resource Allocation, Negotiation, Social Welfare, Agent Society, Behavior, Emergence

Anytime Coalition Structure Generation with Worst Case Guarantees

Coalition formation is a key topic in multiagent systems. One would prefer a coalition structure that maximizes the sum of the values of the coalitions, but often the number of coalition structures is too large to allow exhaustive search for the optimal one. But then, can the coalition structure found via a partial search be guaranteed to be within a bound from optimum? We show that none of the previous coalition structure generation algorithms can establish any bound because they search fewer nodes than a threshold that we show necessary for establishing a bound. We present an algorithm that establishes a tight bound within this minimal amount of search, and show that any other algorithm would have to search strictly more. The fraction of nodes needed to be searched approaches zero as the number of agents grows. If additional time remains, our anytime algorithm searches further, and establishes a progressively lower tight bound. Surprisingly, just searching one more node drops the bound in half. As desired, our algorithm lowers the bound rapidly early on, and exhibits diminishing returns to computation. It also drastically outperforms its obvious contenders. Finally, we show how to distribute the desired search across self-interested manipulative agents