Identification of Hessian matrix in distributed gradient-based multi-agent coordination control systems

Multi-agent coordination control usually involves a potential function that encodes information of a global control task, while the control input for individual agents is often designed by a gradient-based control law. The property of Hessian matrix associated with a potential function plays an important role in the stability analysis of equilibrium points in gradient-based coordination control systems. Therefore, the identification of Hessian matrix in gradient-based multi-agent coordination systems becomes a key step in multi-agent equilibrium analysis. However, very often the identification of Hessian matrix via the entry-wise calculation is a very tedious task and can easily introduce calculation errors. In this paper we present some general and fast approaches for the identification of Hessian matrix based on matrix differentials and calculus rules, which can easily derive a compact form of Hessian matrix for multi-agent coordination systems. We also present several examples on Hessian identification for certain typical potential functions involving edge-tension distance functions and triangular-area functions, and illustrate their applications in the context of distributed coordination and formation control

Route Swarm: Wireless Network Optimization through Mobility

In this paper, we demonstrate a novel hybrid architecture for coordinating networked robots in sensing and information routing applications. The proposed INformation and Sensing driven PhysIcally REconfigurable robotic network (INSPIRE), consists of a Physical Control Plane (PCP) which commands agent position, and an Information Control Plane (ICP) which regulates information flow towards communication/sensing objectives. We describe an instantiation where a mobile robotic network is dynamically reconfigured to ensure high quality routes between static wireless nodes, which act as source/destination pairs for information flow. The ICP commands the robots towards evenly distributed inter-flow allocations, with intra-flow configurations that maximize route quality. The PCP then guides the robots via potential-based control to reconfigure according to ICP commands. This formulation, deemed Route Swarm, decouples information flow and physical control, generating a feedback between routing and sensing needs and robotic configuration. We demonstrate our propositions through simulation under a realistic wireless network regime.Comment: 9 pages, 4 figures, submitted to the IEEE International Conference on Intelligent Robots and Systems (IROS) 201

Counterfactual Multi-Agent Policy Gradients

Cooperative multi-agent systems can be naturally used to model many real world problems, such as network packet routing and the coordination of autonomous vehicles. There is a great need for new reinforcement learning methods that can efficiently learn decentralised policies for such systems. To this end, we propose a new multi-agent actor-critic method called counterfactual multi-agent (COMA) policy gradients. COMA uses a centralised critic to estimate the Q-function and decentralised actors to optimise the agents' policies. In addition, to address the challenges of multi-agent credit assignment, it uses a counterfactual baseline that marginalises out a single agent's action, while keeping the other agents' actions fixed. COMA also uses a critic representation that allows the counterfactual baseline to be computed efficiently in a single forward pass. We evaluate COMA in the testbed of StarCraft unit micromanagement, using a decentralised variant with significant partial observability. COMA significantly improves average performance over other multi-agent actor-critic methods in this setting, and the best performing agents are competitive with state-of-the-art centralised controllers that get access to the full state