Learning to Communicate: A Machine Learning Framework for Heterogeneous Multi-Agent Robotic Systems

We present a machine learning framework for multi-agent systems to learn both the optimal policy for maximizing the rewards and the encoding of the high dimensional visual observation. The encoding is useful for sharing local visual observations with other agents under communication resource constraints. The actor-encoder encodes the raw images and chooses an action based on local observations and messages sent by the other agents. The machine learning agent generates not only an actuator command to the physical device, but also a communication message to the other agents. We formulate a reinforcement learning problem, which extends the action space to consider the communication action as well. The feasibility of the reinforcement learning framework is demonstrated using a 3D simulation environment with two collaborating agents. The environment provides realistic visual observations to be used and shared between the two agents.Comment: AIAA SciTech 201

Nothing can compare with a population, besides agents

15 pagesLeveraging the resemblances between two areas explored so far independently enables to provide a theoretical framework for dis- tributed systems where global behaviors emerge from a set of local in- teractions. The contribution of this paper arise from the observation that population protocols and multi-agent systems (MAS) bear many resemblances. Particularly, some subclasses of MAS seem to fit the same computational power than population protocols. Population protocols provide theoretical foundations for mobile tiny device networks. On the other hand, from long-standing research study in distributed artificial in- telligence, MAS forms an interesting model for society and owns a broad spectrum of application field, from simple reactive system to social sci- ences. Linking the both model should offers several extremely interesting outcomes

Information Propagation Algorithms for Consensus Formation in Decentralized Multi-Agent Systems

Consensus occurs within a multi-agent system when every agent is in agreement about the value of some particular state. For example, the color of an LED, the position or magnitude of a vector, a rendezvous location, the most recent state of data within a database, or the identity of a leader are all states that agents might need to agree on in order to execute their tasking. The task of the decentralized consensus problem for multi-agent systems is to design an algorithm that enables agents to communicate and exchange information such that, in finite time, agents are able to form a consensus without the use of a centralized control mechanism. The primary goal of this research is to introduce and provide supporting evidence for Stochastic Local Observation/Gossip (SLOG) algorithms as a new class of solutions to the decentralized consensus problem for multi-agent systems that lack a centralized controller, with the additional constraints that agents act asynchronously, information is discrete, and all consensus options are equally preferable to all agents. Examples of where these constraints might apply include the spread of social norms and conventions in artificial populations, rendezvous among a set of specific locations, and task assignment. This goal is achieved through a combination of theory and experimentation. Information propagation process and an information propagation algorithm are derived by unifying the general structure of multiple existing solutions to the decentralized consensus problem. They are then used to define two classes of algorithms that spread information across a network and solve the decentralized consensus problem: buffered gossip algorithms and local observation algorithms. Buffered gossip algorithms generalize the behavior of many push-based solutions to the decentralized consensus problem. Local observation algorithms generalize the behavior of many pull-based solutions to the decentralized consensus problem. In the language of object oriented design, buffered gossip algorithms and local observation algorithms are abstract classes; information propagation processes are interfaces. SLOG algorithms combine the transmission mechanisms of buffered gossip algorithms and local observation algorithms into a single hybrid algorithm that is able to push and pull information within the local neighborhood. A common mathematical framework is constructed and used to determine the conditions under which each of these algorithms are guaranteed to produce a consensus, and thus solve the decentralized consensus problem. Finally, a series of simulation experiments are conducted to study the performance of SLOG algorithms. These experiments compare the average speed of consensus formation between buffered gossip algorithms, local observation algorithms, and SLOG algorithms over four distinct network topologies. Beyond the introduction of the SLOG algorithm, this research also contributes to the existing literature on the decentralized consensus problem by: specifying a theoretical framework that can be used to explore the consensus behavior of push-based and pull-based information propagation algorithms; using this framework to define buffered gossip algorithms and local observation algorithms as generalizations for existing solutions to the decentralized consensus problem; highlighting the similarities between consensus algorithms within control theory and opinion dynamics within computational sociology, and showing how these research areas can be successfully combined to create new and powerful algorithms; and providing an empirical comparison between multiple information propagation algorithms

Learning probabilistic interaction models

We live in a multi-modal world; therefore it comes as no surprise that the human brain is tailored for the integration of multi-sensory input. Inspired by the human brain, the multi-sensory data is used in Artificial Intelligence (AI) for teaching different concepts to computers. Autonomous Agents (AAs) are AI systems that sense and act autonomously in complex dynamic environments. Such agents can build up Self-Awareness (SA) by describing their experiences through multi-sensorial information with appropriate models and correlating them incrementally with the currently perceived situation to continuously expand their knowledge. This thesis proposes methods to learn such awareness models for AAs. These models include SA and situational awareness models in order to perceive and understand itself (self variables) and its surrounding environment (external variables) at the same time. An agent is considered self-aware when it can dynamically observe and understand itself and its surrounding through different proprioceptive and exteroceptive sensors which facilitate learning and maintaining a contextual representation by processing the observed multi-sensorial data. We proposed a probabilistic framework for generative and descriptive dynamic models that can lead to a computationally efficient SA system. In general, generative models facilitate the prediction of future states while descriptive models enable to select the representation that best fits the current observation. The proposed framework employs a Probabilistic Graphical Models (PGMs) such as Dynamic Bayesian Networks (DBNs) that represent a set of variables and their conditional dependencies. Once we obtain this probabilistic representation, the latter allows the agent to model interactions between itself, as observed through proprioceptive sensors, and the environment, as observed through exteroceptive sensors. In order to develop an awareness system, not only an agent needs to recognize the normal states and perform predictions accordingly, but also it is necessary to detect the abnormal states with respect to its previously learned knowledge. Therefore, there is a need to measure anomalies or irregularities in an observed situation. In this case, the agent should be aware that an abnormality (i.e., a non-stationary condition) never experienced before, is currently present. Due to our specific way of representation, which makes it possible to model multi-sensorial data into a uniform interaction model, the proposed work not only improves predictions of future events but also can be potentially used to effectuate a transfer learning process where information related to the learned model can be moved and interpreted by another body

Distributed agents for autonomous spacecraft

Space missions have evolved considerably in the last fifty years in both complexity and ambition. In order to enable this continued improvement in the scientific and commercial return of space missions new control systems are needed that can manage complex combinations of state of the art hardware with a minimum of human interaction. Distributed multi-agent systems are one approach to controlling complex multisatellite space missions. A distributed system is not enough on its own however,the spacecraft must be able to carry out complex tasks such as planning,negotiation and close proximity formation flying autonomously. It is the coupling of distributed control with autonomy that is the focus of this thesis. Three contributions to the state of the art are described herein. They all involve the innovative use of multi-agent systems in space missions. The first is the development of a multi-agent architecture, HASA, specifically for space missions. The second is to use embedded agents to autonomously control an interferometric type space telescope. The third is based on software agents that coordinate multiple Earth observation missions coupled with a global optimisation technique for data extraction. The HASA architecture was developed in reaction to the over generality of most multi-agent architectures in the computer science and robotics literature and the ad-hoc, case-by-case approach, to multi-agent architectures when developed and deployed for space missions. The HASA architecture has a recursive nature which allows for the multi-agent system to be completely described throughout its development process as the design evolves and more sub-systems are implemented. It also inherits a focus on the robust generation of a product and safe operation from architectures in use in the manufacturing industry. A multi-agent system was designed using the HASA architecture for an interferometric space telescope type mission. This type of mission puts high requirements on formation flying and cooperation between agents. The formation flying agents were then implemented using a Java framework and tested on a multi-platform distributed simulation suite developed especially for this thesis. Three different control methods were incorporated into the agents and the multi-agent system was shown to be able to acquire and change formation and avoid collisions autonomously. A second multi-agent system was designed for the GMES mission in collaboration with GMV, the industrial partner in this project. This basic MAS design was transferred to the HASA architecture. A novel image selection algorithm was developed to work alongside the GMES multi-agent system. This algorithm uses global optimisation techniques to suggest image parameters to users based on the output of the multi-agent system

Task-Effective Compression of Observations for the Centralized Control of a Multi-agent System Over Bit-Budgeted Channels

peer reviewedWe consider a task-effective quantization problem that arises when multiple agents are controlled via a centralized controller (CC). While agents have to communicate their observations to the CC for decision-making, the bit-budgeted communications of agent-CC links may limit the task-effectiveness of the system which is measured by the system’s average sum of stage costs/rewards. As a result, each agent should compress/quantize its observation such that the average sum of stage costs/rewards of the control task is minimally impacted. We address the problem of maximizing the average sum of stage rewards by proposing two different Action-Based State Aggregation (ABSA) algorithms that carry out the indirect and joint design of control and communication policies in the multi-agent system. While the applicability of ABSA-1 is limited to single-agent systems, it provides an analytical framework that acts as a stepping stone to the design of ABSA-2. ABSA-2 carries out the joint design of control and communication for a multi-agent system. We evaluate the algorithms -with average return as the performance metric -using numerical experiments performed to solve a multi-agent geometric consensus problem. The numerical results are concluded by introducing a new metric that measures the effectiveness of communications in a multi-agent system.U-AGR-7288 - C22/IS/17220888/RUTINE (01/09/2023 - 31/08/2026) - VU Thang Xua