A Hierarchical Game-Theoretic Decision-Making for Cooperative Multi-Agent Systems Under the Presence of Adversarial Agents

Underlying relationships among Multi-Agent Systems (MAS) in hazardous scenarios can be represented as Game-theoretic models. This paper proposes a new hierarchical network-based model called Game-theoretic Utility Tree (GUT), which decomposes high-level strategies into executable low-level actions for cooperative MAS decisions. It combines with a new payoff measure based on agent needs for real-time strategy games. We present an Explore game domain, where we measure the performance of MAS achieving tasks from the perspective of balancing the success probability and system costs. We evaluate the GUT approach against state-of-the-art methods that greedily rely on rewards of the composite actions. Conclusive results on extensive numerical simulations indicate that GUT can organize more complex relationships among MAS cooperation, helping the group achieve challenging tasks with lower costs and higher winning rates. Furthermore, we demonstrated the applicability of the GUT using the simulator-hardware testbed - Robotarium. The performances verified the effectiveness of the GUT in the real robot application and validated that the GUT could effectively organize MAS cooperation strategies, helping the group with fewer advantages achieve higher performance.Comment: This paper is accepted by the ACM Symposium on Applied Computing (SAC) 2023 Technical Track on Intelligent Robotics and Multi-Agent Systems (IRMAS

Situation based strategic positioning for coordinating a team of homogeneous agents

. In this paper we are proposing an approach for coordinating a team ofhomogeneous agents based on a flexible common Team Strategy as well as onthe concepts of Situation Based Strategic Positioning and Dynamic Positioningand Role Exchange. We also introduce an Agent Architecture including a specifichigh-level decision module capable of implementing this strategy. Ourproposal is based on the formalization of what is a team strategy for competingwith an opponent team having opposite goals. A team strategy is composed of aset of agent types and a set of tactics, which are also composed of several formations.Formations are used for different situations and assign each agent a defaultspatial positioning and an agent type (defining its behaviour at several levels).Agents reactivity is also introduced for appropriate response to the dynamicsof the current situation. However, in our approach this is done in a way thatpreserves team coherence instead of permitting uncoordinated agent behaviour.We have applied, with success, this coordination approach to the RoboSoccersimulated domain. The FC Portugal team, developed using this approach wonthe RoboCup2000 (simulation league) European and World championshipsscoring a total of 180 goals and conceding none

Towards Informed Exploration for Deep Reinforcement Learning

In this thesis, we discuss various techniques for improving exploration for deep reinforcement learning. We begin with a brief review of reinforcement learning (RL) and the fundamental v.s. exploitation trade-off. Then we review how deep RL has improved upon classical and summarize six categories of the latest exploration methods for deep RL, in the order increasing usage of prior information. We then explore representative works in three categories discuss their strengths and weaknesses. The first category, represented by Soft Q-learning, uses regularization to encourage exploration. The second category, represented by count-based via hashing, maps states to hash codes for counting and assigns higher exploration to less-encountered states. The third category utilizes hierarchy and is represented by modular architecture for RL agents to play StarCraft II. Finally, we conclude that exploration by prior knowledge is a promising research direction and suggest topics of potentially impact

Subsumption architecture for enabling strategic coordination of robot swarms in a gaming scenario

The field of swarm robotics breaks away from traditional research by maximizing the performance of a group - swarm - of limited robots instead of optimizing the intelligence of a single robot. Similar to current-generation strategy video games, the player controls groups of units - squads - instead of the individual participants. These individuals are rather unintelligent robots, capable of little more than navigating and using their weapons. However, clever control of the squads of autonomous robots by the game players can make for intense, strategic matches. The gaming framework presented in this article provides players with strategic coordination of robot squads. The developed swarm intelligence techniques break up complex squad commands into several commands for each robot using robot formations and path finding while avoiding obstacles. These algorithms are validated through a 'Capture the Flag' gaming scenario where a complex squad command is split up into several robot commands in a matter of milliseconds

A review on multi-robot systems categorised by application domain

Literature reviews on Multi-Robot Systems (MRS) typically focus on fundamental technical aspects, like coordination and communication, that need to be considered in order to coordinate a team of robots to perform a given task effectively and efficiently. Other reviews only consider works that aim to address a specific problem or one particular application of MRS. In contrast, this paper presents a survey of recent research works on MRS and categorises them according to their application domain. Furthermore, this paper compiles a number of seminal review works that have proposed specific taxonomies in classifying fundamental concepts, such as coordination, architecture and communication, in the field of MRS.peer-reviewe

Exploiting opponent behavior in multi-agent systems

Tese de mestrado integrado. Engenharia Informática e Computação. Faculdade de Engenharia. Universidade do Porto. 201

BEHAVIORAL COMPOSITION FOR HETEROGENEOUS SWARMS

Research into swarm robotics has produced a robust library of swarm behaviors that excel at defined tasks such as flocking and area search, many of which have potential for application to a wide range of military problems. However, to be successfully applied to an operational environment, swarms must be flexible enough to achieve a wide array of specific objectives and usable enough to be configured and employed by lay operators. This research explored the use of the Mission-based Architecture for Swarm Composability (MASC) to develop mission-specific tactics as compositions of more general, reusable plays for use with the Advanced Robotic Systems Engineering Laboratory (ARSENL) swarm system. Three tactics were developed to conduct autonomous search of a geographic area and investigation of generated contacts of interest. The tactics were tested in live-flight and virtual environment experiments and compared to a preexisting monolithic behavior implementation completing the same task. Measures of performance were defined and observed that verified the effectiveness of solutions and confirmed the advantages that composition provides with respect to reusability and rapid development of increasingly complex behaviors.Lieutenant Commander, United States NavyApproved for public release. Distribution is unlimited