583 research outputs found
Multi-agent decision-making dynamics inspired by honeybees
When choosing between candidate nest sites, a honeybee swarm reliably chooses
the most valuable site and even when faced with the choice between near-equal
value sites, it makes highly efficient decisions. Value-sensitive
decision-making is enabled by a distributed social effort among the honeybees,
and it leads to decision-making dynamics of the swarm that are remarkably
robust to perturbation and adaptive to change. To explore and generalize these
features to other networks, we design distributed multi-agent network dynamics
that exhibit a pitchfork bifurcation, ubiquitous in biological models of
decision-making. Using tools of nonlinear dynamics we show how the designed
agent-based dynamics recover the high performing value-sensitive
decision-making of the honeybees and rigorously connect investigation of
mechanisms of animal group decision-making to systematic, bio-inspired control
of multi-agent network systems. We further present a distributed adaptive
bifurcation control law and prove how it enhances the network decision-making
performance beyond that observed in swarms
Masked Pretraining for Multi-Agent Decision Making
Building a single generalist agent with zero-shot capability has recently
sparked significant advancements in decision-making. However, extending this
capability to multi-agent scenarios presents challenges. Most current works
struggle with zero-shot capabilities, due to two challenges particular to the
multi-agent settings: a mismatch between centralized pretraining and
decentralized execution, and varying agent numbers and action spaces, making it
difficult to create generalizable representations across diverse downstream
tasks. To overcome these challenges, we propose a \textbf{Mask}ed pretraining
framework for \textbf{M}ulti-\textbf{a}gent decision making (MaskMA). This
model, based on transformer architecture, employs a mask-based collaborative
learning strategy suited for decentralized execution with partial observation.
Moreover, MaskMA integrates a generalizable action representation by dividing
the action space into actions toward self-information and actions related to
other entities. This flexibility allows MaskMA to tackle tasks with varying
agent numbers and thus different action spaces. Extensive experiments in SMAC
reveal MaskMA, with a single model pretrained on 11 training maps, can achieve
an impressive 77.8% zero-shot win rate on 60 unseen test maps by decentralized
execution, while also performing effectively on other types of downstream tasks
(\textit{e.g.,} varied policies collaboration and ad hoc team play).Comment: 17 page
A hybrid approach to multi-agent decision-making
In the aftermath of a large-scale disaster, agents' decisions derive from self-interested (e.g. survival), common-good (e.g. victims' rescue) and teamwork (e.g. fire extinction) motivations. However, current decision-theoretic models are either purely individual or purely collective and find it difficult to deal with motivational attitudes; on the other hand, mental-state based models find it difficult to deal with uncertainty. We propose a hybrid, CvI-JI, approach that combines: i) collective 'versus' individual (CvI) decisions, founded on the Markov decision process (MDP) quantitative evaluation of joint-actions, and ii)joint-intentions (JI) formulation of teamwork, founded on the belief-desire-intention (BDI) architecture of general mental-state based reasoning. The CvI-JI evaluation explores the performance's improvemen
A learning-based approach to multi-agent decision-making
We propose a learning-based methodology to reconstruct private information
held by a population of interacting agents in order to predict an exact outcome
of the underlying multi-agent interaction process, here identified as a
stationary action profile. We envision a scenario where an external observer,
endowed with a learning procedure, is allowed to make queries and observe the
agents' reactions through private action-reaction mappings, whose collective
fixed point corresponds to a stationary profile. By adopting a smart query
process to iteratively collect sensible data and update parametric estimates,
we establish sufficient conditions to assess the asymptotic properties of the
proposed learning-based methodology so that, if convergence happens, it can
only be towards a stationary action profile. This fact yields two main
consequences: i) learning locally-exact surrogates of the action-reaction
mappings allows the external observer to succeed in its prediction task, and
ii) working with assumptions so general that a stationary profile is not even
guaranteed to exist, the established sufficient conditions hence act also as
certificates for the existence of such a desirable profile. Extensive numerical
simulations involving typical competitive multi-agent control and decision
making problems illustrate the practical effectiveness of the proposed
learning-based approach
Models of multi-agent decision making
In this thesis we formalise and study computational aspects of group decision making for rational, self-interested agents. Specifically, we are interested in systems where agents reach consensus according to endogenous thresholds. Natural groups have been shown to make collective decisions according to threshold-mediated behaviours. An individual will commit to some collective endeavour only if the number of others having already committed exceeds their threshold. Consensus is reached only where all individuals express commitment. We present a family of models that describe fundamental aspects of cooperative behaviour in multi-agent systems. These include: coalition formation, participation in joint actions and the achievement of individuals’ goals over time. We associate novel solution concepts with our models and present results concerning the computational complexity of several natural decision problems arising from these. We demonstrate potential applications of our work by modelling a group decision problem common to many cohesive groups: establishing the location of the group. Using model checking tools we compute the effects of agents’ thresholds upon outcomes. We consider our results within an appropriate research context
Models of multi-agent decision making
In this thesis we formalise and study computational aspects of group decision making for rational, self-interested agents. Specifically, we are interested in systems where agents reach consensus according to endogenous thresholds. Natural groups have been shown to make collective decisions according to threshold-mediated behaviours. An individual will commit to some collective endeavour only if the number of others having already committed exceeds their threshold. Consensus is reached only where all individuals express commitment. We present a family of models that describe fundamental aspects of cooperative behaviour in multi-agent systems. These include: coalition formation, participation in joint actions and the achievement of individuals’ goals over time. We associate novel solution concepts with our models and present results concerning the computational complexity of several natural decision problems arising from these. We demonstrate potential applications of our work by modelling a group decision problem common to many cohesive groups: establishing the location of the group. Using model checking tools we compute the effects of agents’ thresholds upon outcomes. We consider our results within an appropriate research context
Data-Driven Predictive Control for Multi-Agent Decision Making With Chance Constraints
In the recent literature, significant and substantial efforts have been
dedicated to the important area of multi-agent decision-making problems.
Particularly here, the model predictive control (MPC) methodology has
demonstrated its effectiveness in various applications, such as mobile robots,
unmanned vehicles, and drones. Nevertheless, in many specific scenarios
involving the MPC methodology, accurate and effective system identification is
a commonly encountered challenge. As a consequence, the overall system
performance could be significantly weakened in outcome when the traditional MPC
algorithm is adopted under such circumstances. To cater to this rather major
shortcoming, this paper investigates an alternate data-driven approach to solve
the multi-agent decision-making problem. Utilizing an innovative modified
methodology with suitable closed-loop input/output measurements that comply
with the appropriate persistency of excitation condition, a non-parametric
predictive model is suitably constructed. This non-parametric predictive model
approach in the work here attains the key advantage of alleviating the rather
heavy computational burden encountered in the optimization procedures typical
in alternative methodologies requiring open-loop input/output measurement data
collection and parametric system identification. Then with a conservative
approximation of probabilistic chance constraints for the MPC problem, a
resulting deterministic optimization problem is formulated and solved
efficiently and effectively. In the work here, this intuitive data-driven
approach is also shown to preserve good robustness properties. Finally, a
multi-drone system is used to demonstrate the practical appeal and highly
effective outcome of this promising development in achieving very good system
performance.Comment: 10 pages, 6 figure
Graph Reinforcement Learning Application to Co-operative Decision-Making in Mixed Autonomy Traffic: Framework, Survey, and Challenges
Proper functioning of connected and automated vehicles (CAVs) is crucial for
the safety and efficiency of future intelligent transport systems. Meanwhile,
transitioning to fully autonomous driving requires a long period of mixed
autonomy traffic, including both CAVs and human-driven vehicles. Thus,
collaboration decision-making for CAVs is essential to generate appropriate
driving behaviors to enhance the safety and efficiency of mixed autonomy
traffic. In recent years, deep reinforcement learning (DRL) has been widely
used in solving decision-making problems. However, the existing DRL-based
methods have been mainly focused on solving the decision-making of a single
CAV. Using the existing DRL-based methods in mixed autonomy traffic cannot
accurately represent the mutual effects of vehicles and model dynamic traffic
environments. To address these shortcomings, this article proposes a graph
reinforcement learning (GRL) approach for multi-agent decision-making of CAVs
in mixed autonomy traffic. First, a generic and modular GRL framework is
designed. Then, a systematic review of DRL and GRL methods is presented,
focusing on the problems addressed in recent research. Moreover, a comparative
study on different GRL methods is further proposed based on the designed
framework to verify the effectiveness of GRL methods. Results show that the GRL
methods can well optimize the performance of multi-agent decision-making for
CAVs in mixed autonomy traffic compared to the DRL methods. Finally, challenges
and future research directions are summarized. This study can provide a
valuable research reference for solving the multi-agent decision-making
problems of CAVs in mixed autonomy traffic and can promote the implementation
of GRL-based methods into intelligent transportation systems. The source code
of our work can be found at https://github.com/Jacklinkk/Graph_CAVs.Comment: 22 pages, 7 figures, 10 tables. Currently under review at IEEE
Transactions on Intelligent Transportation System
- …