18,801 research outputs found
A study on the ephemeral nature of knowledge shared within multiagent systems
Achieving knowledge sharing within an artificial swarm system could lead to
significant development in autonomous multiagent and robotic systems research
and realize collective intelligence. However, this is difficult to achieve
since there is no generic framework to transfer skills between agents other
than a query-response-based approach. Moreover, natural living systems have a
"forgetfulness" property for everything they learn. Analyzing such ephemeral
nature (temporal memory properties of new knowledge gained) in artificial
systems has never been studied in the literature. We propose a behavior
tree-based framework to realize a query-response mechanism for transferring
skills encoded as the condition-action control sub-flow of that portion of the
knowledge between agents to fill this gap. We simulate a multiagent group with
different initial knowledge on a foraging mission. While performing basic
operations, each robot queries other robots to respond to an unknown condition.
The responding robot shares the control actions by sharing a portion of the
behavior tree that addresses the queries. Specifically, we investigate the
ephemeral nature of the new knowledge gained through such a framework, where
the knowledge gained by the agent is either limited due to memory or is
forgotten over time. Our investigations show that knowledge grows
proportionally with the duration of remembrance, which is trivial. However, we
found minimal impact on knowledge growth due to memory. We compare these cases
against a baseline that involved full knowledge pre-coded on all agents. We
found that knowledge-sharing strived to match the baseline condition by sharing
and achieving knowledge growth as a collective system.Comment: In Proceedings of the Fifth International Symposium on Swarm Behavior
and Bio-Inspired Robotics 2022 (SWARM 5th 2022
Coupled Replicator Equations for the Dynamics of Learning in Multiagent Systems
Starting with a group of reinforcement-learning agents we derive coupled
replicator equations that describe the dynamics of collective learning in
multiagent systems. We show that, although agents model their environment in a
self-interested way without sharing knowledge, a game dynamics emerges
naturally through environment-mediated interactions. An application to
rock-scissors-paper game interactions shows that the collective learning
dynamics exhibits a diversity of competitive and cooperative behaviors. These
include quasiperiodicity, stable limit cycles, intermittency, and deterministic
chaos--behaviors that should be expected in heterogeneous multiagent systems
described by the general replicator equations we derive.Comment: 4 pages, 3 figures,
http://www.santafe.edu/projects/CompMech/papers/credlmas.html; updated
references, corrected typos, changed conten
Scale-free memory model for multiagent reinforcement learning. Mean field approximation and rock-paper-scissors dynamics
A continuous time model for multiagent systems governed by reinforcement
learning with scale-free memory is developed. The agents are assumed to act
independently of one another in optimizing their choice of possible actions via
trial-and-error search. To gain awareness about the action value the agents
accumulate in their memory the rewards obtained from taking a specific action
at each moment of time. The contribution of the rewards in the past to the
agent current perception of action value is described by an integral operator
with a power-law kernel. Finally a fractional differential equation governing
the system dynamics is obtained. The agents are considered to interact with one
another implicitly via the reward of one agent depending on the choice of the
other agents. The pairwise interaction model is adopted to describe this
effect. As a specific example of systems with non-transitive interactions, a
two agent and three agent systems of the rock-paper-scissors type are analyzed
in detail, including the stability analysis and numerical simulation.
Scale-free memory is demonstrated to cause complex dynamics of the systems at
hand. In particular, it is shown that there can be simultaneously two modes of
the system instability undergoing subcritical and supercritical bifurcation,
with the latter one exhibiting anomalous oscillations with the amplitude and
period growing with time. Besides, the instability onset via this supercritical
mode may be regarded as "altruism self-organization". For the three agent
system the instability dynamics is found to be rather irregular and can be
composed of alternate fragments of oscillations different in their properties.Comment: 17 pages, 7 figur
- …