9,332 research outputs found
Inverse Reinforcement Learning in Swarm Systems
Inverse reinforcement learning (IRL) has become a useful tool for learning
behavioral models from demonstration data. However, IRL remains mostly
unexplored for multi-agent systems. In this paper, we show how the principle of
IRL can be extended to homogeneous large-scale problems, inspired by the
collective swarming behavior of natural systems. In particular, we make the
following contributions to the field: 1) We introduce the swarMDP framework, a
sub-class of decentralized partially observable Markov decision processes
endowed with a swarm characterization. 2) Exploiting the inherent homogeneity
of this framework, we reduce the resulting multi-agent IRL problem to a
single-agent one by proving that the agent-specific value functions in this
model coincide. 3) To solve the corresponding control problem, we propose a
novel heterogeneous learning scheme that is particularly tailored to the swarm
setting. Results on two example systems demonstrate that our framework is able
to produce meaningful local reward models from which we can replicate the
observed global system dynamics.Comment: 9 pages, 8 figures; ### Version 2 ### version accepted at AAMAS 201
Heterogeneous Stochastic Interactions for Multiple Agents in a Multi-armed Bandit Problem
We define and analyze a multi-agent multi-armed bandit problem in which
decision-making agents can observe the choices and rewards of their neighbors.
Neighbors are defined by a network graph with heterogeneous and stochastic
interconnections. These interactions are determined by the sociability of each
agent, which corresponds to the probability that the agent observes its
neighbors. We design an algorithm for each agent to maximize its own expected
cumulative reward and prove performance bounds that depend on the sociability
of the agents and the network structure. We use the bounds to predict the rank
ordering of agents according to their performance and verify the accuracy
analytically and computationally
Modeling the mobility of living organisms in heterogeneous landscapes: Does memory improve foraging success?
Thanks to recent technological advances, it is now possible to track with an
unprecedented precision and for long periods of time the movement patterns of
many living organisms in their habitat. The increasing amount of data available
on single trajectories offers the possibility of understanding how animals move
and of testing basic movement models. Random walks have long represented the
main description for micro-organisms and have also been useful to understand
the foraging behaviour of large animals. Nevertheless, most vertebrates, in
particular humans and other primates, rely on sophisticated cognitive tools
such as spatial maps, episodic memory and travel cost discounting. These
properties call for other modeling approaches of mobility patterns. We propose
a foraging framework where a learning mobile agent uses a combination of
memory-based and random steps. We investigate how advantageous it is to use
memory for exploiting resources in heterogeneous and changing environments. An
adequate balance of determinism and random exploration is found to maximize the
foraging efficiency and to generate trajectories with an intricate
spatio-temporal order. Based on this approach, we propose some tools for
analysing the non-random nature of mobility patterns in general.Comment: 14 pages, 4 figures, improved discussio
- …