37,585 research outputs found
Recommended from our members
Algorithms for Optimal Paths of One, Many, and an Infinite Number of Agents
In this dissertation, we provide efficient algorithms for modeling the behavior of a single agent, multiple agents, and a continuum of agents. For a single agent, we combine the modeling framework of optimal control with advances in optimization splitting in order to efficiently find optimal paths for problems in very high-dimensions, thus providing alleviation from the curse of dimensionality. For a multiple, but finite, number of agents, we take the framework of multi-agent reinforcement learning and utilize imitation learning in order to decentralize a centralized expert, thus obtaining optimal multi-agents that act in a decentralized fashion. For a continuum of agents, we take the framework of mean-field games and use two neural networks, which we train in an alternating scheme, in order to efficiently find optimal paths for high-dimensional and stochastic problems. These tools cover a wide variety of use-cases that can be immediately deployed for practical applications
Optimizing collective fieldtaxis of swarming agents through reinforcement learning
Swarming of animal groups enthralls scientists in fields ranging from biology
to physics to engineering. Complex swarming patterns often arise from simple
interactions between individuals to the benefit of the collective whole. The
existence and success of swarming, however, nontrivially depend on microscopic
parameters governing the interactions. Here we show that a machine-learning
technique can be employed to tune these underlying parameters and optimize the
resulting performance. As a concrete example, we take an active matter model
inspired by schools of golden shiners, which collectively conduct phototaxis.
The problem of optimizing the phototaxis capability is then mapped to that of
maximizing benefits in a continuum-armed bandit game. The latter problem
accepts a simple reinforcement-learning algorithm, which can tune the
continuous parameters of the model. This result suggests the utility of
machine-learning methodology in swarm-robotics applications.Comment: 6 pages, 3 figure
Continual Reinforcement Learning in 3D Non-stationary Environments
High-dimensional always-changing environments constitute a hard challenge for
current reinforcement learning techniques. Artificial agents, nowadays, are
often trained off-line in very static and controlled conditions in simulation
such that training observations can be thought as sampled i.i.d. from the
entire observations space. However, in real world settings, the environment is
often non-stationary and subject to unpredictable, frequent changes. In this
paper we propose and openly release CRLMaze, a new benchmark for learning
continually through reinforcement in a complex 3D non-stationary task based on
ViZDoom and subject to several environmental changes. Then, we introduce an
end-to-end model-free continual reinforcement learning strategy showing
competitive results with respect to four different baselines and not requiring
any access to additional supervised signals, previously encountered
environmental conditions or observations.Comment: Accepted in the CLVision Workshop at CVPR2020: 13 pages, 4 figures, 5
table
Exact solution of a modified El Farol's bar problem: Efficiency and the role of market impact
We discuss a model of heterogeneous, inductive rational agents inspired by
the El Farol Bar problem and the Minority Game. As in markets, agents interact
through a collective aggregate variable -- which plays a role similar to price
-- whose value is fixed by all of them. Agents follow a simple
reinforcement-learning dynamics where the reinforcement, for each of their
available strategies, is related to the payoff delivered by that strategy. We
derive the exact solution of the model in the ``thermodynamic'' limit of
infinitely many agents using tools of statistical physics of disordered
systems. Our results show that the impact of agents on the market price plays a
key role: even though price has a weak dependence on the behavior of each
individual agent, the collective behavior crucially depends on whether agents
account for such dependence or not. Remarkably, if the adaptive behavior of
agents accounts even ``infinitesimally'' for this dependence they can, in a
whole range of parameters, reduce global fluctuations by a finite amount. Both
global efficiency and individual utility improve with respect to a ``price
taker'' behavior if agents account for their market impact.Comment: 38 pages + 5 figures (needs elsart.sty). New results adde
A Data-driven Model for Interaction-aware Pedestrian Motion Prediction in Object Cluttered Environments
This paper reports on a data-driven, interaction-aware motion prediction
approach for pedestrians in environments cluttered with static obstacles. When
navigating in such workspaces shared with humans, robots need accurate motion
predictions of the surrounding pedestrians. Human navigation behavior is mostly
influenced by their surrounding pedestrians and by the static obstacles in
their vicinity. In this paper we introduce a new model based on Long-Short Term
Memory (LSTM) neural networks, which is able to learn human motion behavior
from demonstrated data. To the best of our knowledge, this is the first
approach using LSTMs, that incorporates both static obstacles and surrounding
pedestrians for trajectory forecasting. As part of the model, we introduce a
new way of encoding surrounding pedestrians based on a 1d-grid in polar angle
space. We evaluate the benefit of interaction-aware motion prediction and the
added value of incorporating static obstacles on both simulation and real-world
datasets by comparing with state-of-the-art approaches. The results show, that
our new approach outperforms the other approaches while being very
computationally efficient and that taking into account static obstacles for
motion predictions significantly improves the prediction accuracy, especially
in cluttered environments.Comment: 8 pages, accepted for publication at the IEEE International
Conference on Robotics and Automation (ICRA) 201
A Data-driven Model for Interaction-aware Pedestrian Motion Prediction in Object Cluttered Environments
This paper reports on a data-driven, interaction-aware motion prediction
approach for pedestrians in environments cluttered with static obstacles. When
navigating in such workspaces shared with humans, robots need accurate motion
predictions of the surrounding pedestrians. Human navigation behavior is mostly
influenced by their surrounding pedestrians and by the static obstacles in
their vicinity. In this paper we introduce a new model based on Long-Short Term
Memory (LSTM) neural networks, which is able to learn human motion behavior
from demonstrated data. To the best of our knowledge, this is the first
approach using LSTMs, that incorporates both static obstacles and surrounding
pedestrians for trajectory forecasting. As part of the model, we introduce a
new way of encoding surrounding pedestrians based on a 1d-grid in polar angle
space. We evaluate the benefit of interaction-aware motion prediction and the
added value of incorporating static obstacles on both simulation and real-world
datasets by comparing with state-of-the-art approaches. The results show, that
our new approach outperforms the other approaches while being very
computationally efficient and that taking into account static obstacles for
motion predictions significantly improves the prediction accuracy, especially
in cluttered environments.Comment: 8 pages, accepted for publication at the IEEE International
Conference on Robotics and Automation (ICRA) 201
- …