45,599 research outputs found
An active inference model of hierarchical action understanding, learning and imitation
We advance a novel active inference model of the cognitive processing that underlies the acquisition of a hierarchical action repertoire and its use for observation, understanding and imitation. We illustrate the model in four simulations of a tennis learner who observes a teacher performing tennis shots, forms hierarchical representations of the observed actions, and imitates them. Our simulations show that the agent's oculomotor activity implements an active information sampling strategy that permits inferring the kinematic aspects of the observed movement, which lie at the lowest level of the action hierarchy. In turn, this low-level kinematic inference supports higher-level inferences about deeper aspects of the observed actions: proximal goals and intentions. Finally, the inferred action representations can steer imitative responses, but interfere with the execution of different actions. Our simulations show that hierarchical active inference provides a unified account of action observation, understanding, learning and imitation and helps explain the neurobiological underpinnings of visuomotor cognition, including the multiple routes for action understanding in the dorsal and ventral streams and mirror mechanisms
Hierarchical Generative Adversarial Imitation Learning with Mid-level Input Generation for Autonomous Driving on Urban Environments
Deriving robust control policies for realistic urban navigation scenarios is
not a trivial task. In an end-to-end approach, these policies must map
high-dimensional images from the vehicle's cameras to low-level actions such as
steering and throttle. While pure Reinforcement Learning (RL) approaches are
based exclusively on rewards,Generative Adversarial Imitation Learning (GAIL)
agents learn from expert demonstrations while interacting with the environment,
which favors GAIL on tasks for which a reward signal is difficult to derive. In
this work, the hGAIL architecture was proposed to solve the autonomous
navigation of a vehicle in an end-to-end approach, mapping sensory perceptions
directly to low-level actions, while simultaneously learning mid-level input
representations of the agent's environment. The proposed hGAIL consists of an
hierarchical Adversarial Imitation Learning architecture composed of two main
modules: the GAN (Generative Adversarial Nets) which generates the Bird's-Eye
View (BEV) representation mainly from the images of three frontal cameras of
the vehicle, and the GAIL which learns to control the vehicle based mainly on
the BEV predictions from the GAN as input.Our experiments have shown that GAIL
exclusively from cameras (without BEV) fails to even learn the task, while
hGAIL, after training, was able to autonomously navigate successfully in all
intersections of the city
From motor babbling to hierarchical learning by imitation: a robot developmental pathway
How does an individual use the knowledge
acquired through self exploration as a manipulable model through which to understand
others and benefit from their knowledge?
How can developmental and social learning be
combined for their mutual benefit? In this
paper we review a hierarchical architecture
(HAMMER) which allows a principled way
for combining knowledge through exploration
and knowledge from others, through the creation and use of multiple inverse and forward
models. We describe how Bayesian Belief Networks can be used to learn the association
between a robot’s motor commands and sensory consequences (forward models), and how
the inverse association can be used for imitation. Inverse models created through self
exploration, as well as those from observing
others can coexist and compete in a principled unified framework, that utilises the simulation theory of mind approach to mentally
rehearse and understand the actions of others
CompILE: Compositional Imitation Learning and Execution
We introduce Compositional Imitation Learning and Execution (CompILE): a
framework for learning reusable, variable-length segments of
hierarchically-structured behavior from demonstration data. CompILE uses a
novel unsupervised, fully-differentiable sequence segmentation module to learn
latent encodings of sequential data that can be re-composed and executed to
perform new tasks. Once trained, our model generalizes to sequences of longer
length and from environment instances not seen during training. We evaluate
CompILE in a challenging 2D multi-task environment and a continuous control
task, and show that it can find correct task boundaries and event encodings in
an unsupervised manner. Latent codes and associated behavior policies
discovered by CompILE can be used by a hierarchical agent, where the high-level
policy selects actions in the latent code space, and the low-level,
task-specific policies are simply the learned decoders. We found that our
CompILE-based agent could learn given only sparse rewards, where agents without
task-specific policies struggle.Comment: ICML (2019
Prediction of intent in robotics and multi-agent systems.
Moving beyond the stimulus contained in observable agent behaviour, i.e. understanding the underlying intent of the observed agent is of immense interest in a variety of domains that involve collaborative and competitive scenarios, for example assistive robotics, computer games, robot-human interaction, decision support and intelligent tutoring. This review paper examines approaches for performing action recognition and prediction of intent from a multi-disciplinary perspective, in both single robot and multi-agent scenarios, and analyses the underlying challenges, focusing mainly on generative approaches
- …