186 research outputs found
Grounding Spatio-Temporal Language with Transformers
Language is an interface to the outside world. In order for embodied agents
to use it, language must be grounded in other, sensorimotor modalities. While
there is an extended literature studying how machines can learn grounded
language, the topic of how to learn spatio-temporal linguistic concepts is
still largely uncharted. To make progress in this direction, we here introduce
a novel spatio-temporal language grounding task where the goal is to learn the
meaning of spatio-temporal descriptions of behavioral traces of an embodied
agent. This is achieved by training a truth function that predicts if a
description matches a given history of observations. The descriptions involve
time-extended predicates in past and present tense as well as spatio-temporal
references to objects in the scene. To study the role of architectural biases
in this task, we train several models including multimodal Transformer
architectures; the latter implement different attention computations between
words and objects across space and time. We test models on two classes of
generalization: 1) generalization to randomly held-out sentences; 2)
generalization to grammar primitives. We observe that maintaining object
identity in the attention computation of our Transformers is instrumental to
achieving good performance on generalization overall, and that summarizing
object traces in a single token has little influence on performance. We then
discuss how this opens new perspectives for language-guided autonomous embodied
agents. We also release our code under open-source license as well as
pretrained models and datasets to encourage the wider community to build upon
and extend our work in the future.Comment: Contains main article and supplementarie
Towards Learning Abstractions via Reinforcement Learning
In this paper we take the first steps in studying a new approach to synthesis
of efficient communication schemes in multi-agent systems, trained via
reinforcement learning. We combine symbolic methods with machine learning, in
what is referred to as a neuro-symbolic system. The agents are not restricted
to only use initial primitives: reinforcement learning is interleaved with
steps to extend the current language with novel higher-level concepts, allowing
generalisation and more informative communication via shorter messages. We
demonstrate that this approach allow agents to converge more quickly on a small
collaborative construction task.Comment: AIC 2022, 8th International Workshop on Artificial Intelligence and
Cognitio
Grounding Artificial Intelligence in the Origins of Human Behavior
Recent advances in Artificial Intelligence (AI) have revived the quest for
agents able to acquire an open-ended repertoire of skills. However, although
this ability is fundamentally related to the characteristics of human
intelligence, research in this field rarely considers the processes that may
have guided the emergence of complex cognitive capacities during the evolution
of the species.
Research in Human Behavioral Ecology (HBE) seeks to understand how the
behaviors characterizing human nature can be conceived as adaptive responses to
major changes in the structure of our ecological niche. In this paper, we
propose a framework highlighting the role of environmental complexity in
open-ended skill acquisition, grounded in major hypotheses from HBE and recent
contributions in Reinforcement learning (RL). We use this framework to
highlight fundamental links between the two disciplines, as well as to identify
feedback loops that bootstrap ecological complexity and create promising
research directions for AI researchers
Distinguishing rule- and exemplar-based generalization in learning systems
Machine learning systems often do not share the same inductive biases as
humans and, as a result, extrapolate or generalize in ways that are
inconsistent with our expectations. The trade-off between exemplar- and
rule-based generalization has been studied extensively in cognitive psychology;
in this work, we present a protocol inspired by these experimental approaches
to probe the inductive biases that control this tradeoff in category-learning
systems. We isolate two such inductive biases: feature-level bias (differences
in which features are more readily learned) and exemplar or rule bias
(differences in how these learned features are used for generalization). We
find that standard neural network models are feature-biased and exemplar-based,
and discuss the implications of these findings for machine learning research on
systematic generalization, fairness, and data augmentation.Comment: To appear at the 39th International Conference on Machine Learning
(ICML 2022
Studying the joint role of partial observability and channel reliability in emergent communication
International audienceMulti-Agent Reinforcement Learning (MARL) provides a powerful conceptual and computational framework for modeling emergent communication as a way to solve complex problems in sequential environments. However, despite the recent advances in this field, there is still a need to better understand the role of heterogeneous factors, e.g. partial observability and channel reliability, in the emergence of communication systems. An important step has recently been done in this direction by proposing new information-theoretic measures of emergent communication. As of yet, very few contributions have taken advantage of these new measures to perform detailed quantitative studies analyzing how different environmental and cognitive factors can foster the emergence of communication systems. This work quantitatively measures the joint role of partial observability and channel reliability in the emergence of communication systems. To this end, we performed experiments in a simulated multi-agent grid-world environment where agents learn how to solve different cooperative tasks through MARL
- …