2 research outputs found
Pre-Learning Environment Representations for Data-Efficient Neural Instruction Following
We consider the problem of learning to map from natural language instructions
to state transitions (actions) in a data-efficient manner. Our method takes
inspiration from the idea that it should be easier to ground language to
concepts that have already been formed through pre-linguistic observation. We
augment a baseline instruction-following learner with an initial
environment-learning phase that uses observations of language-free state
transitions to induce a suitable latent representation of actions before
processing the instruction-following training data. We show that mapping to
pre-learned representations substantially improves performance over systems
whose representations are learned from limited instructional data alone.Comment: ACL 201
Robust and Interpretable Grounding of Spatial References with Relation Networks
Learning representations of spatial references in natural language is a key
challenge in tasks like autonomous navigation and robotic manipulation. Recent
work has investigated various neural architectures for learning multi-modal
representations for spatial concepts. However, the lack of explicit reasoning
over entities makes such approaches vulnerable to noise in input text or state
observations. In this paper, we develop effective models for understanding
spatial references in text that are robust and interpretable, without
sacrificing performance. We design a text-conditioned \textit{relation network}
whose parameters are dynamically computed with a cross-modal attention module
to capture fine-grained spatial relations between entities. This design choice
provides interpretability of learned intermediate outputs. Experiments across
three tasks demonstrate that our model achieves superior performance, with a
17\% improvement in predicting goal locations and a 15\% improvement in
robustness compared to state-of-the-art systems.Comment: Findings of Empirical Methods in Natural Language Processing (EMNLP)
202