15 research outputs found
Learning Interpretable Spatial Operations in a Rich 3D Blocks World
In this paper, we study the problem of mapping natural language instructions
to complex spatial actions in a 3D blocks world. We first introduce a new
dataset that pairs complex 3D spatial operations to rich natural language
descriptions that require complex spatial and pragmatic interpretations such as
"mirroring", "twisting", and "balancing". This dataset, built on the simulation
environment of Bisk, Yuret, and Marcu (2016), attains language that is
significantly richer and more complex, while also doubling the size of the
original dataset in the 2D environment with 100 new world configurations and
250,000 tokens. In addition, we propose a new neural architecture that achieves
competitive results while automatically discovering an inventory of
interpretable spatial operations (Figure 5)Comment: AAAI 201
Learning a Policy for Opportunistic Active Learning
Active learning identifies data points to label that are expected to be the
most useful in improving a supervised model. Opportunistic active learning
incorporates active learning into interactive tasks that constrain possible
queries during interactions. Prior work has shown that opportunistic active
learning can be used to improve grounding of natural language descriptions in
an interactive object retrieval task. In this work, we use reinforcement
learning for such an object retrieval task, to learn a policy that effectively
trades off task completion with model improvement that would benefit future
tasks.Comment: EMNLP 2018 Camera Read
Improving Grounded Natural Language Understanding through Human-Robot Dialog
Natural language understanding for robotics can require substantial domain-
and platform-specific engineering. For example, for mobile robots to
pick-and-place objects in an environment to satisfy human commands, we can
specify the language humans use to issue such commands, and connect concept
words like red can to physical object properties. One way to alleviate this
engineering for a new domain is to enable robots in human environments to adapt
dynamically---continually learning new language constructions and perceptual
concepts. In this work, we present an end-to-end pipeline for translating
natural language commands to discrete robot actions, and use clarification
dialogs to jointly improve language parsing and concept grounding. We train and
evaluate this agent in a virtual setting on Amazon Mechanical Turk, and we
transfer the learned agent to a physical robot platform to demonstrate it in
the real world