438,174 research outputs found
Interactively Picking Real-World Objects with Unconstrained Spoken Language Instructions
Comprehension of spoken natural language is an essential component for robots
to communicate with human effectively. However, handling unconstrained spoken
instructions is challenging due to (1) complex structures including a wide
variety of expressions used in spoken language and (2) inherent ambiguity in
interpretation of human instructions. In this paper, we propose the first
comprehensive system that can handle unconstrained spoken language and is able
to effectively resolve ambiguity in spoken instructions. Specifically, we
integrate deep-learning-based object detection together with natural language
processing technologies to handle unconstrained spoken instructions, and
propose a method for robots to resolve instruction ambiguity through dialogue.
Through our experiments on both a simulated environment as well as a physical
industrial robot arm, we demonstrate the ability of our system to understand
natural instructions from human operators effectively, and how higher success
rates of the object picking task can be achieved through an interactive
clarification process.Comment: 9 pages. International Conference on Robotics and Automation (ICRA)
2018. Accompanying videos are available at the following links:
https://youtu.be/_Uyv1XIUqhk (the system submitted to ICRA-2018) and
http://youtu.be/DGJazkyw0Ws (with improvements after ICRA-2018 submission
Flexibly Instructable Agents
This paper presents an approach to learning from situated, interactive
tutorial instruction within an ongoing agent. Tutorial instruction is a
flexible (and thus powerful) paradigm for teaching tasks because it allows an
instructor to communicate whatever types of knowledge an agent might need in
whatever situations might arise. To support this flexibility, however, the
agent must be able to learn multiple kinds of knowledge from a broad range of
instructional interactions. Our approach, called situated explanation, achieves
such learning through a combination of analytic and inductive techniques. It
combines a form of explanation-based learning that is situated for each
instruction with a full suite of contextually guided responses to incomplete
explanations. The approach is implemented in an agent called Instructo-Soar
that learns hierarchies of new tasks and other domain knowledge from
interactive natural language instructions. Instructo-Soar meets three key
requirements of flexible instructability that distinguish it from previous
systems: (1) it can take known or unknown commands at any instruction point;
(2) it can handle instructions that apply to either its current situation or to
a hypothetical situation specified in language (as in, for instance,
conditional instructions); and (3) it can learn, from instructions, each class
of knowledge it uses to perform tasks.Comment: See http://www.jair.org/ for any accompanying file
Grounding Hindsight Instructions in Multi-Goal Reinforcement Learning for Robotics
This paper focuses on robotic reinforcement learning with sparse rewards for
natural language goal representations. An open problem is the
sample-inefficiency that stems from the compositionality of natural language,
and from the grounding of language in sensory data and actions. We address
these issues with three contributions. We first present a mechanism for
hindsight instruction replay utilizing expert feedback. Second, we propose a
seq2seq model to generate linguistic hindsight instructions. Finally, we
present a novel class of language-focused learning tasks. We show that
hindsight instructions improve the learning performance, as expected. In
addition, we also provide an unexpected result: We show that the learning
performance of our agent can be improved by one third if, in a sense, the agent
learns to talk to itself in a self-supervised manner. We achieve this by
learning to generate linguistic instructions that would have been appropriate
as a natural language goal for an originally unintended behavior. Our results
indicate that the performance gain increases with the task-complexity.Comment: Preprint ICDL 202
- …