2 research outputs found
Grounded Language Understanding for Manipulation Instructions Using GAN-Based Classification
The target task of this study is grounded language understanding for domestic
service robots (DSRs). In particular, we focus on instruction understanding for
short sentences where verbs are missing. This task is of critical importance to
build communicative DSRs because manipulation is essential for DSRs. Existing
instruction understanding methods usually estimate missing information only
from non-grounded knowledge; therefore, whether the predicted action is
physically executable or not was unclear.
In this paper, we present a grounded instruction understanding method to
estimate appropriate objects given an instruction and situation. We extend the
Generative Adversarial Nets (GAN) and build a GAN-based classifier using latent
representations. To quantitatively evaluate the proposed method, we have
developed a data set based on the standard data set used for Visual QA.
Experimental results have shown that the proposed method gives the better
result than baseline methods.Comment: 6 pages, 3 figures, published at IEEE ASRU 201
A Multimodal Classifier Generative Adversarial Network for Carry and Place Tasks from Ambiguous Language Instructions
This paper focuses on a multimodal language understanding method for
carry-and-place tasks with domestic service robots. We address the case of
ambiguous instructions, that is, when the target area is not specified. For
instance "put away the milk and cereal" is a natural instruction where there is
ambiguity regarding the target area, considering environments in daily life.
Conventionally, this instruction can be disambiguated from a dialogue system,
but at the cost of time and cumbersome interaction. Instead, we propose a
multimodal approach, in which the instructions are disambiguated using the
robot's state and environment context. We develop the Multi-Modal Classifier
Generative Adversarial Network (MMC-GAN) to predict the likelihood of different
target areas considering the robot's physical limitation and the target
clutter. Our approach, MMC-GAN, significantly improves accuracy compared with
baseline methods that use instructions only or simple deep neural networks.Comment: 9 pages, 7 figures, accepted for IEEE Robotics and Automation Letters
(RA-L