3 research outputs found
Active Learning within Constrained Environments through Imitation of an Expert Questioner
Active learning agents typically employ a query selection algorithm which
solely considers the agent's learning objectives. However, this may be
insufficient in more realistic human domains. This work uses imitation learning
to enable an agent in a constrained environment to concurrently reason about
both its internal learning goals and environmental constraints externally
imposed, all within its objective function. Experiments are conducted on a
concept learning task to test generalization of the proposed algorithm to
different environmental conditions and analyze how time and resource
constraints impact efficacy of solving the learning problem. Our findings show
the environmentally-aware learning agent is able to statistically outperform
all other active learners explored under most of the constrained conditions. A
key implication is adaptation for active learning agents to more realistic
human environments, where constraints are often externally imposed on the
learner.Comment: In Conference Proceedings for IJCAI 201
Sampling Approach Matters: Active Learning for Robotic Language Acquisition
Ordering the selection of training data using active learning can lead to
improvements in learning efficiently from smaller corpora. We present an
exploration of active learning approaches applied to three grounded language
problems of varying complexity in order to analyze what methods are suitable
for improving data efficiency in learning. We present a method for analyzing
the complexity of data in this joint problem space, and report on how
characteristics of the underlying task, along with design decisions such as
feature selection and classification model, drive the results. We observe that
representativeness, along with diversity, is crucial in selecting data samples.Comment: To appear in IEEE Big Data 202
Here's What I've Learned: Asking Questions that Reveal Reward Learning
Robots can learn from humans by asking questions. In these questions the
robot demonstrates a few different behaviors and asks the human for their
favorite. But how should robots choose which questions to ask? Today's robots
optimize for informative questions that actively probe the human's preferences
as efficiently as possible. But while informative questions make sense from the
robot's perspective, human onlookers often find them arbitrary and misleading.
In this paper we formalize active preference-based learning from the human's
perspective. We hypothesize that -- from the human's point-of-view -- the
robot's questions reveal what the robot has and has not learned. Our insight
enables robots to use questions to make their learning process transparent to
the human operator. We develop and test a model that robots can leverage to
relate the questions they ask to the information these questions reveal. We
then introduce a trade-off between informative and revealing questions that
considers both human and robot perspectives: a robot that optimizes for this
trade-off actively gathers information from the human while simultaneously
keeping the human up to date with what it has learned. We evaluate our approach
across simulations, online surveys, and in-person user studies. Videos of our
user studies and results are available here: https://youtu.be/tC6y_jHN7Vw