54,023 research outputs found
Learning a Policy for Opportunistic Active Learning
Active learning identifies data points to label that are expected to be the
most useful in improving a supervised model. Opportunistic active learning
incorporates active learning into interactive tasks that constrain possible
queries during interactions. Prior work has shown that opportunistic active
learning can be used to improve grounding of natural language descriptions in
an interactive object retrieval task. In this work, we use reinforcement
learning for such an object retrieval task, to learn a policy that effectively
trades off task completion with model improvement that would benefit future
tasks.Comment: EMNLP 2018 Camera Read
WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents
Existing benchmarks for grounding language in interactive environments either
lack real-world linguistic elements, or prove difficult to scale up due to
substantial human involvement in the collection of data or feedback signals. To
bridge this gap, we develop WebShop -- a simulated e-commerce website
environment with million real-world products and crowd-sourced
text instructions. Given a text instruction specifying a product requirement,
an agent needs to navigate multiple types of webpages and issue diverse actions
to find, customize, and purchase an item. WebShop provides several challenges
for language grounding including understanding compositional instructions,
query (re-)formulation, comprehending and acting on noisy text in webpages, and
performing strategic exploration. We collect over human demonstrations
for the task, and train and evaluate a diverse range of agents using
reinforcement learning, imitation learning, and pre-trained image and language
models. Our best model achieves a task success rate of , which
outperforms rule-based heuristics () but is far lower than human expert
performance (). We also analyze agent and human trajectories and ablate
various model components to provide insights for developing future agents with
stronger language understanding and decision making abilities. Finally, we show
that agents trained on WebShop exhibit non-trivial sim-to-real transfer when
evaluated on amazon.com and ebay.com, indicating the potential value of WebShop
in developing practical web-based agents that can operate in the wild.Comment: Project page with code, data, demos: https://webshop-pnlp.github.io.
v2 adds transfer to eBa
Knowledge-enhanced Agents for Interactive Text Games
Communication via natural language is a key aspect of machine intelligence,
and it requires computational models to learn and reason about world concepts,
with varying levels of supervision. Significant progress has been made on
fully-supervised non-interactive tasks, such as question-answering and
procedural text understanding. Yet, various sequential interactive tasks, as in
text-based games, have revealed limitations of existing approaches in terms of
coherence, contextual awareness, and their ability to learn effectively from
the environment. In this paper, we propose a knowledge-injection framework for
improved functional grounding of agents in text-based games. Specifically, we
consider two forms of domain knowledge that we inject into learning-based
agents: memory of previous correct actions and affordances of relevant objects
in the environment. Our framework supports two representative model classes:
reinforcement learning agents and language model agents. Furthermore, we devise
multiple injection strategies for the above domain knowledge types and agent
architectures, including injection via knowledge graphs and augmentation of the
existing input encoding strategies. We experiment with four models on the 10
tasks in the ScienceWorld text-based game environment, to illustrate the impact
of knowledge injection on various model configurations and challenging task
settings. Our findings provide crucial insights into the interplay between task
properties, model architectures, and domain knowledge for interactive contexts.Comment: Published at K-CAP '2
Learning how to learn: an adaptive dialogue agent for incrementally learning visually grounded word meanings
We present an optimised multi-modal dialogue agent for interactive learning
of visually grounded word meanings from a human tutor, trained on real
human-human tutoring data. Within a life-long interactive learning period, the
agent, trained using Reinforcement Learning (RL), must be able to handle
natural conversations with human users and achieve good learning performance
(accuracy) while minimising human effort in the learning process. We train and
evaluate this system in interaction with a simulated human tutor, which is
built on the BURCHAK corpus -- a Human-Human Dialogue dataset for the visual
learning task. The results show that: 1) The learned policy can coherently
interact with the simulated user to achieve the goal of the task (i.e. learning
visual attributes of objects, e.g. colour and shape); and 2) it finds a better
trade-off between classifier accuracy and tutoring costs than hand-crafted
rule-based policies, including ones with dynamic policies.Comment: 10 pages, RoboNLP Workshop from ACL Conferenc
A discriminative approach to grounded spoken language understanding in interactive robotics
Spoken Language Understanding in Interactive Robotics provides computational models of human-machine communication based on the vocal input. However, robots operate in specific environments and the correct interpretation of the spoken sentences depends on the physical, cognitive and linguistic aspects triggered by the operational environment. Grounded language processing should exploit both the physical constraints of the context as well as knowledge assumptions of the robot. These include the subjective perception of the environment that explicitly affects linguistic reasoning. In this work, a standard linguistic pipeline for semantic parsing is extended toward a form of perceptually informed natural language processing that combines discriminative learning and distributional semantics. Empirical results achieve up to a 40% of relative error reduction
Grounding Language for Transfer in Deep Reinforcement Learning
In this paper, we explore the utilization of natural language to drive
transfer for reinforcement learning (RL). Despite the wide-spread application
of deep RL techniques, learning generalized policy representations that work
across domains remains a challenging problem. We demonstrate that textual
descriptions of environments provide a compact intermediate channel to
facilitate effective policy transfer. Specifically, by learning to ground the
meaning of text to the dynamics of the environment such as transitions and
rewards, an autonomous agent can effectively bootstrap policy learning on a new
domain given its description. We employ a model-based RL approach consisting of
a differentiable planning module, a model-free component and a factorized state
representation to effectively use entity descriptions. Our model outperforms
prior work on both transfer and multi-task scenarios in a variety of different
environments. For instance, we achieve up to 14% and 11.5% absolute improvement
over previously existing models in terms of average and initial rewards,
respectively.Comment: JAIR 201
- …