3,634 research outputs found
Knowledge-enhanced Agents for Interactive Text Games
Communication via natural language is a key aspect of machine intelligence,
and it requires computational models to learn and reason about world concepts,
with varying levels of supervision. Significant progress has been made on
fully-supervised non-interactive tasks, such as question-answering and
procedural text understanding. Yet, various sequential interactive tasks, as in
text-based games, have revealed limitations of existing approaches in terms of
coherence, contextual awareness, and their ability to learn effectively from
the environment. In this paper, we propose a knowledge-injection framework for
improved functional grounding of agents in text-based games. Specifically, we
consider two forms of domain knowledge that we inject into learning-based
agents: memory of previous correct actions and affordances of relevant objects
in the environment. Our framework supports two representative model classes:
reinforcement learning agents and language model agents. Furthermore, we devise
multiple injection strategies for the above domain knowledge types and agent
architectures, including injection via knowledge graphs and augmentation of the
existing input encoding strategies. We experiment with four models on the 10
tasks in the ScienceWorld text-based game environment, to illustrate the impact
of knowledge injection on various model configurations and challenging task
settings. Our findings provide crucial insights into the interplay between task
properties, model architectures, and domain knowledge for interactive contexts.Comment: Published at K-CAP '2
A conceptual framework for externally-influenced agents: an assisted reinforcement learning review
A long-term goal of reinforcement learning agents is to be able to perform tasks in complex real-world scenarios. The use of external information is one way of scaling agents to more complex problems. However, there is a general lack of collaboration or interoperability between different approaches using external information. In this work, while reviewing externally-influenced methods, we propose a conceptual framework and taxonomy for assisted reinforcement learning, aimed at fostering collaboration by classifying and comparing various methods that use external information in the learning process. The proposed taxonomy details the relationship between the external information source and the learner agent, highlighting the process of information decomposition, structure, retention, and how it can be used to influence agent learning. As well as reviewing state-of-the-art methods, we identify current streams of reinforcement learning that use external information in order to improve the agent’s performance and its decision-making process. These include heuristic reinforcement learning, interactive reinforcement learning, learning from demonstration, transfer learning, and learning from multiple sources, among others. These streams of reinforcement learning operate with the shared objective of scaffolding the learner agent. Lastly, we discuss further possibilities for future work in the field of assisted reinforcement learning systems. © 2021, The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature
Social interaction for efficient agent learning from human reward
Abstract - Learning from rewards generated by a human trainer observing an agent in action has been proven to be a powerful method for teaching autonomous agents to perform challenging tasks, especially for those non-technical users. Since the efficacy of this approach depends critically on the reward the trainer provides, we consider how the interaction between the trainer and the agent should be designed so as to increase the efficiency of the training process. This article investigates the influence of the agent’s socio-competitive feedback on the human trainer’s training behavior and the agent’s learning. The results of our user study with 85 participants suggest that the agent’s passive socio-competitive feedback—showing performance and score of agents trained by trainers in a leaderboard—substantially increases the engagement of the participants in the game task and improves the agents’ performance, even though the participants do not directly play the game but instead train the agent to do so. Moreover, making this feedback active—sending the trainer her agent’s performance relative to others—further induces more participants to train agents longer and improves the agent’s learning. Our further analysis shows that agents trained by trainers affected by both the passive and active social feedback could obtain a higher performance under a score mechanism that could be optimized from the trainer’s perspective and the agent’s additional active social feedback can keep participants to further train agents to learn policies that can obtain a higher performance under such a score mechanism.Fundamental Research Funds for the Central Universities of China (Grant No. 841713015)China Postdoctoral Science Foundatio
Spatial representation for planning and executing robot behaviors in complex environments
Robots are already improving our well-being and productivity in
different applications such as industry, health-care and indoor
service applications. However, we are still far from developing (and
releasing) a fully functional robotic agent that can autonomously
survive in tasks that require human-level
cognitive capabilities. Robotic systems on the market, in fact, are
designed to address specific applications, and can only run
pre-defined behaviors to robustly repeat few tasks (e.g., assembling
objects parts, vacuum cleaning). They internal representation of the
world is usually constrained to the task they are performing, and
does not allows for generalization to other
scenarios. Unfortunately, such a paradigm only apply to a very
limited set of domains, where the environment can be assumed to be
static, and its dynamics can be handled before
deployment. Additionally, robots configured in this way will
eventually fail if their "handcrafted'' representation of the
environment does not match the external world.
Hence, to enable more sophisticated cognitive skills, we investigate
how to design robots to properly represent the environment and
behave accordingly. To this end, we formalize a representation of
the environment that enhances the robot spatial knowledge to
explicitly include a representation of its own actions. Spatial
knowledge constitutes the core of the robot understanding of the
environment, however it is not sufficient to represent what the
robot is capable to do in it. To overcome such a limitation, we
formalize SK4R, a spatial knowledge representation for robots which
enhances spatial knowledge with a novel and "functional"
point of view that explicitly models robot actions. To this end, we
exploit the concept of affordances, introduced to express
opportunities (actions) that objects offer to an agent. To encode
affordances within SK4R, we define the "affordance
semantics" of actions that is used to annotate an environment, and
to represent to which extent robot actions support goal-oriented
behaviors.
We demonstrate the benefits of a functional representation of the
environment in multiple robotic scenarios that traverse and
contribute different research topics relating to: robot knowledge
representations, social robotics, multi-robot systems and robot
learning and planning. We show how a domain-specific representation,
that explicitly encodes affordance semantics, provides the robot
with a more concrete understanding of the environment and of the
effects that its actions have on it. The goal of our work is to
design an agent that will no longer execute an action, because of
mere pre-defined routine, rather, it will execute an actions because
it "knows'' that the resulting state leads one step closer to
success in its task
Language-conditioned Learning for Robotic Manipulation: A Survey
Language-conditioned robotic manipulation represents a cutting-edge area of
research, enabling seamless communication and cooperation between humans and
robotic agents. This field focuses on teaching robotic systems to comprehend
and execute instructions conveyed in natural language. To achieve this, the
development of robust language understanding models capable of extracting
actionable insights from textual input is essential. In this comprehensive
survey, we systematically explore recent advancements in language-conditioned
approaches within the context of robotic manipulation. We analyze these
approaches based on their learning paradigms, which encompass reinforcement
learning, imitation learning, and the integration of foundational models, such
as large language models and vision-language models. Furthermore, we conduct an
in-depth comparative analysis, considering aspects like semantic information
extraction, environment & evaluation, auxiliary tasks, and task representation.
Finally, we outline potential future research directions in the realm of
language-conditioned learning for robotic manipulation, with the topic of
generalization capabilities and safety issues. The GitHub repository of this
paper can be found at
https://github.com/hk-zh/language-conditioned-robot-manipulation-model
Learning to Speak and Act in a Fantasy Text Adventure Game
We introduce a large scale crowdsourced text adventure game as a research
platform for studying grounded dialogue. In it, agents can perceive, emote, and
act whilst conducting dialogue with other agents. Models and humans can both
act as characters within the game. We describe the results of training
state-of-the-art generative and retrieval models in this setting. We show that
in addition to using past dialogue, these models are able to effectively use
the state of the underlying world to condition their predictions. In
particular, we show that grounding on the details of the local environment,
including location descriptions, and the objects (and their affordances) and
characters (and their previous actions) present within it allows better
predictions of agent behavior and dialogue. We analyze the ingredients
necessary for successful grounding in this setting, and how each of these
factors relate to agents that can talk and act successfully
- …