10 research outputs found
A Review of Symbolic, Subsymbolic and Hybrid Methods for Sequential Decision Making
The field of Sequential Decision Making (SDM) provides tools for solving
Sequential Decision Processes (SDPs), where an agent must make a series of
decisions in order to complete a task or achieve a goal. Historically, two
competing SDM paradigms have view for supremacy. Automated Planning (AP)
proposes to solve SDPs by performing a reasoning process over a model of the
world, often represented symbolically. Conversely, Reinforcement Learning (RL)
proposes to learn the solution of the SDP from data, without a world model, and
represent the learned knowledge subsymbolically. In the spirit of
reconciliation, we provide a review of symbolic, subsymbolic and hybrid methods
for SDM. We cover both methods for solving SDPs (e.g., AP, RL and techniques
that learn to plan) and for learning aspects of their structure (e.g., world
models, state invariants and landmarks). To the best of our knowledge, no other
review in the field provides the same scope. As an additional contribution, we
discuss what properties an ideal method for SDM should exhibit and argue that
neurosymbolic AI is the current approach which most closely resembles this
ideal method. Finally, we outline several proposals to advance the field of SDM
via the integration of symbolic and subsymbolic AI
Complex Interactions between Multiple Goal Operations in Agent Goal Management
A significant issue in cognitive systems research is to make an agent formulate and manage its own goals. Some cognitive scientists have implemented several goal operations to support this issue, but no one has implemented more than a couple of goal operations within a single agent. One of the reasons for this limitation is the lack of knowledge about how various goals operations interact with one another. This thesis addresses this knowledge gap by implementing multiple-goal operations, including goal formulation, goal change, goal selection, and designing an algorithm to manage any positive or negative interaction between them. These are integrated with a cognitive architecture called MIDCA and applied in five different test domains. We will compare and contrast the architecture\u27s performance with intelligent interaction management with a randomized linearization of goal operations
Complex Interactions between Multiple Goal Operations in Agent Goal Management
A significant issue in cognitive systems research is to make an agent formulate and manage its own goals. Some cognitive scientists have implemented several goal operations to support this issue, but no one has implemented more than a couple of goal operations within a single agent. One of the reasons for this limitation is the lack of knowledge about how various goals operations interact with one another. This thesis addresses this knowledge gap by implementing multiple-goal operations, including goal formulation, goal change, goal selection, and designing an algorithm to manage any positive or negative interaction between them. These are integrated with a cognitive architecture called MIDCA and applied in five different test domains. We will compare and contrast the architecture\u27s performance with intelligent interaction management with a randomized linearization of goal operations
Self Monitoring Goal Driven Autonomy Agents
The growing abundance of autonomous systems is driving the need for robust performance. Most current systems are not fully autonomous and often fail when placed in real environments. Via self-monitoring, agents can identify when their own, or externally given, boundaries are violated, thereby increasing their performance and reliability. Specifically, self-monitoring is the identification of unexpected situations that either (1) prohibit the agent from reaching its goal(s) or (2) result in the agent acting outside of its boundaries. Increasingly complex and open environments warrant the use of such robust autonomy (e.g., self-driving cars, delivery drones, and all types of future digital and physical assistants). The techniques presented herein advance the current state of the art in self-monitoring, demonstrating improved performance in a variety of challenging domains. In the aforementioned domains, there is an inability to plan for all possible situations. In many cases all aspects of a domain are not known beforehand, and, even if they were, the cost of encoding them is high. Self-monitoring agents are able to identify and then respond to previously unexpected situations, or never-before-encountered situations. When dealing with unknown situations, one must start with what is expected behavior and use that to derive unexpected behavior. The representation of expectations will vary among domains; in a real-time strategy game like Starcraft, it could be logically inferred concepts; in a mars rover domain, it could be an accumulation of actions\u27 effects. Nonetheless, explicit expectations are necessary to identify the unexpected. This thesis lays the foundation for self-monitoring in goal driven autonomy agents in both rich and expressive domains and in partially observable domains. We introduce multiple techniques for handling such environments. We show how inferred expectations are needed to enable high level planning in real-time strategy games. We show how a hierarchical structure of Goal-driven Autonomy (GDA) enables agents to operate within large state spaces. Within Hierarchical Task Network planning, we show how informed expectations identify states that are likely to prevent an agent from reaching its goals in dynamic domains. Finally, we give a model of expectations for self-monitoring at the meta-cognitive level, and empirical results of agents equipped with and without metacognitive expectations
Goal reasoning for autonomous agents using automated planning
MenciĂłn Internacional en el tĂtulo de doctorAutomated planning deals with the task of finding a sequence of actions, namely
a plan, which achieves a goal from a given initial state. Most planning research
consider goals are provided by a external user, and agents just have to find a
plan to achieve them. However, there exist many real world domains where
agents should not only reason about their actions but also about their goals,
generating new ones or changing them according to the perceived environment.
In this thesis we aim at broadening the goal reasoning capabilities of planningbased
agents, both when acting in isolation and when operating in the same
environment as other agents.
In single-agent settings, we firstly explore a special type of planning tasks
where we aim at discovering states that fulfill certain cost-based requirements
with respect to a given set of goals. By computing these states, agents are able
to solve interesting tasks such as find escape plans that move agents in to safe
places, hide their true goal to a potential observer, or anticipate dynamically arriving
goals. We also show how learning the environmentâs dynamics may help
agents to solve some of these tasks. Experimental results show that these states
can be quickly found in practice, making agents able to solve new planning
tasks and helping them in solving some existing ones.
In multi-agent settings, we study the automated generation of goals based on
other agentsâ behavior. We focus on competitive scenarios, where we are interested
in computing counterplans that prevent opponents from achieving their
goals. We frame these tasks as counterplanning, providing theoretical properties
of the counterplans that solve them. We also show how agents can benefit
from computing some of the states we propose in the single-agent setting to
anticipate their opponentâs movements, thus increasing the odds of blocking
them. Experimental results show how counterplans can be found in different
environments ranging from competitive planning domains to real-time strategy
games.Programa de Doctorado en Ciencia y TecnologĂa InformĂĄtica por la Universidad Carlos III de MadridPresidenta: Eva OnaindĂa de la Rivaherrera.- Secretario: Ăngel GarcĂa Olaya.- Vocal: Mark Robert
Relational reinforcement learning for planning with exogenous effects
Probabilistic planners have improved recently to the point that they can solve difficult tasks with complex and expressive models. In contrast, learners cannot tackle yet the expressive models that planners do, which forces complex models to be mostly handcrafted. We propose a new learning approach that can learn relational probabilistic models with both action effects and exogenous effects. The proposed learning approach combines a multi-valued variant of inductive logic programming for the generation of candidate models, with an optimization method to select the best set of planning operators to model a problem. We also show how to combine this learner with reinforcement learning algorithms to solve complete problems. Finally, experimental validation is provided that shows improvements over previous work in both simulation and a robotic task. The robotic task involves a dynamic scenario with several agents where a manipulator robot has to clear the tableware on a table. We show that the exogenous effects learned by our approach allowed the robot to clear the table in a more efficient way.Peer ReviewedPostprint (published version
Goal Reasoning: Papers from the ACS Workshop
This technical report contains the 14 accepted papers presented at the Workshop on Goal Reasoning,
which was held as part of the 2015 Conference on Advances in Cognitive Systems (ACS-15) in Atlanta,
Georgia on 28 May 2015. This is the fourth in a series of workshops related to this topic, the first of
which was the AAAI-10 Workshop on Goal-Directed Autonomy; the second was the Self-Motivated
Agents (SeMoA) Workshop, held at Lehigh University in November 2012; and the third was the Goal
Reasoning Workshop at ACS-13 in Baltimore, Maryland in December 2013
Goal Reasoning: Papers from the ACS workshop
This technical report contains the 11 accepted papers presented at the Workshop on Goal Reasoning,
which was held as part of the 2013 Conference on Advances in Cognitive Systems (ACS-13) in
Baltimore, Maryland on 14 December 2013. This is the third in a series of workshops related to this
topic, the first of which was the AAAI-10 Workshop on Goal-Directed Autonomy while the second was
the Self-Motivated Agents (SeMoA) Workshop, held at Lehigh University in November 2012.
Our objective for holding this meeting was to encourage researchers to share information on the study,
development, integration, evaluation, and application of techniques related to goal reasoning, which
concerns the ability of an intelligent agent to reason about, formulate, select, and manage its
goals/objectives. Goal reasoning differs from frameworks in which agents are told what goals to
achieve, and possibly how goals can be decomposed into subgoals, but not how to dynamically and
autonomously decide what goals they should pursue. This constraint can be limiting for agents that solve
tasks in complex environments when it is not feasible to manually engineer/encode complete knowledge
of what goal(s) should be pursued for every conceivable state. Yet, in such environments, states can be
reached in which actions can fail, opportunities can arise, and events can otherwise take place that
strongly motivate changing the goal(s) that the agent is currently trying to achieve.
This topic is not new; researchers in several areas have studied goal reasoning (e.g., in the context of
cognitive architectures, automated planning, game AI, and robotics). However, it has infrequently been
the focus of intensive study, and (to our knowledge) no other series of meetings has focused specifically
on goal reasoning. As shown in these papers, providing an agent with the ability to reason about its goals
can increase performance measures for some tasks. Recent advances in hardware and software platforms
(involving the availability of interesting/complex simulators or databases) have increasingly permitted
the application of intelligent agents to tasks that involve partially observable and dynamically-updated
states (e.g., due to unpredictable exogenous events), stochastic actions, multiple (cooperating, neutral, or
adversarial) agents, and other complexities. Thus, this is an appropriate time to foster dialogue among
researchers with interests in goal reasoning.
Research on goal reasoning is still in its early stages; no mature application of it yet exists (e.g., for
controlling autonomous unmanned vehicles or in a deployed decision aid). However, it appears to have a
bright future. For example, leaders in the automated planning community have specifically
acknowledged that goal reasoning has a prominent role among intelligent agents that act on their own
plans, and it is gathering increasing attention from roboticists and cognitive systems researchers.
In addition to a survey, the papers in this workshop relate to, among other topics, cognitive architectures
and models, environment modeling, game AI, machine learning, meta-reasoning, planning, selfmotivated
systems, simulation, and vehicle control. The authors discuss a wide range of issues
pertaining to goal reasoning, including representations and reasoning methods for dynamically revising
goal priorities. We hope that readers will find that this theme for enhancing agent autonomy to be
appealing and relevant to their own interests, and that these papers will spur further investigations on
this important yet (mostly) understudied topic
Learning Unknown Event Models
Agents with incomplete environment models are likely to be surprised, and this represents an opportunity to learn. We investigate approaches for situated agents to detect surprises, discriminate among different forms of surprise, and hypothesize new models for the unknown events that surprised them. We instantiate these approaches in a new goal reasoning agent (named FoolMeTwice), investigate its performance in simulation studies, and report that it produces plans with significantly reduced execution cost in comparison to not learning models for surprising events