Plan merging by reuse for multi-agent planning

Multi-Agent Planning deals with the task of generating a plan for/by a set of agents that jointly solve a planning problem. One of the biggest challenges is how to handle interactions arising from agents' actions. The first contribution of the paper is Plan Merging by Reuse, pmr, an algorithm that automatically adjusts its behaviour to the level of interaction. Given a multi-agent planning task, pmr assigns goals to specific agents. The chosen agents solve their individual planning tasks and the resulting plans are merged. Since merged plans are not always valid, pmr performs planning by reuse to generate a valid plan. The second contribution of the paper is rrpt-plan, a stochastic plan-reuse planner that combines plan reuse, standard search and sampling. We have performed extensive sets of experiments in order to analyze the performance of pmr in relation to state of the art multi-agent planning techniques.This work has been partially supported by the MINECO projects TIN2017-88476-C2-2-R, RTC-2016-5407-4, and TIN2014-55637-C2-1-R and MICINN project TIN2011-27652-C03-02

ASCoL: Automated Acquisition of Domain Specific Static Constraints from Plan Traces

Domain-independent planning systems require that domain constraints and invariants are specified as part of the input domain model. In AI Planning, the generated plan is correct provided the constraints of the world in which the agent is operating are satisfied. Specifying operator descriptions by hand for planning domain models that also require domain specific constraints is time consuming, error prone and still a challenge for the AI planning community. The LOCM (Cresswell, McCluskey, and West 2013) system carries out automated generation of the dynamic aspects of a planning domain model from a set of example training plans. We enhance the output domain model of the LOCM system to capture static domain constraints from the same set of input training plans as used by LOCM to learn dynamic aspects of the world. In this paper we propose a new framework ASCoL (Automated Static Constraint Learner), to make constraint acquisition more efficient, by observing a set of training plan traces. Most systems that learn constraints automatically do so by analysing the operators of the planning world. Out proposed system will discover static constraints by analysing plan traces for correlations in the data. To do this an algorithm is in the process of development for graph discovery from the collection of ground action instances used in the input plan traces. The proposed algorithm will analyse the complete set of plan traces, based on a predefined set of constraints, and deduces facts from it. We then augment components of the LOCM generated domain with enriched constraints

Learning to Act with RVRL Agents

The use of reinforcement learning to guide action selection of cognitive agents has been shown to be a powerful technique for stochastic environments. Standard Reinforcement learning techniques used to provide decision theoretic policies rely, however, on explicit state-based computations of value for each state-action pair. This requires the computation of a number of values exponential to the number of state variables and actions in the system. This research extends existing work with an acquired probabilistic rule representation of an agent environment by developing an algorithm to apply reinforcement learning to values attached to the rules themselves. Structure captured by the rules is then used to learn a policy directly. The resulting value attached to each rule represents the utility of taking an action if the conditions of the rule are present in the agent’s current set of percepts. This has several advantages for planning purposes: generalization over many states and over unseen states; effective decisions can therefore be made with less training data than state based modelling systems (e.g. Dyna Q-Learning); and the problem of computation in an exponential state-action space is alleviated. The results of application of this algorithm to rules in a specific environment are presented, with comparison to standard reinforcement learning policies developed from related work

Discovering User-Interpretable Capabilities of Black-Box Planning Agents

Several approaches have been developed for answering users' specific questions about AI behavior and for assessing their core functionality in terms of primitive executable actions. However, the problem of summarizing an AI agent's broad capabilities for a user is comparatively new. This paper presents an algorithm for discovering from scratch the suite of high-level "capabilities" that an AI system with arbitrary internal planning algorithms/policies can perform. It computes conditions describing the applicability and effects of these capabilities in user-interpretable terms. Starting from a set of user-interpretable state properties, an AI agent, and a simulator that the agent can interact with, our algorithm returns a set of high-level capabilities with their parameterized descriptions. Empirical evaluation on several game-based scenarios shows that this approach efficiently learns descriptions of various types of AI agents in deterministic, fully observable settings. User studies show that such descriptions are easier to understand and reason with than the agent's primitive actions.Comment: KR 202

A prototype for incremental learning finite factored planning domains from continuous perceptions and some case studies

This master project takes place within the research activity carried on by the Fondazione Bruno Kessler (FBK) in Trento, in Data and Knowledge Management unit (DKM), with Luciano Serafini as internal tutor, and Paolo Traverso as external tutor. The project focuses on automated learning of planning domains. Automated learning and planning often leverages on an abstract representation of the world called planning domains. A planning domain is described by a set of states that correspond to possible configurations of the environment, a set of actions, and a state-transition function between states, which describes the effects of actions on the environment. Planning domain models are used by agents to develop their strategies on how to act in a given environment in order to achieve their goals. The construction of these models is normally entrusted to the engineer who manually programs the agent. This work, however, requires human intervention for each new environment, while it would be desirable for an autonomous agent to be able to build a model of the environment autonomously, even if it is in an unknown environment. The aim of the research carried out by the DKM unit is to use automatic methods to learn these models by carrying out actions, and observing their effects on the environment. The specific objectives of this master thesis are to: (1) acquire specific knowledge in the field of automatic learning and planning; (2) acquire specific knowledge in the context of learning planning models; (3) extend the method developed in the research activity in FBK on states variables that are described by a set of variables taking values over domains; (4) deal with the problem of learning entirely the state-transition function; (5) extend the ALP algorithm with a prediction part to predict the effects of the actions; (6) complete the state-transition function; (7) implement the algorithm with all the new parts, provide examples, and test this extension in some simulators previously created; (8) validate and analyse the results, and measure the goodness of the model