Learning Generalized Reactive Policies using Deep Neural Networks

We present a new approach to learning for planning, where knowledge acquired while solving a given set of planning problems is used to plan faster in related, but new problem instances. We show that a deep neural network can be used to learn and represent a \emph{generalized reactive policy} (GRP) that maps a problem instance and a state to an action, and that the learned GRPs efficiently solve large classes of challenging problem instances. In contrast to prior efforts in this direction, our approach significantly reduces the dependence of learning on handcrafted domain knowledge or feature selection. Instead, the GRP is trained from scratch using a set of successful execution traces. We show that our approach can also be used to automatically learn a heuristic function that can be used in directed search algorithms. We evaluate our approach using an extensive suite of experiments on two challenging planning problem domains and show that our approach facilitates learning complex decision making policies and powerful heuristic functions with minimal human input. Videos of our results are available at goo.gl/Hpy4e3

Knowledge formalization in experience feedback processes : an ontology-based approach

Because of the current trend of integration and interoperability of industrial systems, their size and complexity continue to grow making it more difficult to analyze, to understand and to solve the problems that happen in their organizations. Continuous improvement methodologies are powerful tools in order to understand and to solve problems, to control the effects of changes and finally to capitalize knowledge about changes and improvements. These tools involve suitably represent knowledge relating to the concerned system. Consequently, knowledge management (KM) is an increasingly important source of competitive advantage for organizations. Particularly, the capitalization and sharing of knowledge resulting from experience feedback are elements which play an essential role in the continuous improvement of industrial activities. In this paper, the contribution deals with semantic interoperability and relates to the structuring and the formalization of an experience feedback (EF) process aiming at transforming information or understanding gained by experience into explicit knowledge. The reuse of such knowledge has proved to have significant impact on achieving themissions of companies. However, the means of describing the knowledge objects of an experience generally remain informal. Based on an experience feedback process model and conceptual graphs, this paper takes domain ontology as a framework for the clarification of explicit knowledge and know-how, the aim of which is to get lessons learned descriptions that are significant, correct and applicable

Analytical learning and term-rewriting systems

Analytical learning is a set of machine learning techniques for revising the representation of a theory based on a small set of examples of that theory. When the representation of the theory is correct and complete but perhaps inefficient, an important objective of such analysis is to improve the computational efficiency of the representation. Several algorithms with this purpose have been suggested, most of which are closely tied to a first order logical language and are variants of goal regression, such as the familiar explanation based generalization (EBG) procedure. But because predicate calculus is a poor representation for some domains, these learning algorithms are extended to apply to other computational models. It is shown that the goal regression technique applies to a large family of programming languages, all based on a kind of term rewriting system. Included in this family are three language families of importance to artificial intelligence: logic programming, such as Prolog; lambda calculus, such as LISP; and combinatorial based languages, such as FP. A new analytical learning algorithm, AL-2, is exhibited that learns from success but is otherwise quite different from EBG. These results suggest that term rewriting systems are a good framework for analytical learning research in general, and that further research should be directed toward developing new techniques

A general technique for automatically optimizing programs through the use of proof plans

The use of {\em proof plans} -- formal patterns of reasoning for theorem proving -- to control the (automatic) synthesis of efficient programs from standard definitional equations is described. A general framework for synthesizing efficient programs, using tools such as higher-order unification, has been developed and holds promise for encapsulating an otherwise diverse, and often ad hoc, range of transformation techniques. A prototype system has been implemented. We illustrate the methodology by a novel means of affecting {\em constraint-based} program optimization through the use of proof plans for mathematical induction. Proof plans are used to control the (automatic) synthesis of functional programs, specified in a standard equational form, {$\cal E$}, by using the proofs as programs principle. The goal is that the program extracted from a constructive proof of the specification is an optimization of that defined solely by {$\cal E$}. Thus the theorem proving process is a form of program optimization allowing for the construction of an efficient, {\em target}, program from the definition of an inefficient, {\em source}, program. The general technique for controlling the syntheses of efficient programs involves using {$\cal E$} to specify the target program and then introducing a new sub-goal into the proof of that specification. Different optimizations are achieved by placing different characterizing restrictions on the form of this new sub-goal and hence on the subsequent proof. Meta-variables and higher-order unification are used in a technique called {\em middle-out reasoning} to circumvent eureka steps concerning, amongst other things, the identification of recursive data-types, and unknown constraint functions. Such problems typically require user intervention

A survey of program transformation with special reference to unfold/fold style program development

This paper consists of a survey of current, and past, work on *program transformation* for the purpose of optimization. We first discuss some of the general methodological frameworks for program modification, such as *analogy*, *explanation based learning*, *partial evaluation*, *proof theoretic optimization*, and the *unfold/fold* technique. These frameworks are not mutually exclusive, and the latter, unfold/fold, is certainly the most widely used technique, in various guises, for program transformation. Thus we shall often have occasion to: compare the relative merits of systems that employ the technique in some form, *and*; compare the unfold/fold systems with those that employ alternative techniques. We also include (and compare with unfold/fold) a brief survey of recent work concerning the use of *formal methods* for program transformation

Approaches to the reuse of plan schemata in planning formalisms

Planning in complex domains is normally a resource and time consuming process when it is purely based on first principles. Once a plan is generated it represents problem solving knowledge. It implicitly describes knowledge used by the planning system to achieve a given goal state from a particular initial state. In classical planning systems, this knowledge is often lost after the plan has been successfully executed. If such a planner has to solve the same problem again, it will spend the same planning effort to solve it and is not capable of "learning\u27; from its "experience\u27;. Therefore it seems to be useful to save generated plans for a later reuse and thus, extending the problem solving knowledge possessed by the planner. The planning knowledge can now be applied to find out whether a problem can be solved by adapting an already existing plan. The aim of this paper is to analyze the problem of plan reuse and to describe the state of the art based on a variety of approaches which might contribute to a solution of the problem. It describes the main problems and results that could be of some relevance for the integration of plan reuse into a deductive planning formalism. As a result, this description of the state of the art leads to a deeper insight into the complex problem of plan reuse, but also shows that the problem itself is still far from being solved

Extending OntoUML Modelling Capabilities on the OpenPonk Platform

Tato práce se zaměřuje na rozšíření možností pro vytváření OntoUML modelů na platformě OpenPonk. Toto rozšíření je rozděleno do čtyř částí. Prvním rozšířením je grafické uživatelské rozhraní pro zobrazování výsledků verifikačního frameworku. Druhá část je prezenotvána novým frameworkem, sloužícím k automatické aktualizaci OunoUML verifikací. Třetím rozšířením je automatická detekce OntoUML anit-patternů. Poslední část se sestává z vybudování nové sekce portálu ontouml.org, obsahující dokumentaci k jednotlivým anti-patternům. V závěru práce je detekce anti-patternů demostrována na referenčním modelu.This work focuses on extending OntoUML modelling capabilities on the OpenPonk platform. This is done in four parts. First part of the expansion is graphical user interface for displaying results of the verification framework. Second part is represented by new framework, which is used for automatic updating of OntoUML verifications. Third part of the expansion is automatic detection of OntoUML anti-patterns. Last part consists of new section on portal ontouml.org, dedicated to anti-pattern documentation. End of this thesis focuses on demonstration of the anti-pattern detection using reference model

Recolha e conceptualização de experiências de atividades robóticas baseadas em planos para melhoria de competências no longo prazo

Robot learning is a prominent research direction in intelligent robotics. Robotics involves dealing with the issue of integration of multiple technologies, such as sensing, planning, acting, and learning. In robot learning, the long term goal is to develop robots that learn to perform tasks and continuously improve their knowledge and skills through observation and exploration of the environment and interaction with users. While significant research has been performed in the area of learning motor behavior primitives, the topic of learning high-level representations of activities and classes of activities that, decompose into sequences of actions, has not been sufficiently addressed. Learning at the task level is key to increase the robots’ autonomy and flexibility. High-level task knowledge is essential for intelligent robotics since it makes robot programs less dependent on the platform and eases knowledge exchange between robots with different kinematics. The goal of this thesis is to contribute to the development of cognitive robotic capabilities, including supervised experience acquisition through human-robot interaction, high-level task learning from the acquired experiences, and task planning using the acquired task knowledge. A framework containing the required cognitive functions for learning and reproduction of high-level aspects of experiences is proposed. In particular, we propose and formalize the notion of Experience-Based Planning Domains (EBPDs) for long-term learning and planning. A human-robot interaction interface is used to provide a robot with step-by-step instructions on how to perform tasks. Approaches to recording plan-based robot activity experiences including relevant perceptions of the environment and actions taken by the robot are presented. A conceptualization methodology is presented for acquiring task knowledge in the form of activity schemata from experiences. The conceptualization approach is a combination of different techniques including deductive generalization, different forms of abstraction and feature extraction. Conceptualization includes loop detection, scope inference and goal inference. Problem solving in EBPDs is achieved using a two-layer problem solver comprising an abstract planner, to derive an abstract solution for a given task problem by applying a learned activity schema, and a concrete planner, to refine the abstract solution towards a concrete solution. The architecture and the learning and planning methods are applied and evaluated in several real and simulated world scenarios. Finally, the developed learning methods are compared, and conditions where each of them has better applicability are discussed.Aprendizagem de robôs é uma direção de pesquisa proeminente em robótica inteligente. Em robótica, é necessário lidar com a questão da integração de várias tecnologias, como percepção, planeamento, atuação e aprendizagem. Na aprendizagem de robôs, o objetivo a longo prazo é desenvolver robôs que aprendem a executar tarefas e melhoram continuamente os seus conhecimentos e habilidades através da observação e exploração do ambiente e interação com os utilizadores. A investigação tem-se centrado na aprendizagem de comportamentos básicos, ao passo que a aprendizagem de representações de atividades de alto nível, que se decompõem em sequências de ações, e de classes de actividades, não tem sido suficientemente abordada. A aprendizagem ao nível da tarefa é fundamental para aumentar a autonomia e a flexibilidade dos robôs. O conhecimento de alto nível permite tornar o software dos robôs menos dependente da plataforma e facilita a troca de conhecimento entre robôs diferentes. O objetivo desta tese é contribuir para o desenvolvimento de capacidades cognitivas para robôs, incluindo aquisição supervisionada de experiência através da interação humano-robô, aprendizagem de tarefas de alto nível com base nas experiências acumuladas e planeamento de tarefas usando o conhecimento adquirido. Propõe-se uma abordagem que integra diversas funcionalidades cognitivas para aprendizagem e reprodução de aspetos de alto nível detetados nas experiências acumuladas. Em particular, nós propomos e formalizamos a noção de Domínio de Planeamento Baseado na Experiência (Experience-Based Planning Domain, or EBPD) para aprendizagem e planeamento num âmbito temporal alargado. Uma interface para interação humano-robô é usada para fornecer ao robô instruções passo-a-passo sobre como realizar tarefas. Propõe-se uma abordagem para extrair experiências de atividades baseadas em planos, incluindo as percepções relevantes e as ações executadas pelo robô. Uma metodologia de conceitualização é apresentada para a aquisição de conhecimento de tarefa na forma de schemata a partir de experiências. São utilizadas diferentes técnicas, incluindo generalização dedutiva, diferentes formas de abstracção e extração de características. A metodologia inclui detecção de ciclos, inferência de âmbito de aplicação e inferência de objetivos. A resolução de problemas em EBPDs é alcançada usando um sistema de planeamento com duas camadas, uma para planeamento abstrato, aplicando um schema aprendido, e outra para planeamento detalhado. A arquitetura e os métodos de aprendizagem e planeamento são aplicados e avaliados em vários cenários reais e simulados. Finalmente, os métodos de aprendizagem desenvolvidos são comparados e as condições onde cada um deles tem melhor aplicabilidade são discutidos.Programa Doutoral em Informátic

Improving performance through concept formation and conceptual clustering

Research from June 1989 through October 1992 focussed on concept formation, clustering, and supervised learning for purposes of improving the efficiency of problem-solving, planning, and diagnosis. These projects resulted in two dissertations on clustering, explanation-based learning, and means-ends planning, and publications in conferences and workshops, several book chapters, and journals; a complete Bibliography of NASA Ames supported publications is included. The following topics are studied: clustering of explanations and problem-solving experiences; clustering and means-end planning; and diagnosis of space shuttle and space station operating modes

Modeling dialogue with mixed initiative in design space exploration

Exploration with a generative formalism must necessarily account for the nature of interaction between humans and the design space explorer. Established accounts of design interaction are made complicated by two propositions in Woodbury and Burrow\u27s Keynote on design space exploration. First, the emphasis on the primacy of the design space as an ordered collection of partial designs (version, alternatives, extensions). Few studies exist in the design interaction literature on working with multiple threads simultaneously. Second, the need to situate, aid, and amplify human design intentions using computational tools. Although specific research and practice tools on amplification (sketching, generation, variation) have had success, there is a lack of generic, flexible, interoperable, and extensible representation to support amplification. This paper addresses the above, working with design threads and computer-assisted design amplification through a theoretical model of dialogue based on Grice\u27s model of rational conversation. Using the concept of mixed initiative, the paper presents a visual notation for representing dialogue between designer and design space formalism through abstract examples of exploration tasks and dialogue integration.<br /