Behavior Trees in Robotics and AI: An Introduction

A Behavior Tree (BT) is a way to structure the switching between different tasks in an autonomous agent, such as a robot or a virtual entity in a computer game. BTs are a very efficient way of creating complex systems that are both modular and reactive. These properties are crucial in many applications, which has led to the spread of BT from computer game programming to many branches of AI and Robotics. In this book, we will first give an introduction to BTs, then we describe how BTs relate to, and in many cases generalize, earlier switching structures. These ideas are then used as a foundation for a set of efficient and easy to use design principles. Properties such as safety, robustness, and efficiency are important for an autonomous system, and we describe a set of tools for formally analyzing these using a state space description of BTs. With the new analysis tools, we can formalize the descriptions of how BTs generalize earlier approaches. We also show the use of BTs in automated planning and machine learning. Finally, we describe an extended set of tools to capture the behavior of Stochastic BTs, where the outcomes of actions are described by probabilities. These tools enable the computation of both success probabilities and time to completion

Desarrollo automatizado de sistemas teleo-reactivos a partir de objetivos: un enfoque basado en componentes y dirigido por modelos

[SPA] Esta tesis doctoral se presenta bajo la modalidad de compendio de publicaciones. Está formada por un total de cuatro artículos publicados en revistas del segundo cuartil del Journal Citation Reports. El artículo “A systematic literature review of the Teleo-Reactive paradigm” ofrece una completa revisión sistemática de la literatura existente sobre el paradigma Teleo-Reactivo desde su presentación por el profesor Nils Nilsson en el año 1994. Su papel en esta tesis es el de servir de estado del arte de dicho paradigma, ofreciendo una buena perspectiva de la evolución de los sistemas Teleo-Reactivos desde su formulación hasta el presente. Para poder desarrollar sistemas Teleo-Reactivos a partir de objetivos, surgió la necesidad de especificar los requisitos de estos sistemas usando el lenguaje más apropiado. Ese es uno de los objetivos principales del artículo “A controlled experiment to evaluate the understandability of KAOS and i* for modeling Teleo-Reactive systems”. Como resultado de dicho trabajo se decidió utilizar i* dado que el experimento realizado mostró que las especificaciones realizadas con dicho lenguaje resultaban ligeramente más comprensibles que las realizadas con KAOS. Aunque i* resultaba más comprensible a la hora de especificar requisitos para sistemas Teleo-Reactivos, también presentaba ciertas debilidades. Estas debilidades han sido descritas detalladamente en el artículo “A family of experiments to evaluate the understandability of TRiStar and i* for modeling Teleo-Reactive systems”, en el que además se propone una extensión al lenguaje que permite superarlas. La extensión propuesta se denomina TRiStar y fue inicialmente presentada en [Morales15]. TRiStar ha demostrado superar los problemas de comprensibilidad identificados en i* en el modelado de sistemas Teleo-Reactivos mediante una familia de experimentos realizada con estudiantes de últimos cursos de grado y con desarrolladores software experimentados, cuyos resultados se exponen exhaustivamente en el artículo mencionado. En él se describe, además, un mecanismo que permite obtener mediante transformación de modelos el programa Teleo-Reactivo equivalente a un diagrama TRiStar dado. TRiStar permite, por lo tanto, partiendo de los objetivos de un sistema Teleo-Reactivo obtener un diagrama que especifique su comportamiento. Ese diagrama puede ser transformado en un programa Teleo-Reactivo equivalente. Y siguiendo las transformaciones descritas en “From Teleo-Reactive specifications to architectural components: a model-driven approach” se puede obtener a partir del programa Teleo-Reactivo el modelo de componentes y la máquina de estados que describe el comportamiento de cada uno de esos componentes. Con estos elementos y usando un framework como el descrito en [Iborra09] se cerraría el proceso de desarrollo del sistema Teleo-Reactivo. Como resultado de las investigaciones realizadas en el transcurso de esta tesis, y aunque no forma parte del compendio, hay un quinto artículo [Sánchez16] que está en segunda revisión en el Journal of Systems and Software en el que se estudian las posibilidades de introducir requisitos de tiempo real cuando se sigue el enfoque Teleo-Reactivo desde el modelado a la implementación de un sistema. Tras realizar un estudio del tipo de restricciones temporales que se pueden imponer desde el punto de vista Teleo-Reactivo, se considera la posibilidad de utilizar TeleoR [Clark14] para incorporar dichas restricciones y se proponene una serie de extensiones a TRiStar para permitir representar requisitos temporales. Estas extensiones dan lugar a lo que hemos llamado TRiStar+. [ENG] This doctoral dissertation has been presented in the form of thesis by publication. It is comprised of four articles indexed in the second quartile of the Journal Citation Reports. The article “A systematic literature review of the Teleo-Reactive paradigm” offers a complete systematic review of the existing literature on the Teleo-Reactive paradigm since Prof. Nils Nilsson presented it in 1994. It plays the role of state of the art of that paradigm, showing a perspective of the evolution of Teleo-Reactive systems from their formulation to present time. In order to develop Teleo-Reactive systems starting from its goals, there is the need of specifying the requirements of these systems using the most adequate language. That is one of the main objectives of the article “A controlled experiment to evaluate the understandability of KAOS and i* for modeling Teleo-Reactive systems”. As a result, we decided to use i* because the experiment showed that i* specifications where slightly more understandable than those made using KAOS. Although i* was more understandable when specifying requirements for Teleo-Reactive systems, the experiment also showed some shortcomings. These shortcomings have been deeply described in the article “A family of experiments to evaluate the understandability of TRiStar and i* for modeling Teleo-Reactive systems”. In this article, an extension to i* is proposed in order to overcome the identified limitations. The proposed extension is named TRiStar and was initially presented at [Morales15]. TRiStar has shown to be more understandable than i* when modeling Teleo-Reactive systems through a family of experiments done with last year students and experienced software developers, whose results are described in the aforementioned article. In that article, a mechanism to obtain a Teleo-Reactive program starting from a TRiStar diagram is also described. Therefore, TRiStar allows obtaining a diagram which specifies the behavior of a Teleo-Reactive system starting from its goals. That diagram can be transformed into an equivalent Teleo-Reactive program. Then, following the transformations described in “From Teleo-Reactive specifications to architectural components: a model-driven approach”, a component model and the state machine describing the behavior of each of those components can be obtained. With these elements and using a framework as that described in [Iborra09], the development process of the Teleo-Reactive system would be finished. As a result of the research carried out during this dissertation there is another article, which is not comprised in the compilation, in second revision at the Journal of Systems and Software [Sánchez16]. In that article, after making a study of the type of timing constraints from the TR perspective, we consider the possibility of using TeleoR [Clark14] for incorporating such constraints. Some extensions on TRiStar notation are proposed to represent temporal requirements. Those extensions have been named TRiStar+.[ENG] This doctoral dissertation has been presented in the form of thesis by publication. It is comprised of four articles indexed in the second quartile of the Journal Citation Reports. The article “A systematic literature review of the Teleo-Reactive paradigm” offers a complete systematic review of the existing literature on the Teleo-Reactive paradigm since Prof. Nils Nilsson presented it in 1994. It plays the role of state of the art of that paradigm, showing a perspective of the evolution of Teleo-Reactive systems from their formulation to present time. In order to develop Teleo-Reactive systems starting from its goals, there is the need of specifying the requirements of these systems using the most adequate language. That is one of the main objectives of the article “A controlled experiment to evaluate the understandability of KAOS and i* for modeling Teleo-Reactive systems”. As a result, we decided to use i* because the experiment showed that i* specifications where slightly more understandable than those made using KAOS. Although i* was more understandable when specifying requirements for Teleo-Reactive systems, the experiment also showed some shortcomings. These shortcomings have been deeply described in the article “A family of experiments to evaluate the understandability of TRiStar and i* for modeling Teleo-Reactive systems”. In this article, an extension to i* is proposed in order to overcome the identified limitations. The proposed extension is named TRiStar and was initially presented at [Morales15]. TRiStar has shown to be more understandable than i* when modeling Teleo-Reactive systems through a family of experiments done with last year students and experienced software developers, whose results are described in the aforementioned article. In that article, a mechanism to obtain a Teleo-Reactive program starting from a TRiStar diagram is also described. Therefore, TRiStar allows obtaining a diagram which specifies the behavior of a Teleo-Reactive system starting from its goals. That diagram can be transformed into an equivalent Teleo-Reactive program. Then, following the transformations described in “From Teleo-Reactive specifications to architectural components: a model-driven approach”, a component model and the state machine describing the behavior of each of those components can be obtained. With these elements and using a framework as that described in [Iborra09], the development process of the Teleo-Reactive system would be finished. As a result of the research carried out during this dissertation there is another article, which is not comprised in the compilation, in second revision at the Journal of Systems and Software [Sánchez16]. In that article, after making a study of the type of timing constraints from the TR perspective, we consider the possibility of using TeleoR [Clark14] for incorporating such constraints. Some extensions on TRiStar notation are proposed to represent temporal requirements. Those extensions have been named TRiStar+.Universidad Politécnica de CartagenaPrograma Oficial de Doctorado en Tecnologías de la Información y Comunicacione

‘Teach Me - Show Me’- End-user personalisation of a smart home and companion robot

© 2015 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications standards/publications/rights/index.html for more information.Care issues and costs associated with an increasing elderly population is becoming a major concern for many countries. The use of assistive robots in ‘smart-home’ environments has been suggested as a possible partial solution to these concerns. One of the many challenges faced is the personalisation of the robot to meet the changing needs of the elderly person over time. One approach is to allow the elderly person, or their carers or relatives, to teach the robot to both recognise activities in the smart home and to teach it to carry out behaviours in response to these activities. The overriding premise being that such teaching is both intuitive and ‘non-technical’. As part of a European project researching and evaluating these issues a commercially available autonomous robot has been deployed in a fully sensorised but otherwise ordinary suburban house. Occupants of the house are equipped with a non-technical teaching and learning system. This paper details the design approach to the teaching, learning, robot and smart home systems as an integrated unit and presents results from an evaluation of the teaching component and a preliminary evaluation of the learning component in a Human-Robot interaction experiment. Results from this evaluation indicated that participants overall found this approach to personalisation useful, easy to use, and felt that they would be capable of using it in a real-life situation both for themselves and for others. However there were also some salient individual differences within the sample.Peer reviewe

Artificial Intelligence Research Branch future plans

This report contains information on the activities of the Artificial Intelligence Research Branch (FIA) at NASA Ames Research Center (ARC) in 1992, as well as planned work in 1993. These activities span a range from basic scientific research through engineering development to fielded NASA applications, particularly those applications that are enabled by basic research carried out in FIA. Work is conducted in-house and through collaborative partners in academia and industry. All of our work has research themes with a dual commitment to technical excellence and applicability to NASA short, medium, and long-term problems. FIA acts as the Agency's lead organization for research aspects of artificial intelligence, working closely with a second research laboratory at the Jet Propulsion Laboratory (JPL) and AI applications groups throughout all NASA centers. This report is organized along three major research themes: (1) Planning and Scheduling: deciding on a sequence of actions to achieve a set of complex goals and determining when to execute those actions and how to allocate resources to carry them out; (2) Machine Learning: techniques for forming theories about natural and man-made phenomena; and for improving the problem-solving performance of computational systems over time; and (3) Research on the acquisition, representation, and utilization of knowledge in support of diagnosis design of engineered systems and analysis of actual systems

Self Monitoring Goal Driven Autonomy Agents

The growing abundance of autonomous systems is driving the need for robust performance. Most current systems are not fully autonomous and often fail when placed in real environments. Via self-monitoring, agents can identify when their own, or externally given, boundaries are violated, thereby increasing their performance and reliability. Specifically, self-monitoring is the identification of unexpected situations that either (1) prohibit the agent from reaching its goal(s) or (2) result in the agent acting outside of its boundaries. Increasingly complex and open environments warrant the use of such robust autonomy (e.g., self-driving cars, delivery drones, and all types of future digital and physical assistants). The techniques presented herein advance the current state of the art in self-monitoring, demonstrating improved performance in a variety of challenging domains. In the aforementioned domains, there is an inability to plan for all possible situations. In many cases all aspects of a domain are not known beforehand, and, even if they were, the cost of encoding them is high. Self-monitoring agents are able to identify and then respond to previously unexpected situations, or never-before-encountered situations. When dealing with unknown situations, one must start with what is expected behavior and use that to derive unexpected behavior. The representation of expectations will vary among domains; in a real-time strategy game like Starcraft, it could be logically inferred concepts; in a mars rover domain, it could be an accumulation of actions\u27 effects. Nonetheless, explicit expectations are necessary to identify the unexpected. This thesis lays the foundation for self-monitoring in goal driven autonomy agents in both rich and expressive domains and in partially observable domains. We introduce multiple techniques for handling such environments. We show how inferred expectations are needed to enable high level planning in real-time strategy games. We show how a hierarchical structure of Goal-driven Autonomy (GDA) enables agents to operate within large state spaces. Within Hierarchical Task Network planning, we show how informed expectations identify states that are likely to prevent an agent from reaching its goals in dynamic domains. Finally, we give a model of expectations for self-monitoring at the meta-cognitive level, and empirical results of agents equipped with and without metacognitive expectations

A synthesis of automated planning and reinforcement learning for efficient, robust decision-making

Automated planning and reinforcement learning are characterized by complementary views on decision making: the former relies on previous knowledge and computation, while the latter on interaction with the world, and experience. Planning allows robots to carry out different tasks in the same domain, without the need to acquire knowledge about each one of them, but relies strongly on the accuracy of the model. Reinforcement learning, on the other hand, does not require previous knowledge, and allows robots to robustly adapt to the environment, but often necessitates an infeasible amount of experience. We present Domain Approximation for Reinforcement LearnING (DARLING), a method that takes advantage of planning to constrain the behavior of the agent to reasonable choices, and of reinforcement learning to adapt to the environment, and increase the reliability of the decision making process. We demonstrate the effectiveness of the proposed method on a service robot, carrying out a variety of tasks in an office building. We find that when the robot makes decisions by planning alone on a given model it often fails, and when it makes decisions by reinforcement learning alone it often cannot complete its tasks in a reasonable amount of time. When employing DARLING, even when seeded with the same model that was used for planning alone, however, the robot can quickly learn a behavior to carry out all the tasks, improves over time, and adapts to the environment as it changes

GRUE : an architecture for agents in games and other real-time environments

This thesis presents an architecture, which we call GRUE, for intelligent agents in real-time dynamic worlds. Such environments require agents to be able to flexibly adjust their behaviour to take into account changes in the environment or other agents’ actions. Our architecture is based on work done in robotics (Nilsson, 1994; Benson and Nilsson, 1995; Benson, 1996), which also deals with complex, dynamic environments. Our work focuses on goal arbitration, the method used by the agent to choose an appropriate goal for the current situation, and to re-evaluate when the situation changes. In the process, we have also developed a method for representing items in the environment, which we call resources, in terms of their properties. This allows the agent to specify a needed object in terms of required properties and use available objects with appropriate properties interchangeably. We show that the GRUE architecture can be used successfully in both a typical AI test bed and a commercial game environment. In addition, we have undertaken to experimentally test the effects of the features included in our architecture by comparing agents using the standard GRUE architecture to agents with one or more features removed and find that these features do improve the performance of the agent where expected