Behavior Trees in Robotics and AI: An Introduction

A Behavior Tree (BT) is a way to structure the switching between different tasks in an autonomous agent, such as a robot or a virtual entity in a computer game. BTs are a very efficient way of creating complex systems that are both modular and reactive. These properties are crucial in many applications, which has led to the spread of BT from computer game programming to many branches of AI and Robotics. In this book, we will first give an introduction to BTs, then we describe how BTs relate to, and in many cases generalize, earlier switching structures. These ideas are then used as a foundation for a set of efficient and easy to use design principles. Properties such as safety, robustness, and efficiency are important for an autonomous system, and we describe a set of tools for formally analyzing these using a state space description of BTs. With the new analysis tools, we can formalize the descriptions of how BTs generalize earlier approaches. We also show the use of BTs in automated planning and machine learning. Finally, we describe an extended set of tools to capture the behavior of Stochastic BTs, where the outcomes of actions are described by probabilities. These tools enable the computation of both success probabilities and time to completion

Desarrollo automatizado de sistemas teleo-reactivos a partir de objetivos: un enfoque basado en componentes y dirigido por modelos

[SPA] Esta tesis doctoral se presenta bajo la modalidad de compendio de publicaciones. Está formada por un total de cuatro artículos publicados en revistas del segundo cuartil del Journal Citation Reports. El artículo “A systematic literature review of the Teleo-Reactive paradigm” ofrece una completa revisión sistemática de la literatura existente sobre el paradigma Teleo-Reactivo desde su presentación por el profesor Nils Nilsson en el año 1994. Su papel en esta tesis es el de servir de estado del arte de dicho paradigma, ofreciendo una buena perspectiva de la evolución de los sistemas Teleo-Reactivos desde su formulación hasta el presente. Para poder desarrollar sistemas Teleo-Reactivos a partir de objetivos, surgió la necesidad de especificar los requisitos de estos sistemas usando el lenguaje más apropiado. Ese es uno de los objetivos principales del artículo “A controlled experiment to evaluate the understandability of KAOS and i* for modeling Teleo-Reactive systems”. Como resultado de dicho trabajo se decidió utilizar i* dado que el experimento realizado mostró que las especificaciones realizadas con dicho lenguaje resultaban ligeramente más comprensibles que las realizadas con KAOS. Aunque i* resultaba más comprensible a la hora de especificar requisitos para sistemas Teleo-Reactivos, también presentaba ciertas debilidades. Estas debilidades han sido descritas detalladamente en el artículo “A family of experiments to evaluate the understandability of TRiStar and i* for modeling Teleo-Reactive systems”, en el que además se propone una extensión al lenguaje que permite superarlas. La extensión propuesta se denomina TRiStar y fue inicialmente presentada en [Morales15]. TRiStar ha demostrado superar los problemas de comprensibilidad identificados en i* en el modelado de sistemas Teleo-Reactivos mediante una familia de experimentos realizada con estudiantes de últimos cursos de grado y con desarrolladores software experimentados, cuyos resultados se exponen exhaustivamente en el artículo mencionado. En él se describe, además, un mecanismo que permite obtener mediante transformación de modelos el programa Teleo-Reactivo equivalente a un diagrama TRiStar dado. TRiStar permite, por lo tanto, partiendo de los objetivos de un sistema Teleo-Reactivo obtener un diagrama que especifique su comportamiento. Ese diagrama puede ser transformado en un programa Teleo-Reactivo equivalente. Y siguiendo las transformaciones descritas en “From Teleo-Reactive specifications to architectural components: a model-driven approach” se puede obtener a partir del programa Teleo-Reactivo el modelo de componentes y la máquina de estados que describe el comportamiento de cada uno de esos componentes. Con estos elementos y usando un framework como el descrito en [Iborra09] se cerraría el proceso de desarrollo del sistema Teleo-Reactivo. Como resultado de las investigaciones realizadas en el transcurso de esta tesis, y aunque no forma parte del compendio, hay un quinto artículo [Sánchez16] que está en segunda revisión en el Journal of Systems and Software en el que se estudian las posibilidades de introducir requisitos de tiempo real cuando se sigue el enfoque Teleo-Reactivo desde el modelado a la implementación de un sistema. Tras realizar un estudio del tipo de restricciones temporales que se pueden imponer desde el punto de vista Teleo-Reactivo, se considera la posibilidad de utilizar TeleoR [Clark14] para incorporar dichas restricciones y se proponene una serie de extensiones a TRiStar para permitir representar requisitos temporales. Estas extensiones dan lugar a lo que hemos llamado TRiStar+. [ENG] This doctoral dissertation has been presented in the form of thesis by publication. It is comprised of four articles indexed in the second quartile of the Journal Citation Reports. The article “A systematic literature review of the Teleo-Reactive paradigm” offers a complete systematic review of the existing literature on the Teleo-Reactive paradigm since Prof. Nils Nilsson presented it in 1994. It plays the role of state of the art of that paradigm, showing a perspective of the evolution of Teleo-Reactive systems from their formulation to present time. In order to develop Teleo-Reactive systems starting from its goals, there is the need of specifying the requirements of these systems using the most adequate language. That is one of the main objectives of the article “A controlled experiment to evaluate the understandability of KAOS and i* for modeling Teleo-Reactive systems”. As a result, we decided to use i* because the experiment showed that i* specifications where slightly more understandable than those made using KAOS. Although i* was more understandable when specifying requirements for Teleo-Reactive systems, the experiment also showed some shortcomings. These shortcomings have been deeply described in the article “A family of experiments to evaluate the understandability of TRiStar and i* for modeling Teleo-Reactive systems”. In this article, an extension to i* is proposed in order to overcome the identified limitations. The proposed extension is named TRiStar and was initially presented at [Morales15]. TRiStar has shown to be more understandable than i* when modeling Teleo-Reactive systems through a family of experiments done with last year students and experienced software developers, whose results are described in the aforementioned article. In that article, a mechanism to obtain a Teleo-Reactive program starting from a TRiStar diagram is also described. Therefore, TRiStar allows obtaining a diagram which specifies the behavior of a Teleo-Reactive system starting from its goals. That diagram can be transformed into an equivalent Teleo-Reactive program. Then, following the transformations described in “From Teleo-Reactive specifications to architectural components: a model-driven approach”, a component model and the state machine describing the behavior of each of those components can be obtained. With these elements and using a framework as that described in [Iborra09], the development process of the Teleo-Reactive system would be finished. As a result of the research carried out during this dissertation there is another article, which is not comprised in the compilation, in second revision at the Journal of Systems and Software [Sánchez16]. In that article, after making a study of the type of timing constraints from the TR perspective, we consider the possibility of using TeleoR [Clark14] for incorporating such constraints. Some extensions on TRiStar notation are proposed to represent temporal requirements. Those extensions have been named TRiStar+.[ENG] This doctoral dissertation has been presented in the form of thesis by publication. It is comprised of four articles indexed in the second quartile of the Journal Citation Reports. The article “A systematic literature review of the Teleo-Reactive paradigm” offers a complete systematic review of the existing literature on the Teleo-Reactive paradigm since Prof. Nils Nilsson presented it in 1994. It plays the role of state of the art of that paradigm, showing a perspective of the evolution of Teleo-Reactive systems from their formulation to present time. In order to develop Teleo-Reactive systems starting from its goals, there is the need of specifying the requirements of these systems using the most adequate language. That is one of the main objectives of the article “A controlled experiment to evaluate the understandability of KAOS and i* for modeling Teleo-Reactive systems”. As a result, we decided to use i* because the experiment showed that i* specifications where slightly more understandable than those made using KAOS. Although i* was more understandable when specifying requirements for Teleo-Reactive systems, the experiment also showed some shortcomings. These shortcomings have been deeply described in the article “A family of experiments to evaluate the understandability of TRiStar and i* for modeling Teleo-Reactive systems”. In this article, an extension to i* is proposed in order to overcome the identified limitations. The proposed extension is named TRiStar and was initially presented at [Morales15]. TRiStar has shown to be more understandable than i* when modeling Teleo-Reactive systems through a family of experiments done with last year students and experienced software developers, whose results are described in the aforementioned article. In that article, a mechanism to obtain a Teleo-Reactive program starting from a TRiStar diagram is also described. Therefore, TRiStar allows obtaining a diagram which specifies the behavior of a Teleo-Reactive system starting from its goals. That diagram can be transformed into an equivalent Teleo-Reactive program. Then, following the transformations described in “From Teleo-Reactive specifications to architectural components: a model-driven approach”, a component model and the state machine describing the behavior of each of those components can be obtained. With these elements and using a framework as that described in [Iborra09], the development process of the Teleo-Reactive system would be finished. As a result of the research carried out during this dissertation there is another article, which is not comprised in the compilation, in second revision at the Journal of Systems and Software [Sánchez16]. In that article, after making a study of the type of timing constraints from the TR perspective, we consider the possibility of using TeleoR [Clark14] for incorporating such constraints. Some extensions on TRiStar notation are proposed to represent temporal requirements. Those extensions have been named TRiStar+.Universidad Politécnica de CartagenaPrograma Oficial de Doctorado en Tecnologías de la Información y Comunicacione

Designing effective policies for minimal agents

A policy for a minimal reactive agent is a set of condition-action rules used to determine its response to perceived environmental stimuli. When the policy pre-disposes the agent to achieving a stipulated goal we call it a teleo-reactive policy. This paper presents a framework for constructing and evaluating teleo-reactive policies for one or more minimal agents, based upon discounted-reward evaluation of policy-restricted subgraphs of complete situation-graphs. The main feature of the method is that it exploits explicit and definite associations of the agent’s perceptions with states. The combinatorial burden that would potentially ensue from such associations can be ameliorated by suitable use of abstractions. The framework allows one to plan for a number of agents by focusing upon the behaviour of a single representative of them. It allows for varied behaviour to be modelled, including communication between agents. Simulation results presented here indicate that the method affords a good degree of scalability and predictive power

Investigation of a teleo-reactive approach for the development of autonomic manager systems

As the demand for more capable and more feature-rich software increases, the complexity in design, implementation and maintenance also increases exponentially. This becomes a problem when the complexity prevents developers from writing, improving, fixing or otherwise maintaining software to meet specified demands whilst still reaching an acceptable level of robustness. When complexity becomes too great, the software becomes impossible to effectively be managed by even large teams of people. One way to address the problem is an Autonomic approach to software development. Autonomic software aims to tackle complexity by allowing the software to manage itself, thus reducing the need for human intervention and allowing it to reach a maintainable state. Many techniques have been investigated for development of autonomic systems including policy-based designs, utility-functions and advanced architectures. A unique approach to the problem is the teleo-reactive programming paradigm. This paradigm offers a robust and simple structure on which to develop systems. It allows the developer the freedom to express their intentions in a logical manner whilst the increased robustness reduces the maintenance cost. Teleo-Reactive programming is an established solution to low-level agent based problems such as robot navigation and obstacle avoidance, but this technique shows behaviour which is consistent with higher-level autonomic solutions. This project therefore investigates the extent of the applicability of teleo-reactive programming as an autonomic solution. Can the technique be adapted to allow a more ideal fitness for purpose' for autonomics whilst causing minimal changes to the tried and tested original structure and meaning? Does the technique introduce any additional problems and can these be addressed with improvements to the teleo-reactive framework? Teleo-Reactive programming is an interesting approach to autonomic computing because in a Teleo-Reactive program, its state is not predetermined at any moment in time and is based on a priority system where rules execute based on the current environmental context (i.e. not in any strict procedural way) whilst still aiming at the intended goal. This method has been shown to be very robust and exhibits some of the qualities of autonomic software

Designing Effective Policies for Minimal Agents

A policy for a minimal reactive agent is a set of condition-action rules used to determine its response to perceived environmental stimuli. When the policy pre-disposes the agent to achieving a stipulated goal we call it a teleo-reactive policy. This paper presents a framework for constructing and evaluating teleo-reactive policies for one or more minimal agents, based upon discounted-reward evaluation of policy-restricted subgraphs of complete situation graphs. The main feature of the method is that it exploits explicit associations of the agent's perceptions with states. The framework allows to construct and evaluate policies for a number of cooperating agents by focusing upon the behaviour of a single representative of them. This abstraction ameliorates the potential combinatorial burden. Within the framework varied behaviours can be modelled, including communication between agents. Simulation results presented here indicate that the method affords a good degree of predictive power. The paper presents two different branch and bound algorithms used to optimize policy evaluation

Evolutionary approaches to robot path planning

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.The ultimate goal in robotics is to create machines which are more independent and rely less on humans to guide them in their operation. There are many sub-systems which may be present in such a robot, one of which is path planning — the ability to determine a sequence of positions or configurations between an initial and goal position within a particular obstacle cluttered workspace. Many classical path planning techniques have been developed, but these tend to have drawbacks such as their computational requirements; the suitability of the plans they produce for a particular application; or how well they are able to generalise to unseen problems. In recent years, evolutionary based problem solving techniques have seen a rise in popularity, possibly coinciding with the improvement in the computational power afforded researches by successful developments in hardware. These techniques adopt some of the features of natural evolution and mimic them in a computer. The increase in the number of publications in the areas of Genetic Algorithms (GA) and Genetic Programming (GP) demonstrate the success achieved when applying these techniques to ever more problem areas. This dissertation presents research conducted to determine whether there is a place for Evolutionary Approaches, and specifically GA and GP, in the development of future path planning techniques

Adaptive Modelling and Planning for Learning Intelligent Behaviour

Institute of Perception, Action and BehaviourAn intelligent agent must be capable of using its past experience to develop an understanding of how its actions affect the world in which it is situated. Given some objective, the agent must be able to effectively use its understanding of the world to produce a plan that is robust to the uncertainty present in the world. This thesis presents a novel computational framework called the Adaptive Modelling and Planning System (AMPS) that aims to meet these requirements for intelligence. The challenge of the agent is to use its experience in the world to generate a model. In problems with large state and action spaces, the agent can generalise from limited experience by grouping together similar states and actions, effectively partitioning the state and action spaces into finite sets of regions. This process is called abstraction. Several different abstraction approaches have been proposed in the literature, but the existing algorithms have many limitations. They generally only increase resolution, require a large amount of data before changing the abstraction, do not generalise over actions, and are computationally expensive. AMPS aims to solve these problems using a new kind of approach. AMPS splits and merges existing regions in its abstraction according to a set of heuristics. The system introduces splits using a mechanism related to supervised learning and is defined in a general way, allowing AMPS to leverage a wide variety of representations. The system merges existing regions when an analysis of the current plan indicates that doing so could be useful. Because several different regions may require revision at any given time, AMPS prioritises revision to best utilise whatever computational resources are available. Changes in the abstraction lead to changes in the model, requiring changes to the plan. AMPS prioritises the planning process, and when the agent has time, it replans in high-priority regions. This thesis demonstrates the flexibility and strength of this approach in learning intelligent behaviour from limited experience

1993 Annual report on scientific programs: A broad research program on the sciences of complexity

This report provides a summary of many of the research projects completed by the Santa Fe Institute (SFI) during 1993. These research efforts continue to focus on two general areas: the study of, and search for, underlying scientific principles governing complex adaptive systems, and the exploration of new theories of computation that incorporate natural mechanisms of adaptation (mutation, genetics, evolution)

Utilising restricted for-loops in genetic programming

Genetic programming is an approach that utilises the power of evolution to allow computers to evolve programs. While loops are natural components of most programming languages and appear in every reasonably-sized application, they are rarely used in genetic programming. The work is to investigate a number of restricted looping constructs to determine whether any significant benefits can be obtained in genetic programming. Possible benefits include: Solving problems which cannot be solved without loops, evolving smaller sized solutions which can be more easily understood by human programmers and solving existing problems quicker by using fewer evaluations. In this thesis, a number of explicit restricted loop formats were formulated and tested on the Santa Fe ant problem, a modified ant problem, a sorting problem, a visit-every-square problem and a difficult object classificat ion problem. The experimental results showed that these explicit loops can be successfully used in genetic programming. The evolutionary process can decide when, where and how to use them. Runs with these loops tended to generate smaller sized solutions in fewer evaluations. Solutions with loops were found to some problems that could not be solved without loops. The results and analysis of this thesis have established that there are significant benefits in using loops in genetic programming. Restricted loops can avoid the difficulties of evolving consistent programs and the infinite iterations problem. Researchers and other users of genetic programming should not be afraid of loops