Plan Projection, Execution, and Learning for Mobile Robot Control

Most state-of-the-art hybrid control systems for mobile robots are decomposed into different layers. While the deliberation layer reasons about the actions required for the robot in order to achieve a given goal, the behavioral layer is designed to enable the robot to quickly react to unforeseen events. This decomposition guarantees a safe operation even in the presence of unforeseen and dynamic obstacles and enables the robot to cope with situations it was not explicitly programmed for. The layered design, however, also leaves us with the problem of plan execution. The problem of plan execution is the problem of arbitrating between the deliberation- and the behavioral layer. Abstract symbolic actions have to be translated into streams of local control commands. Simultaneously, execution failures have to be handled on an appropriate level of abstraction. It is now widely accepted that plan execution should form a third layer of a hybrid robot control system. The resulting layered architectures are called three-tiered architectures, or 3T architectures for short. Although many high level programming frameworks have been proposed to support the implementation of the intermediate layer, there is no generally accepted algorithmic basis for plan execution in three-tiered architectures. In this thesis, we propose to base plan execution on plan projection and learning and present a general framework for the self-supervised improvement of plan execution. This framework has been implemented in APPEAL, an Architecture for Plan Projection, Execution And Learning, which extends the well known RHINO control system by introducing an execution layer. This thesis contributes to the field of plan-based mobile robot control which investigates the interrelation between planning, reasoning, and learning techniques based on an explicit representation of the robot's intended course of action, a plan. In McDermott's terminology, a plan is that part of a robot control program, which the robot cannot only execute, but also reason about and manipulate. According to that broad view, a plan may serve many purposes in a robot control system like reasoning about future behavior, the revision of intended activities, or learning. In this thesis, plan-based control is applied to the self-supervised improvement of mobile robot plan execution

On the definition of non-player character behaviour for real-time simulated virtual environments.

Computer games with complex virtual worlds, which are populated by artificial characters and creatures, are the most visible application of artificial intelligence techniques. In recent years game development has been fuelled by dramatic advances in computer graphics hardware which have led to a rise in the quality of real-time computer graphics and increased realism in computer games. As a result of these developments video games are gaining acceptance and cultural significance as a form of art and popular culture. An important factor for the attainment of realism in games is the artificially intelligent behaviour displayed by the virtual entities that populate the games' virtual worlds. It is our firm belief that to further improve the behaviour of virtual entities, game AI development will have to mirror the advances achieved in game graphics. A major contributing factor for these advancements has been the advent of programmable shaders for real-time graphics, which in turn has been significantly simplified by the introduction of higher level programming languages for the creation of shaders. This has demonstrated that a good system can be vastly improved by the addition of a programming language. This thesis presents a similar (syntactic) approach to the definition of the behaviour of virtual entities in computer games. We introduce the term behaviour definition language (BDL), describing a programming language for the definition of game entity behaviour. We specify the requirements for this type of programming language, which are applied to the development and implementation of several behaviour definition languages, culminating in the design of a new game-genre independent behaviour definition (scripting) language. This extension programming language includes several game AI techniques within a single unified system, allowing the use of different methods of behaviour definition. A subset of the language (itself a BDL) was implemented as a proof of concept of this design, providing a framework for the syntactic definition of the behaviour of virtual entities in computer games

Acting, Planning, and Learning Using Hierarchical Operational Models

The most common representation formalisms for planning are descriptive models that abstractly describe what the actions do and are tailored for efficiently computing the next state(s) in a state-transition system. However, real-world acting requires operational models that describe how to do things, with rich control structures for closed-loop online decision-making in a dynamic environment. Use of a different action model for planning than the one used for acting causes problems with combining acting and planning, in particular for the development and consistency verification of the different models. As an alternative, this dissertation defines and implements an integrated acting-and-planning system in which both planning and acting use the same operational models, which are written in a general-purpose hierarchical task-oriented language offering rich control structures. The acting component, called Reactive Acting Engine (RAE), is inspired by the well-known PRS system, except that instead of being purely reactive, it can get advice from a planner. The dissertation also describes three planning algorithms which plan by doing several Monte Carlo rollouts in the space of operational models. The best of these three planners, Plan-with-UPOM uses a UCT-like Monte Carlo Tree Search procedure called UPOM (UCT Procedure for Operational Models), whose rollouts are simulated executions of the actor's operational models. The dissertation also presents learning strategies for use with RAE and UPOM that acquire from online acting experiences and/or simulated planning results, a mapping from decision contexts to method instances as well as a heuristic function to guide UPOM. The experimental results show that Plan-with-UPOM and the learning strategies significantly improve the acting efficiency and robustness of RAE. It can be proved that UPOM converges asymptotically by mapping its search space to an MDP. The dissertation also describes a real-world prototype of RAE and Plan-with-UPOM to defend software-defined networks, a relatively new network management architecture, against incoming attacks

Temporal and Hierarchical Models for Planning and Acting in Robotics

The field of AI planning has seen rapid progress over the last decade and planners are now able to find plan with hundreds of actions in a matter of seconds. Despite those important progresses, robotic systems still tend to have a reactive architecture with very little deliberation on the course of the plan they might follow. In this thesis, we argue that a successful integration with a robotic system requires the planner to have capacities for both temporal and hierarchical reasoning. The former is indeed a universal resource central in many robot activities while the latter is a critical component for the integration of reasoning capabilities at different abstraction levels, typically starting with a high level view of an activity that is iteratively refined down to motion primitives. As a first step to carry out this vision, we present a model for temporal planning unifying the generative and hierarchical approaches. At the center of the model are temporal action templates, similar to those of PDDL complemented with a specification of the initial state as well as the expected evolution of the environment over time. In addition, our model allows for the specification of hierarchical knowledge possibly with a partial coverage. Consequently, our model generalizes the existing generative and HTN approaches together with an explicit time representation. In the second chapter, we introduce a planning procedure suitable for our planning model. In order to support hierarchical features, we extend the existing Partial-Order Causal Link approach used in many constraintbased planners, with the notions of task and decomposition. We implement it in FAPE (Flexible Acting and Planning Environment) together with automated problem analysis techniques used for search guidance. We show FAPE to have performance similar to state of the art temporal planners when used in a generative setting. The addition of hierarchical information leads to further performance gain and allows us to outperform traditional planners. In the third chapter, we study the usual methods used to reason on temporal uncertainty while planning. We relax the usual assumption of total observability and instead provide techniques to reason on the observations needed to maintain a plan dispatchable. We show how such needed observations can be detected at planning time and incrementally dealt with by considering the appropriate sensing actions. In a final chapter, we discuss the place of the proposed planning system as a central component for the control of a robotic actor. We demonstrate how the explicit time representation facilitates plan monitoring and action dispatching when dealing with contingent events that require observation. We take advantage of the constraint-based and hierarchical representation to facilitate both plan-repair procedures as well opportunistic plan refinement at acting time

Szenenabhängige Online-Adaption von Manipulationssequenzen für einen Serviceroboter

Um in alltäglichen Umgebungen agieren zu können, muss ein Serviceroboter in der Lage sein, seine Handlung zielgerichtet an neue Szenen anzupassen. In dieser Arbeit werden dazu Handlungspläne zur Manipulation durch symbolische Planung erzeugt. Damit diese konsistent mit der realen Umgebung sind, wird aufgrund von sensorischer Wahrnehmung ein symbolisches Modell der Umgebung erzeugt. Dieses erlaubt es dem Roboter, die Effekte einer Aktion, bzw. Aktionsfolge auf seine Umwelt vorherzusagen

Behavior-based Control for Service Robots inspired by Human Motion Patterns : a Robotic Shopping Assistant

Es wurde, unter Verwendung menschenähnlicher Bewegungsmuster und eines verhaltensbasierten Ansatzes, eine Steuerung für mobile Serviceroboter entwickelt, die Aufgabenplanung, globale und lokale Navigation in dynamischen Umgebungen, sowie die gemeinsame Aufgabenausführung mit einem Benutzer umfasst. Das Verhaltensnetzwerk besteht aus Modulen mit voneinander unabhängigen Aufgaben. Das komplexe Gesamtverhalten des Systems ergibt sich durch die Vereinigung der Einzelverhalten (\u27Emergenz\u27)