Finding and Exploiting LTL Trajectory Constraints in Heuristic Search

We suggest the use of linear temporal logic (LTL) for expressing declarative information about optimal solutions of search problems. We describe a general framework that associates LTLf formulas with search nodes in a heuristic search algorithm. Compared to previous approaches that integrate specific kinds of path information like landmarks into heuristic search, the approach is general, easy to prove correct and easy to integrate with other kinds of path information

Reinforcement Learning With Temporal Logic Rewards

Reinforcement learning (RL) depends critically on the choice of reward functions used to capture the de- sired behavior and constraints of a robot. Usually, these are handcrafted by a expert designer and represent heuristics for relatively simple tasks. Real world applications typically involve more complex tasks with rich temporal and logical structure. In this paper we take advantage of the expressive power of temporal logic (TL) to specify complex rules the robot should follow, and incorporate domain knowledge into learning. We propose Truncated Linear Temporal Logic (TLTL) as specifications language, that is arguably well suited for the robotics applications, together with quantitative semantics, i.e., robustness degree. We propose a RL approach to learn tasks expressed as TLTL formulae that uses their associated robustness degree as reward functions, instead of the manually crafted heuristics trying to capture the same specifications. We show in simulated trials that learning is faster and policies obtained using the proposed approach outperform the ones learned using heuristic rewards in terms of the robustness degree, i.e., how well the tasks are satisfied. Furthermore, we demonstrate the proposed RL approach in a toast-placing task learned by a Baxter robot

Model Predictive Control for Signal Temporal Logic Specification

We present a mathematical programming-based method for model predictive control of cyber-physical systems subject to signal temporal logic (STL) specifications. We describe the use of STL to specify a wide range of properties of these systems, including safety, response and bounded liveness. For synthesis, we encode STL specifications as mixed integer-linear constraints on the system variables in the optimization problem at each step of a receding horizon control framework. We prove correctness of our algorithms, and present experimental results for controller synthesis for building energy and climate control

Inference and Learning with Planning Models

[ES] Inferencia y aprendizaje son los actos de razonar sobre evidencia recogida con el fin de alcanzar conclusiones lógicas sobre el proceso que la originó. En el contexto de un modelo de espacio de estados, inferencia y aprendizaje se refieren normalmente a explicar el comportamiento pasado de un agente, predecir sus acciones futuras, o identificar su modelo. En esta tesis, presentamos un marco para inferencia y aprendizaje en el modelo de espacio de estados subyacente al modelo de planificación clásica, y formulamos una paleta de problemas de inferencia y aprendizaje bajo este paraguas unificador. También desarrollamos métodos efectivos basados en planificación que nos permiten resolver estos problemas utilizando algoritmos de planificación genéricos del estado del arte. Mostraremos que un gran número de problemas de inferencia y aprendizaje claves que han sido tratados como desconectados se pueden formular de forma cohesiva y resolver siguiendo procedimientos homogéneos usando nuestro marco. Además, nuestro trabajo abre las puertas a nuevas aplicaciones para tecnología de planificación ya que resalta las características que hacen que el modelo de espacio de estados de planificación clásica sea diferente a los demás modelos.[CA] Inferència i aprenentatge són els actes de raonar sobre evidència arreplegada a fi d'aconseguir conclusions lògiques sobre el procés que la va originar. En el context d'un model d'espai d'estats, inferència i aprenentatge es referixen normalment a explicar el comportament passat d'un agent, predir les seues accions futures, o identificar el seu model. En esta tesi, presentem un marc per a inferència i aprenentatge en el model d'espai d'estats subjacent al model de planificació clàssica, i formulem una paleta de problemes d'inferència i aprenentatge davall este paraigua unificador. També desenrotllem mètodes efectius basats en planificació que ens permeten resoldre estos problemes utilitzant algoritmes de planificació genèrics de l'estat de l'art. Mostrarem que un gran nombre de problemes d'inferència i aprenentatge claus que han sigut tractats com desconnectats es poden formular de forma cohesiva i resoldre seguint procediments homogenis usant el nostre marc. A més, el nostre treball obri les portes a noves aplicacions per a tecnologia de planificació ja que ressalta les característiques que fan que el model d'espai d'estats de planificació clàssica siga diferent dels altres models.[EN] Inference and learning are the acts of reasoning about some collected evidence in order to reach a logical conclusion regarding the process that originated it. In the context of a state-space model, inference and learning are usually concerned with explaining an agent's past behaviour, predicting its future actions or identifying its model. In this thesis, we present a framework for inference and learning in the state-space model underlying the classical planning model, and formulate a palette of inference and learning problems under this unifying umbrella. We also develop effective planning-based approaches to solve these problems using off-the-shelf, state-of-the-art planning algorithms. We will show that several core inference and learning problems that previous research has treated as disconnected can be formulated in a cohesive way and solved following homogeneous procedures using the proposed framework. Further, our work opens the way for new applications of planning technology as it highlights the features that make the state-space model of classical planning different from other models.The work developed in this doctoral thesis has been possible thanks to the FPU16/03184 fellowship that I have enjoyed for the duration of my PhD studies. I have also been supported by my advisors’ grants TIN2017-88476-C2-1-R, TIN2014-55637-C2-2-R-AR, and RYC-2015-18009.Aineto García, D. (2022). Inference and Learning with Planning Models [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/18535

From Uncertainty Data to Robust Policies for Temporal Logic Planning

We consider the problem of synthesizing robust disturbance feedback policies for systems performing complex tasks. We formulate the tasks as linear temporal logic specifications and encode them into an optimization framework via mixed-integer constraints. Both the system dynamics and the specifications are known but affected by uncertainty. The distribution of the uncertainty is unknown, however realizations can be obtained. We introduce a data-driven approach where the constraints are fulfilled for a set of realizations and provide probabilistic generalization guarantees as a function of the number of considered realizations. We use separate chance constraints for the satisfaction of the specification and operational constraints. This allows us to quantify their violation probabilities independently. We compute disturbance feedback policies as solutions of mixed-integer linear or quadratic optimization problems. By using feedback we can exploit information of past realizations and provide feasibility for a wider range of situations compared to static input sequences. We demonstrate the proposed method on two robust motion-planning case studies for autonomous driving

Iterative Depth-First Search for Fully Observable Non-Deterministic Planning

Fully Observable Non-Deterministic (FOND) planning models uncertainty through actions with non-deterministic effects. Existing FOND planning algorithms are effective and employ a wide range of techniques. However, most of the existing algorithms are not robust for dealing with both non-determinism and task size. In this paper, we develop a novel iterative depth-first search algorithm that solves FOND planning tasks and produces strong cyclic policies. Our algorithm is explicitly designed for FOND planning, addressing more directly the non-deterministic aspect of FOND planning, and it also exploits the benefits of heuristic functions to make the algorithm more effective during the iterative searching process. We compare our proposed algorithm to well-known FOND planners, and show that it has robust performance over several distinct types of FOND domains considering different metrics

Simulateur tutoriel intelligent pour les opérations robotisées application au bras canadien sur la station spatiale internationale

Cette thèse a pour objectif de développer un simulateur tutoriel intelligent pour l'apprentissage de manipulations robotisées, applicable au bras robot canadien sur la station spatiale internationale. Le simulateur appelé Roman Tutor est une preuve de concept de simulateur d'apprentissage autonome et continu pour des manipulations robotisées complexes. Un tel concept est notamment pertinent pour les futures missions spatiales sur Mars ou sur la Lune, et ce en dépit de l'inadéquation du bras canadien pour de telles missions en raison de sa trop grande complexité. Le fait de démontrer la possibilité de conception d'un simulateur capable, dans une certaine mesure, de donner des rétroactions similaires à celles d'un enseignant humain, pourrait inspirer de nouvelles idées pour des concepts similaires, applicables à des robots plus simples, qui seraient utilisés dans les prochaines missions spatiales. Afin de réaliser ce prototype, il est question de développer et d'intégrer trois composantes originales : premièrement, un planificateur de trajectoires pour des environnements dynamiques présentant des contraintes dures et flexibles ; deuxièmement, un générateur automatique de démonstrations de tâches, lequel fait appel au planificateur de trajectoires pour trouver une trajectoire solution à une tâche de déplacement du bras robot et à des techniques de planification des animations pour filmer la solution obtenue ; et troisièmement, un modèle pédagogique implémentant des stratégies d'intervention pour donner de l'aide à un opérateur manipulant le SSRMS. L'assistance apportée à un opérateur sur Roman Tutor fait appel d'une part à des démonstrations de tâches générées par le générateur automatique de démonstrations, et d'autre part au planificateur de trajectoires pour suivre la progression de l'opérateur sur sa tâche, lui fournir de l'aide et le corriger au besoin

Provably-Correct Task Planning for Autonomous Outdoor Robots

Autonomous outdoor robots should be able to accomplish complex tasks safely and reliably while considering constraints that arise from both the environment and the physical platform. Such tasks extend basic navigation capabilities to specify a sequence of events over time. For example, an autonomous aerial vehicle can be given a surveillance task with contingency plans while complying with rules in regulated airspace, or an autonomous ground robot may need to guarantee a given probability of success while searching for the quickest way to complete the mission. A promising approach for the automatic synthesis of trusted controllers for complex tasks is to employ techniques from formal methods. In formal methods, tasks are formally specified symbolically with temporal logic. The robot then synthesises a controller automatically to execute trusted behaviour that guarantees the satisfaction of specified tasks and regulations. However, a difficulty arises from the lack of expressivity, which means the constraints affecting outdoor robots cannot be specified naturally with temporal logic. The goal of this thesis is to extend the capabilities of formal methods to express the constraints that arise from outdoor applications and synthesise provably-correct controllers with trusted behaviours over time. This thesis focuses on two important types of constraints, resource and safety constraints, and presents three novel algorithms that express tasks with these constraints and synthesise controllers that satisfy the specification. Firstly, this thesis proposes an extension to probabilistic computation tree logic (PCTL) called resource threshold PCTL (RT-PCTL) that naturally defines the mission specification with continuous resource threshold constraints; furthermore, it synthesises an optimal control policy with respect to the probability of success. With RT-PCTL, a state with accumulated resource out of the specified bound is considered to be failed or saturated depending on the specification. The requirements on resource bounds are naturally encoded in the symbolic specification, followed by the automatic synthesis of an optimal controller with respect to the probability of success. Secondly, the thesis proposes an online algorithm called greedy Buchi algorithm (GBA) that reduces the synthesis problem size to avoid the scalability problem. A framework is then presented with realistic control dynamics and physical assumptions in the environment such as wind estimation and fuel constraints. The time and space complexity for the framework is polynomial in the size of the system state, which is efficient for online synthesis. Lastly, the thesis proposes a synthesis algorithm for an optimal controller with respect to completion time given the minimum safety constraints. The algorithm naturally balances between completion time and safety. This work proves an analytical relationship between the probability of success and the conditional completion time given the mission specification. The theoretical contributions in this thesis are validated through realistic simulation examples. This thesis identifies and solves two core problems that contribute to the overall vision of developing a theoretical basis for trusted behaviour in outdoor robots. These contributions serve as a foundation for further research in multi-constrained task planning where a number of different constraints are considered simultaneously within a single framework

Asymptotically Optimal Sampling-Based Motion Planning Methods

Motion planning is a fundamental problem in autonomous robotics that requires finding a path to a specified goal that avoids obstacles and takes into account a robot's limitations and constraints. It is often desirable for this path to also optimize a cost function, such as path length. Formal path-quality guarantees for continuously valued search spaces are an active area of research interest. Recent results have proven that some sampling-based planning methods probabilistically converge toward the optimal solution as computational effort approaches infinity. This survey summarizes the assumptions behind these popular asymptotically optimal techniques and provides an introduction to the significant ongoing research on this topic.Comment: Posted with permission from the Annual Review of Control, Robotics, and Autonomous Systems, Volume 4. Copyright 2021 by Annual Reviews, https://www.annualreviews.org/. 25 pages. 2 figure

Sequence-Oriented Diagnosis of Discrete-Event Systems

Model-based diagnosis has always been conceived as set-oriented, meaning that a candidate is a set of faults, or faulty components, that explains a collection of observations. This perspective applies equally to both static and dynamical systems. Diagnosis of discrete-event systems (DESs) is no exception: a candidate is traditionally a set of faults, or faulty events, occurring in a trajectory of the DES that conforms with a given sequence of observations. As such, a candidate does not embed any temporal relationship among faults, nor does it account for multiple occurrences of the same fault. To improve diagnostic explanation and support decision making, a sequence-oriented perspective to diagnosis of DESs is presented, where a candidate is a sequence of faults occurring in a trajectory of the DES, called a fault sequence. Since a fault sequence is possibly unbounded, as the same fault may occur an unlimited number of times in the trajectory, the set of (output) candidates may be unbounded also, which contrasts with set-oriented diagnosis, where the set of candidates is bounded by the powerset of the domain of faults. Still, a possibly unbounded set of fault sequences is shown to be a regular language, which can be defined by a regular expression over the domain of faults, a property that makes sequence-oriented diagnosis feasible in practice. The task of monitoring-based diagnosis is considered, where a new candidate set is generated at the occurrence of each observation. The approach is based on three different techniques: (1) blind diagnosis, with no compiled knowledge, (2) greedy diagnosis, with total knowledge compilation, and (3) lazy diagnosis, with partial knowledge compilation. By knowledge we mean a data structure slightly similar to a classical DES diagnoser, which can be generated (compiled) either entirely offline (greedy diagnosis) or incrementally online (lazy diagnosis). Experimental evidence suggests that, among these techniques, only lazy diagnosis may be viable in non-trivial application domains