Prescribed Performance Control Guided Policy Improvement for Satisfying Signal Temporal Logic Tasks

Signal temporal logic (STL) provides a user-friendly interface for defining complex tasks for robotic systems. Recent efforts aim at designing control laws or using reinforcement learning methods to find policies which guarantee satisfaction of these tasks. While the former suffer from the trade-off between task specification and computational complexity, the latter encounter difficulties in exploration as the tasks become more complex and challenging to satisfy. This paper proposes to combine the benefits of the two approaches and use an efficient prescribed performance control (PPC) base law to guide exploration within the reinforcement learning algorithm. The potential of the method is demonstrated in a simulated environment through two sample navigational tasks.Comment: This is the extended version of the paper accepted to the 2019 American Control Conference (ACC), Philadelphia (to be published

Sampling-based Synthesis of Controllers for Multiple Agents under Signal Temporal Logic Specifications

openL’ampia applicazione dei robot nelle industrie e nella società ha portato alla necessità di prescrivere complessi compiti di alto livello ad agenti autonomi. Signal Temporal Logic (STL) è una logica temporale che consente di esprimere requisiti spazio-temporali e quantificare il livello di soddisfazione delle preferenze. Quando si pianifica considerando specifiche STL, la sfida principale è generare traiettorie che soddisfino le formule logiche e seguire le traiettorie così ottenute. Il progetto propone una soluzione per il problema di pianificazione del movimento di multipli agenti autonomi, soggetti a specifiche STL accoppiate. Partendo da uno scenario in cui sono coinvolti solo due agenti, un algoritmo basato sul campionamento, Coupled STL_RRT*, è progettato. L’approccio proposto, basato su RRT*, costruisce in modo distribuito due alberi nel dominio del tempo e dello stato accoppiati. Per ogni sistema dinamico, data una posizione iniziale, la strategia sviluppata trova la traiettoria probabilisticamente ottimale in termini di una funzione di costo che dipende dagli input di controllo richiesti. Prima di aggiungere nuovi stati all’albero corrispondente, l’algoritmo controlla se la formula logica non viene violata, assicurando quindi che la traiettoria finale, variabile nel tempo, soddisfi le specifiche spazio-temporali. La dinamica dell’agente autonomo è presa direttamente in considerazione e il concetto di raggiungibilità viene sfruttato per ottenere traiettorie ammissibili rispetto ai vincoli dinamici. L’algoritmo è quindi simulato, considerando un ambiente con ostacoli statici e diversi requisiti STL, specificati dall’utente. L’approccio viene poi esteso al caso di sistemi multi-agente con più di tre agenti. Come nel caso precedente, l’algoritmo costruisce un albero spazio-temporale per ciascun agente, assicurando che la traiettoria finale soddisfi i requisiti STL. La soluzione proposta è poi verificata in scenari simulati, considerando sistemi con 4 o 6 agenti.The wide application of robots in industries and society has brought the need to prescribe complex high-level tasks to autonomous agents. Signal Temporal Logic (STL) is a temporal logic that allows to express desired spatio-temporal requirements, while quantifying the satisfaction of the preferences. When planning under STL specifications, the main challenge is to generate trajectories that satisfy the logical formulas and to track those trajectories. The project proposes a solution for the motion planning problem of multiple autonomous agents, subject to coupled STL specifications. Starting from a scenario where only two agents are involved, a sampling-based algorithm, Coupled STL_RRT*, is designed. The proposed RRT*-based approach builds two trees in the coupled time and state domain in a distributed manner. For each dynamical system, given an initial position, the developed strategy finds a probabilistic optimal trajectory in terms of a cost function that depends on the required control inputs. Before adding new states to the corresponding tree, the algorithm checks if the logical formula is not violated, hence ensuring that the final time-varying trajectory satisfies the spatio-temporal specifications. The dynamics of the autonomous agent is directly taken into account and reachability is exploited to obtain a trajectory that is feasible with respect to the dynamic constraints. The algorithm is then simulated, considering an environment with static obstacles and different STL requirements, specified by the user. The approach is then extended to the case of multi-agent systems with more than three agents. As in the previous case, the algorithm builds a spatiotemporal tree for each agent, ensuring that the final trajectory satisfies the STL requirements. The proposed solution is then verified in simulated scenarios, considering 4-agents and 6-agents systems

Funnel-based Reward Shaping for Signal Temporal Logic Tasks in Reinforcement Learning

Signal Temporal Logic (STL) is a powerful framework for describing the complex temporal and logical behaviour of the dynamical system. Numerous studies have attempted to employ reinforcement learning to learn a controller that enforces STL specifications; however, they have been unable to effectively tackle the challenges of ensuring robust satisfaction in continuous state space and maintaining tractability. In this paper, leveraging the concept of funnel functions, we propose a tractable reinforcement learning algorithm to learn a time-dependent policy for robust satisfaction of STL specification in continuous state space. We demonstrate the utility of our approach on several STL tasks using different environments.Comment: 8 pages, 10 figure

Continuous-time control synthesis under nested signal temporal logic specifications

Signal temporal logic (STL) has gained popularity in robotics for expressing complex specifications that may involve timing requirements or deadlines. While the control synthesis for STL specifications without nested temporal operators has been studied in the literature, the case of nested temporal operators is substantially more challenging and requires new theoretical advancements. In this work, we propose an efficient continuous-time control synthesis framework for nonlinear systems under nested STL specifications. The framework is based on the notions of signal temporal logic tree (sTLT) and control barrier function (CBF). In particular, we detail the construction of an sTLT from a given STL formula and a continuous-time dynamical system, the sTLT semantics (i.e., satisfaction condition), and the equivalence or under-approximation relation between sTLT and STL. Leveraging the fact that the satisfaction condition of an sTLT is essentially keeping the state within certain sets during certain time intervals, it provides explicit guidelines for the CBF design. The resulting controller is obtained through the utilization of an online CBF-based program coupled with an event-triggered scheme for online updating the activation time interval of each CBF, with which the correctness of the system behavior can be established by construction. We demonstrate the efficacy of the proposed method for single-integrator and unicycle models under nested STL formulas.Comment: Link to accompanying code: https://github.com/xiaotanKTH/sTL

The 1990 progress report and future plans

This document describes the progress and plans of the Artificial Intelligence Research Branch (RIA) at ARC in 1990. Activities span a range from basic scientific research to engineering development and to fielded NASA applications, particularly those applications that are enabled by basic research carried out at RIA. Work is conducted in-house and through collaborative partners in academia and industry. Our major focus is on a limited number of research themes with a dual commitment to technical excellence and proven applicability to NASA short, medium, and long-term problems. RIA acts as the Agency's lead organization for research aspects of artificial intelligence, working closely with a second research laboratory at JPL and AI applications groups at all NASA centers

Measurements, Models, Systems and Design

531 s.

On Neuromechanical Approaches for the Study of Biological Grasp and Manipulation

Biological and robotic grasp and manipulation are undeniably similar at the level of mechanical task performance. However, their underlying fundamental biological vs. engineering mechanisms are, by definition, dramatically different and can even be antithetical. Even our approach to each is diametrically opposite: inductive science for the study of biological systems vs. engineering synthesis for the design and construction of robotic systems. The past 20 years have seen several conceptual advances in both fields and the quest to unify them. Chief among them is the reluctant recognition that their underlying fundamental mechanisms may actually share limited common ground, while exhibiting many fundamental differences. This recognition is particularly liberating because it allows us to resolve and move beyond multiple paradoxes and contradictions that arose from the initial reasonable assumption of a large common ground. Here, we begin by introducing the perspective of neuromechanics, which emphasizes that real-world behavior emerges from the intimate interactions among the physical structure of the system, the mechanical requirements of a task, the feasible neural control actions to produce it, and the ability of the neuromuscular system to adapt through interactions with the environment. This allows us to articulate a succinct overview of a few salient conceptual paradoxes and contradictions regarding under-determined vs. over-determined mechanics, under- vs. over-actuated control, prescribed vs. emergent function, learning vs. implementation vs. adaptation, prescriptive vs. descriptive synergies, and optimal vs. habitual performance. We conclude by presenting open questions and suggesting directions for future research. We hope this frank assessment of the state-of-the-art will encourage and guide these communities to continue to interact and make progress in these important areas