8 research outputs found
Technical Report: A Receding Horizon Algorithm for Informative Path Planning with Temporal Logic Constraints
This technical report is an extended version of the paper 'A Receding Horizon
Algorithm for Informative Path Planning with Temporal Logic Constraints'
accepted to the 2013 IEEE International Conference on Robotics and Automation
(ICRA). This paper considers the problem of finding the most informative path
for a sensing robot under temporal logic constraints, a richer set of
constraints than have previously been considered in information gathering. An
algorithm for informative path planning is presented that leverages tools from
information theory and formal control synthesis, and is proven to give a path
that satisfies the given temporal logic constraints. The algorithm uses a
receding horizon approach in order to provide a reactive, on-line solution
while mitigating computational complexity. Statistics compiled from multiple
simulation studies indicate that this algorithm performs better than a baseline
exhaustive search approach.Comment: Extended version of paper accepted to 2013 IEEE International
Conference on Robotics and Automation (ICRA
Reinforcement Learning With Temporal Logic Rewards
Reinforcement learning (RL) depends critically on the choice of reward
functions used to capture the de- sired behavior and constraints of a robot.
Usually, these are handcrafted by a expert designer and represent heuristics
for relatively simple tasks. Real world applications typically involve more
complex tasks with rich temporal and logical structure. In this paper we take
advantage of the expressive power of temporal logic (TL) to specify complex
rules the robot should follow, and incorporate domain knowledge into learning.
We propose Truncated Linear Temporal Logic (TLTL) as specifications language,
that is arguably well suited for the robotics applications, together with
quantitative semantics, i.e., robustness degree. We propose a RL approach to
learn tasks expressed as TLTL formulae that uses their associated robustness
degree as reward functions, instead of the manually crafted heuristics trying
to capture the same specifications. We show in simulated trials that learning
is faster and policies obtained using the proposed approach outperform the ones
learned using heuristic rewards in terms of the robustness degree, i.e., how
well the tasks are satisfied. Furthermore, we demonstrate the proposed RL
approach in a toast-placing task learned by a Baxter robot
Lazy Abstraction-Based Controller Synthesis
We present lazy abstraction-based controller synthesis (ABCS) for
continuous-time nonlinear dynamical systems against reach-avoid and safety
specifications. State-of-the-art multi-layered ABCS pre-computes multiple
finite-state abstractions of varying granularity and applies reactive synthesis
to the coarsest abstraction whenever feasible, but adaptively considers finer
abstractions when necessary. Lazy ABCS improves this technique by constructing
abstractions on demand. Our insight is that the abstract transition relation
only needs to be locally computed for a small set of frontier states at the
precision currently required by the synthesis algorithm. We show that lazy ABCS
can significantly outperform previous multi-layered ABCS algorithms: on
standard benchmarks, lazy ABCS is more than 4 times faster
Optimal Receding Horizon Control for Finite Deterministic Systems with Temporal Logic Constraints
In this paper, we develop a provably correct optimal control strategy for a
finite deterministic transition system. By assuming that penalties with known
probabilities of occurrence and dynamics can be sensed locally at the states of
the system, we derive a receding horizon strategy that minimizes the expected
average cumulative penalty incurred between two consecutive satisfactions of a
desired property. At the same time, we guarantee the satisfaction of
correctness specifications expressed as Linear Temporal Logic formulas. We
illustrate the approach with a persistent surveillance robotics application.Comment: Technical report accompanying the ACC 2013 pape
Temporal Logic Control for Stochastic Linear Systems using Abstraction Refinement of Probabilistic Games
We consider the problem of computing the set of initial states of a dynamical
system such that there exists a control strategy to ensure that the
trajectories satisfy a temporal logic specification with probability 1
(almost-surely). We focus on discrete-time, stochastic linear dynamics and
specifications given as formulas of the Generalized Reactivity(1) fragment of
Linear Temporal Logic over linear predicates in the states of the system. We
propose a solution based on iterative abstraction-refinement, and turn-based
2-player probabilistic games. While the theoretical guarantee of our algorithm
after any finite number of iterations is only a partial solution, we show that
if our algorithm terminates, then the result is the set of satisfying initial
states. Moreover, for any (partial) solution our algorithm synthesizes witness
control strategies to ensure almost-sure satisfaction of the temporal logic
specification. We demonstrate our approach on an illustrative case study.Comment: Technical report accompanying HSCC'15 pape
Formal methods paradigms for estimation and machine learning in dynamical systems
Formal methods are widely used in engineering to determine whether a system exhibits a certain property (verification) or to design controllers that are guaranteed to drive the system to achieve a certain property (synthesis). Most existing techniques require a large amount of accurate information about the system in order to be successful. The methods presented in this work can operate with significantly less prior information. In the domain of formal synthesis for robotics, the assumptions of perfect sensing and perfect knowledge of system dynamics are unrealistic. To address this issue, we present control algorithms that use active estimation and reinforcement learning to mitigate the effects of uncertainty. In the domain of cyber-physical system analysis, we relax the assumption that the system model is known and identify system properties automatically from execution data.
First, we address the problem of planning the path of a robot under temporal logic constraints (e.g. "avoid obstacles and periodically visit a recharging station") while simultaneously minimizing the uncertainty about the state of an unknown feature of the environment (e.g. locations of fires after a natural disaster). We present synthesis algorithms and evaluate them via simulation and experiments with aerial robots. Second, we develop a new specification language for tasks that require gathering information about and interacting with a partially observable environment, e.g. "Maintain localization error below a certain level while also avoiding obstacles.'' Third, we consider learning temporal logic properties of a dynamical system from a finite set of system outputs. For example, given maritime surveillance data we wish to find the specification that corresponds only to those vessels that are deemed law-abiding. Algorithms for performing off-line supervised and unsupervised learning and on-line supervised learning are presented. Finally, we consider the case in which we want to steer a system with unknown dynamics to satisfy a given temporal logic specification. We present a novel reinforcement learning paradigm to solve this problem. Our procedure gives "partial credit'' for executions that almost satisfy the specification, which can
lead to faster convergence rates and produce better solutions when the specification is not satisfiable
Language guided controller synthesis for discrete time linear systems
This paper considers the problem of controlling discrete-time linear systems from specifications given as formulas of syntactically co-safe linear temporal logic over linear predicates in the state variables of the system. A systematic procedure is developed for the automatic computation of sets of initial states and feedback controllers such that all the resulting trajectories of the corresponding closed-loop system satisfy the given specifications. The procedure is based on the iterative construction and refinement of an automaton that enforces the satisfaction of the formula. Interpolation and polyhedral Lyapunov function based approaches are proposed to compute the polytope-to-polytope controllers that label the transitions of the automaton. The algorithms developed in this paper were implemented as a software package that is available for download. Their application and effectiveness are demonstrated for two challenging case studies