1,086 research outputs found
Language-based Abstractions for Dynamical Systems
Ordinary differential equations (ODEs) are the primary means to modelling
dynamical systems in many natural and engineering sciences. The number of
equations required to describe a system with high heterogeneity limits our
capability of effectively performing analyses. This has motivated a large body
of research, across many disciplines, into abstraction techniques that provide
smaller ODE systems while preserving the original dynamics in some appropriate
sense. In this paper we give an overview of a recently proposed
computer-science perspective to this problem, where ODE reduction is recast to
finding an appropriate equivalence relation over ODE variables, akin to
classical models of computation based on labelled transition systems.Comment: In Proceedings QAPL 2017, arXiv:1707.0366
Performance Guarantees for Homomorphisms Beyond Markov Decision Processes
Most real-world problems have huge state and/or action spaces. Therefore, a
naive application of existing tabular solution methods is not tractable on such
problems. Nonetheless, these solution methods are quite useful if an agent has
access to a relatively small state-action space homomorphism of the true
environment and near-optimal performance is guaranteed by the map. A plethora
of research is focused on the case when the homomorphism is a Markovian
representation of the underlying process. However, we show that near-optimal
performance is sometimes guaranteed even if the homomorphism is non-Markovian.
Moreover, we can aggregate significantly more states by lifting the Markovian
requirement without compromising on performance. In this work, we expand
Extreme State Aggregation (ESA) framework to joint state-action aggregations.
We also lift the policy uniformity condition for aggregation in ESA that allows
even coarser modeling of the true environment
Robust satisfaction of temporal logic specifications via reinforcement learning
We consider the problem of steering a system with unknown, stochastic dynamics to satisfy a rich, temporally-layered task given as a signal temporal logic formula. We represent the system as a finite-memory Markov decision process with unknown transition probabilities and whose states are built from a partition of the state space. We present provably convergent reinforcement learning algorithms to maximize the probability of satisfying a given specification and to maximize the average expected robustness, i.e. a measure of how strongly the formula is satisfied. Robustness allows us to quantify progress towards satisfying a given specification. We demonstrate via a pair of robot navigation simulation case studies that, due to the quantification of progress towards satisfaction, reinforcement learning with robustness maximization performs better than probability maximization in terms of both probability of satisfaction and expected robustness with a low number of training examples
Robust Satisfaction of Temporal Logic Specifications via Reinforcement Learning
We consider the problem of steering a system with unknown, stochastic
dynamics to satisfy a rich, temporally layered task given as a signal temporal
logic formula. We represent the system as a Markov decision process in which
the states are built from a partition of the state space and the transition
probabilities are unknown. We present provably convergent reinforcement
learning algorithms to maximize the probability of satisfying a given formula
and to maximize the average expected robustness, i.e., a measure of how
strongly the formula is satisfied. We demonstrate via a pair of robot
navigation simulation case studies that reinforcement learning with robustness
maximization performs better than probability maximization in terms of both
probability of satisfaction and expected robustness.Comment: 8 pages, 4 figure
Compositional Verification and Optimization of Interactive Markov Chains
Interactive Markov chains (IMC) are compositional behavioural models
extending labelled transition systems and continuous-time Markov chains. We
provide a framework and algorithms for compositional verification and
optimization of IMC with respect to time-bounded properties. Firstly, we give a
specification formalism for IMC. Secondly, given a time-bounded property, an
IMC component and the assumption that its unknown environment satisfies a given
specification, we synthesize a scheduler for the component optimizing the
probability that the property is satisfied in any such environment
When are Stochastic Transition Systems Tameable?
A decade ago, Abdulla, Ben Henda and Mayr introduced the elegant concept of
decisiveness for denumerable Markov chains [1]. Roughly speaking, decisiveness
allows one to lift most good properties from finite Markov chains to
denumerable ones, and therefore to adapt existing verification algorithms to
infinite-state models. Decisive Markov chains however do not encompass
stochastic real-time systems, and general stochastic transition systems (STSs
for short) are needed. In this article, we provide a framework to perform both
the qualitative and the quantitative analysis of STSs. First, we define various
notions of decisiveness (inherited from [1]), notions of fairness and of
attractors for STSs, and make explicit the relationships between them. Then, we
define a notion of abstraction, together with natural concepts of soundness and
completeness, and we give general transfer properties, which will be central to
several verification algorithms on STSs. We further design a generic
construction which will be useful for the analysis of {\omega}-regular
properties, when a finite attractor exists, either in the system (if it is
denumerable), or in a sound denumerable abstraction of the system. We next
provide algorithms for qualitative model-checking, and generic approximation
procedures for quantitative model-checking. Finally, we instantiate our
framework with stochastic timed automata (STA), generalized semi-Markov
processes (GSMPs) and stochastic time Petri nets (STPNs), three models
combining dense-time and probabilities. This allows us to derive decidability
and approximability results for the verification of these models. Some of these
results were known from the literature, but our generic approach permits to
view them in a unified framework, and to obtain them with less effort. We also
derive interesting new approximability results for STA, GSMPs and STPNs.Comment: 77 page
Formal Controller Synthesis for Markov Jump Linear Systems with Uncertain Dynamics
Automated synthesis of provably correct controllers for cyber-physical
systems is crucial for deployment in safety-critical scenarios. However, hybrid
features and stochastic or unknown behaviours make this problem challenging. We
propose a method for synthesising controllers for Markov jump linear systems
(MJLSs), a class of discrete-time models for cyber-physical systems, so that
they certifiably satisfy probabilistic computation tree logic (PCTL) formulae.
An MJLS consists of a finite set of stochastic linear dynamics and discrete
jumps between these dynamics that are governed by a Markov decision process
(MDP). We consider the cases where the transition probabilities of this MDP are
either known up to an interval or completely unknown. Our approach is based on
a finite-state abstraction that captures both the discrete (mode-jumping) and
continuous (stochastic linear) behaviour of the MJLS. We formalise this
abstraction as an interval MDP (iMDP) for which we compute intervals of
transition probabilities using sampling techniques from the so-called 'scenario
approach', resulting in a probabilistically sound approximation. We apply our
method to multiple realistic benchmark problems, in particular, a temperature
control and an aerial vehicle delivery problem.Comment: 14 pages, 6 figures, under review at QES
- …