1,835 research outputs found
StocHy: automated verification and synthesis of stochastic processes
StocHy is a software tool for the quantitative analysis of discrete-time
stochastic hybrid systems (SHS). StocHy accepts a high-level description of
stochastic models and constructs an equivalent SHS model. The tool allows to
(i) simulate the SHS evolution over a given time horizon; and to automatically
construct formal abstractions of the SHS. Abstractions are then employed for
(ii) formal verification or (iii) control (policy, strategy) synthesis. StocHy
allows for modular modelling, and has separate simulation, verification and
synthesis engines, which are implemented as independent libraries. This allows
for libraries to be easily used and for extensions to be easily built. The tool
is implemented in C++ and employs manipulations based on vector calculus, the
use of sparse matrices, the symbolic construction of probabilistic kernels, and
multi-threading. Experiments show StocHy's markedly improved performance when
compared to existing abstraction-based approaches: in particular, StocHy beats
state-of-the-art tools in terms of precision (abstraction error) and
computational effort, and finally attains scalability to large-sized models (12
continuous dimensions). StocHy is available at www.gitlab.com/natchi92/StocHy
Transient Reward Approximation for Continuous-Time Markov Chains
We are interested in the analysis of very large continuous-time Markov chains
(CTMCs) with many distinct rates. Such models arise naturally in the context of
reliability analysis, e.g., of computer network performability analysis, of
power grids, of computer virus vulnerability, and in the study of crowd
dynamics. We use abstraction techniques together with novel algorithms for the
computation of bounds on the expected final and accumulated rewards in
continuous-time Markov decision processes (CTMDPs). These ingredients are
combined in a partly symbolic and partly explicit (symblicit) analysis
approach. In particular, we circumvent the use of multi-terminal decision
diagrams, because the latter do not work well if facing a large number of
different rates. We demonstrate the practical applicability and efficiency of
the approach on two case studies.Comment: Accepted for publication in IEEE Transactions on Reliabilit
Quantitative Approximation of the Probability Distribution of a Markov Process by Formal Abstractions
The goal of this work is to formally abstract a Markov process evolving in
discrete time over a general state space as a finite-state Markov chain, with
the objective of precisely approximating its state probability distribution in
time, which allows for its approximate, faster computation by that of the
Markov chain. The approach is based on formal abstractions and employs an
arbitrary finite partition of the state space of the Markov process, and the
computation of average transition probabilities between partition sets. The
abstraction technique is formal, in that it comes with guarantees on the
introduced approximation that depend on the diameters of the partitions: as
such, they can be tuned at will. Further in the case of Markov processes with
unbounded state spaces, a procedure for precisely truncating the state space
within a compact set is provided, together with an error bound that depends on
the asymptotic properties of the transition kernel of the original process. The
overall abstraction algorithm, which practically hinges on piecewise constant
approximations of the density functions of the Markov process, is extended to
higher-order function approximations: these can lead to improved error bounds
and associated lower computational requirements. The approach is practically
tested to compute probabilistic invariance of the Markov process under study,
and is compared to a known alternative approach from the literature.Comment: 29 pages, Journal of Logical Methods in Computer Scienc
Probabilistic Guarantees for Safe Deep Reinforcement Learning
Deep reinforcement learning has been successfully applied to many control
tasks, but the application of such agents in safety-critical scenarios has been
limited due to safety concerns. Rigorous testing of these controllers is
challenging, particularly when they operate in probabilistic environments due
to, for example, hardware faults or noisy sensors. We propose MOSAIC, an
algorithm for measuring the safety of deep reinforcement learning agents in
stochastic settings. Our approach is based on the iterative construction of a
formal abstraction of a controller's execution in an environment, and leverages
probabilistic model checking of Markov decision processes to produce
probabilistic guarantees on safe behaviour over a finite time horizon. It
produces bounds on the probability of safe operation of the controller for
different initial configurations and identifies regions where correct behaviour
can be guaranteed. We implement and evaluate our approach on agents trained for
several benchmark control problems
Aggregation and Control of Populations of Thermostatically Controlled Loads by Formal Abstractions
This work discusses a two-step procedure, based on formal abstractions, to
generate a finite-space stochastic dynamical model as an aggregation of the
continuous temperature dynamics of a homogeneous population of Thermostatically
Controlled Loads (TCL). The temperature of a single TCL is described by a
stochastic difference equation and the TCL status (ON, OFF) by a deterministic
switching mechanism. The procedure is formal as it allows the exact
quantification of the error introduced by the abstraction -- as such it builds
and improves on a known, earlier approximation technique in the literature.
Further, the contribution discusses the extension to the case of a
heterogeneous population of TCL by means of two approaches resulting in the
notion of approximate abstractions. It moreover investigates the problem of
global (population-level) regulation and load balancing for the case of TCL
that are dependent on a control input. The procedure is tested on a case study
and benchmarked against the mentioned alternative approach in the literature.Comment: 40 pages, 21 figures; the paper generalizes the result of conference
publication: S. Esmaeil Zadeh Soudjani and A. Abate, "Aggregation of
Thermostatically Controlled Loads by Formal Abstractions," Proceedings of the
European Control Conference 2013, pp. 4232-4237. version 2: added references
for section
When are Stochastic Transition Systems Tameable?
A decade ago, Abdulla, Ben Henda and Mayr introduced the elegant concept of
decisiveness for denumerable Markov chains [1]. Roughly speaking, decisiveness
allows one to lift most good properties from finite Markov chains to
denumerable ones, and therefore to adapt existing verification algorithms to
infinite-state models. Decisive Markov chains however do not encompass
stochastic real-time systems, and general stochastic transition systems (STSs
for short) are needed. In this article, we provide a framework to perform both
the qualitative and the quantitative analysis of STSs. First, we define various
notions of decisiveness (inherited from [1]), notions of fairness and of
attractors for STSs, and make explicit the relationships between them. Then, we
define a notion of abstraction, together with natural concepts of soundness and
completeness, and we give general transfer properties, which will be central to
several verification algorithms on STSs. We further design a generic
construction which will be useful for the analysis of {\omega}-regular
properties, when a finite attractor exists, either in the system (if it is
denumerable), or in a sound denumerable abstraction of the system. We next
provide algorithms for qualitative model-checking, and generic approximation
procedures for quantitative model-checking. Finally, we instantiate our
framework with stochastic timed automata (STA), generalized semi-Markov
processes (GSMPs) and stochastic time Petri nets (STPNs), three models
combining dense-time and probabilities. This allows us to derive decidability
and approximability results for the verification of these models. Some of these
results were known from the literature, but our generic approach permits to
view them in a unified framework, and to obtain them with less effort. We also
derive interesting new approximability results for STA, GSMPs and STPNs.Comment: 77 page
- …