4,330 research outputs found
Probabilistic Bisimulation: Naturally on Distributions
In contrast to the usual understanding of probabilistic systems as stochastic
processes, recently these systems have also been regarded as transformers of
probabilities. In this paper, we give a natural definition of strong
bisimulation for probabilistic systems corresponding to this view that treats
probability distributions as first-class citizens. Our definition applies in
the same way to discrete systems as well as to systems with uncountable state
and action spaces. Several examples demonstrate that our definition refines the
understanding of behavioural equivalences of probabilistic systems. In
particular, it solves a long-standing open problem concerning the
representation of memoryless continuous time by memory-full continuous time.
Finally, we give algorithms for computing this bisimulation not only for finite
but also for classes of uncountably infinite systems
Advancing Dynamic Fault Tree Analysis
This paper presents a new state space generation approach for dynamic fault
trees (DFTs) together with a technique to synthesise failures rates in DFTs.
Our state space generation technique aggressively exploits the DFT structure
--- detecting symmetries, spurious non-determinism, and don't cares. Benchmarks
show a gain of more than two orders of magnitude in terms of state space
generation and analysis time. Our approach supports DFTs with symbolic failure
rates and is complemented by parameter synthesis. This enables determining the
maximal tolerable failure rate of a system component while ensuring that the
mean time of failure stays below a threshold
Viewpoint Development of Stochastic Hybrid Systems
Nowadays, due to the explosive spreading of networked and highly distributed systems, mastering system complexity becomes a critical issue. Two development and verification paradigms have become more popular: viewpoints and randomisation. The viewpoints offer large freedom and introduce concurrency and compositionality in the development process. Randomisation is now a traditional method for reducing complexity (comparing with deterministic models) and it offers finer analytical analysis tools (quantification over non-determinism, multi-valued logics, etc). In this paper, we propose a combination of these two paradigms introducing a viewpoint methodology for systems with stochastic behaviours
Process algebra for performance evaluation
This paper surveys the theoretical developments in the field of stochastic process algebras, process algebras where action occurrences may be subject to a delay that is determined by a random variable. A huge class of resource-sharing systems – like large-scale computers, client–server architectures, networks – can accurately be described using such stochastic specification formalisms. The main emphasis of this paper is the treatment of operational semantics, notions of equivalence, and (sound and complete) axiomatisations of these equivalences for different types of Markovian process algebras, where delays are governed by exponential distributions. Starting from a simple actionless algebra for describing time-homogeneous continuous-time Markov chains, we consider the integration of actions and random delays both as a single entity (like in known Markovian process algebras like TIPP, PEPA and EMPA) and as separate entities (like in the timed process algebras timed CSP and TCCS). In total we consider four related calculi and investigate their relationship to existing Markovian process algebras. We also briefly indicate how one can profit from the separation of time and actions when incorporating more general, non-Markovian distributions
Markov Chains and Dynamical Systems: The Open System Point of View
This article presents several results establishing connections be- tween
Markov chains and dynamical systems, from the point of view of open systems in
physics. We show how all Markov chains can be understood as the information on
one component that we get from a dynamical system on a product system, when
losing information on the other component. We show that passing from the
deterministic dynamics to the random one is character- ized by the loss of
algebra morphism property; it is also characterized by the loss of
reversibility. In the continuous time framework, we show that the solu- tions
of stochastic dierential equations are actually deterministic dynamical systems
on a particular product space. When losing the information on one component, we
recover the usual associated Markov semigroup
Abstractions of Stochastic Hybrid Systems
In this paper we define a stochastic bisimulation concept for a very general class of stochastic hybrid systems, which subsumes most classes of stochastic hybrid systems. The definition of this bisimulation builds on the concept of zigzag morphism defined for strong Markov processes.
The main result is that this stochastic bisimulation is indeed an equivalence relation. The secondary result is that this bisimulation relation for the stochastic hybrid system models used in this paper implies the same
kind of bisimulation for their continuous parts and respectively for their jumping structures
Certified Reinforcement Learning with Logic Guidance
This paper proposes the first model-free Reinforcement Learning (RL)
framework to synthesise policies for unknown, and continuous-state Markov
Decision Processes (MDPs), such that a given linear temporal property is
satisfied. We convert the given property into a Limit Deterministic Buchi
Automaton (LDBA), namely a finite-state machine expressing the property.
Exploiting the structure of the LDBA, we shape a synchronous reward function
on-the-fly, so that an RL algorithm can synthesise a policy resulting in traces
that probabilistically satisfy the linear temporal property. This probability
(certificate) is also calculated in parallel with policy learning when the
state space of the MDP is finite: as such, the RL algorithm produces a policy
that is certified with respect to the property. Under the assumption of finite
state space, theoretical guarantees are provided on the convergence of the RL
algorithm to an optimal policy, maximising the above probability. We also show
that our method produces ''best available'' control policies when the logical
property cannot be satisfied. In the general case of a continuous state space,
we propose a neural network architecture for RL and we empirically show that
the algorithm finds satisfying policies, if there exist such policies. The
performance of the proposed framework is evaluated via a set of numerical
examples and benchmarks, where we observe an improvement of one order of
magnitude in the number of iterations required for the policy synthesis,
compared to existing approaches whenever available.Comment: This article draws from arXiv:1801.08099, arXiv:1809.0782
- …