1,071 research outputs found
Qualitative Reachability for Open Interval Markov Chains
Interval Markov chains extend classical Markov chains with the possibility to
describe transition probabilities using intervals, rather than exact values.
While the standard formulation of interval Markov chains features closed
intervals, previous work has considered also open interval Markov chains, in
which the intervals can also be open or half-open. In this paper we focus on
qualitative reachability problems for open interval Markov chains, which
consider whether the optimal (maximum or minimum) probability with which a
certain set of states can be reached is equal to 0 or 1. We present
polynomial-time algorithms for these problems for both of the standard
semantics of interval Markov chains. Our methods do not rely on the closure of
open intervals, in contrast to previous approaches for open interval Markov
chains, and can characterise situations in which probability 0 or 1 can be
attained not exactly but arbitrarily closely.Comment: Full version of a paper published at RP 201
Reachability in Parametric Interval Markov Chains using Constraints
Parametric Interval Markov Chains (pIMCs) are a specification formalism that
extend Markov Chains (MCs) and Interval Markov Chains (IMCs) by taking into
account imprecision in the transition probability values: transitions in pIMCs
are labeled with parametric intervals of probabilities. In this work, we study
the difference between pIMCs and other Markov Chain abstractions models and
investigate the two usual semantics for IMCs: once-and-for-all and
at-every-step. In particular, we prove that both semantics agree on the
maximal/minimal reachability probabilities of a given IMC. We then investigate
solutions to several parameter synthesis problems in the context of pIMCs --
consistency, qualitative reachability and quantitative reachability -- that
rely on constraint encodings. Finally, we propose a prototype implementation of
our constraint encodings with promising results
When are Stochastic Transition Systems Tameable?
A decade ago, Abdulla, Ben Henda and Mayr introduced the elegant concept of
decisiveness for denumerable Markov chains [1]. Roughly speaking, decisiveness
allows one to lift most good properties from finite Markov chains to
denumerable ones, and therefore to adapt existing verification algorithms to
infinite-state models. Decisive Markov chains however do not encompass
stochastic real-time systems, and general stochastic transition systems (STSs
for short) are needed. In this article, we provide a framework to perform both
the qualitative and the quantitative analysis of STSs. First, we define various
notions of decisiveness (inherited from [1]), notions of fairness and of
attractors for STSs, and make explicit the relationships between them. Then, we
define a notion of abstraction, together with natural concepts of soundness and
completeness, and we give general transfer properties, which will be central to
several verification algorithms on STSs. We further design a generic
construction which will be useful for the analysis of {\omega}-regular
properties, when a finite attractor exists, either in the system (if it is
denumerable), or in a sound denumerable abstraction of the system. We next
provide algorithms for qualitative model-checking, and generic approximation
procedures for quantitative model-checking. Finally, we instantiate our
framework with stochastic timed automata (STA), generalized semi-Markov
processes (GSMPs) and stochastic time Petri nets (STPNs), three models
combining dense-time and probabilities. This allows us to derive decidability
and approximability results for the verification of these models. Some of these
results were known from the literature, but our generic approach permits to
view them in a unified framework, and to obtain them with less effort. We also
derive interesting new approximability results for STA, GSMPs and STPNs.Comment: 77 page
PrIC3: Property Directed Reachability for MDPs
IC3 has been a leap forward in symbolic model checking. This paper proposes
PrIC3 (pronounced pricy-three), a conservative extension of IC3 to symbolic
model checking of MDPs. Our main focus is to develop the theory underlying
PrIC3. Alongside, we present a first implementation of PrIC3 including the key
ingredients from IC3 such as generalization, repushing, and propagation
Qualitative Analysis of Concurrent Mean-payoff Games
We consider concurrent games played by two-players on a finite-state graph,
where in every round the players simultaneously choose a move, and the current
state along with the joint moves determine the successor state. We study a
fundamental objective, namely, mean-payoff objective, where a reward is
associated to each transition, and the goal of player 1 is to maximize the
long-run average of the rewards, and the objective of player 2 is strictly the
opposite. The path constraint for player 1 could be qualitative, i.e., the
mean-payoff is the maximal reward, or arbitrarily close to it; or quantitative,
i.e., a given threshold between the minimal and maximal reward. We consider the
computation of the almost-sure (resp. positive) winning sets, where player 1
can ensure that the path constraint is satisfied with probability 1 (resp.
positive probability). Our main results for qualitative path constraints are as
follows: (1) we establish qualitative determinacy results that show that for
every state either player 1 has a strategy to ensure almost-sure (resp.
positive) winning against all player-2 strategies, or player 2 has a spoiling
strategy to falsify almost-sure (resp. positive) winning against all player-1
strategies; (2) we present optimal strategy complexity results that precisely
characterize the classes of strategies required for almost-sure and positive
winning for both players; and (3) we present quadratic time algorithms to
compute the almost-sure and the positive winning sets, matching the best known
bound of algorithms for much simpler problems (such as reachability
objectives). For quantitative constraints we show that a polynomial time
solution for the almost-sure or the positive winning set would imply a solution
to a long-standing open problem (the value problem for turn-based deterministic
mean-payoff games) that is not known to be solvable in polynomial time
Petri nets for systems and synthetic biology
We give a description of a Petri net-based framework for
modelling and analysing biochemical pathways, which uni¯es the qualita-
tive, stochastic and continuous paradigms. Each perspective adds its con-
tribution to the understanding of the system, thus the three approaches
do not compete, but complement each other. We illustrate our approach
by applying it to an extended model of the three stage cascade, which
forms the core of the ERK signal transduction pathway. Consequently
our focus is on transient behaviour analysis. We demonstrate how quali-
tative descriptions are abstractions over stochastic or continuous descrip-
tions, and show that the stochastic and continuous models approximate
each other. Although our framework is based on Petri nets, it can be
applied more widely to other formalisms which are used to model and
analyse biochemical networks
Compositional Verification and Optimization of Interactive Markov Chains
Interactive Markov chains (IMC) are compositional behavioural models
extending labelled transition systems and continuous-time Markov chains. We
provide a framework and algorithms for compositional verification and
optimization of IMC with respect to time-bounded properties. Firstly, we give a
specification formalism for IMC. Secondly, given a time-bounded property, an
IMC component and the assumption that its unknown environment satisfies a given
specification, we synthesize a scheduler for the component optimizing the
probability that the property is satisfied in any such environment
On Zone-Based Analysis of Duration Probabilistic Automata
We propose an extension of the zone-based algorithmics for analyzing timed
automata to handle systems where timing uncertainty is considered as
probabilistic rather than set-theoretic. We study duration probabilistic
automata (DPA), expressing multiple parallel processes admitting memoryfull
continuously-distributed durations. For this model we develop an extension of
the zone-based forward reachability algorithm whose successor operator is a
density transformer, thus providing a solution to verification and performance
evaluation problems concerning acyclic DPA (or the bounded-horizon behavior of
cyclic DPA).Comment: In Proceedings INFINITY 2010, arXiv:1010.611
Ergodicity of the zigzag process
The zigzag process is a Piecewise Deterministic Markov Process which can be
used in a MCMC framework to sample from a given target distribution. We prove
the convergence of this process to its target under very weak assumptions, and
establish a central limit theorem for empirical averages under stronger
assumptions on the decay of the target measure. We use the classical
"Meyn-Tweedie" approach. The main difficulty turns out to be the proof that the
process can indeed reach all the points in the space, even if we consider the
minimal switching rates
Conditional Value-at-Risk for Reachability and Mean Payoff in Markov Decision Processes
We present the conditional value-at-risk (CVaR) in the context of Markov
chains and Markov decision processes with reachability and mean-payoff
objectives. CVaR quantifies risk by means of the expectation of the worst
p-quantile. As such it can be used to design risk-averse systems. We consider
not only CVaR constraints, but also introduce their conjunction with
expectation constraints and quantile constraints (value-at-risk, VaR). We
derive lower and upper bounds on the computational complexity of the respective
decision problems and characterize the structure of the strategies in terms of
memory and randomization
- …