17 research outputs found
A Verification Framework for Fictitious Play Based Learning Algorithms
Distributed optimisation techniques have gained increasing attention due to fast development of autonomous robots. Many algorithms have been proposed to make optimisation more efficient. In this paper we propose a framework, which is based on probabilistic verification techniques, in order to compare the performance of various game-theoretic algorithms, in particular, fictitious play and its variants, after a finite number of iterations. To demonstrate the effectiveness of the framework, we apply the framework to a game which is inspired by wireless communication network problems, on five variations of fictitious play algorithms
Logic and model checking for hidden Markov models
The branching-time temporal logic PCTL* has been introduced to specify quantitative properties over probability systems, such as discrete-time Markov chains. Until now, however, no logics have been defined to specify properties over hidden Markov models (HMMs). In HMMs the states are hidden, and the hidden processes produce a sequence of observations. In this paper we extend the logic PCTL* to POCTL*. With our logic one can state properties such as "there is at least a 90 percent probability that the model produces a given sequence of observations" over HMMs. Subsequently, we give model checking algorithms for POCTL* over HMMs
A Markov Chain Model Checker
Markov chains are widely used in the context of performance and reliability evaluation of systems of various nature. Model checking of such chains with respect to a given (branching) temporal logic formula has been proposed for both the discrete [17,6] and the continuous time setting [4,8]. In this paper, we describe a prototype model checker for discrete and continuous-time Markov chains, the Erlangen Twente Markov Chain Checker ), where properties are expressed in appropriate extensions of CTL. We illustrate the general bene ts of this approach and discuss the structure of the tool. Furthermore we report on first successful applications of the tool to non-trivial examples, highlighting lessons learned during development and application of )
Real-time and Probabilistic Temporal Logics: An Overview
Over the last two decades, there has been an extensive study on logical
formalisms for specifying and verifying real-time systems. Temporal logics have
been an important research subject within this direction. Although numerous
logics have been introduced for the formal specification of real-time and
complex systems, an up to date comprehensive analysis of these logics does not
exist in the literature. In this paper we analyse real-time and probabilistic
temporal logics which have been widely used in this field. We extrapolate the
notions of decidability, axiomatizability, expressiveness, model checking, etc.
for each logic analysed. We also provide a comparison of features of the
temporal logics discussed
Decisive Markov Chains
We consider qualitative and quantitative verification problems for
infinite-state Markov chains. We call a Markov chain decisive w.r.t. a given
set of target states F if it almost certainly eventually reaches either F or a
state from which F can no longer be reached. While all finite Markov chains are
trivially decisive (for every set F), this also holds for many classes of
infinite Markov chains. Infinite Markov chains which contain a finite attractor
are decisive w.r.t. every set F. In particular, this holds for probabilistic
lossy channel systems (PLCS). Furthermore, all globally coarse Markov chains
are decisive. This class includes probabilistic vector addition systems (PVASS)
and probabilistic noisy Turing machines (PNTM). We consider both safety and
liveness problems for decisive Markov chains, i.e., the probabilities that a
given set of states F is eventually reached or reached infinitely often,
respectively. 1. We express the qualitative problems in abstract terms for
decisive Markov chains, and show an almost complete picture of its decidability
for PLCS, PVASS and PNTM. 2. We also show that the path enumeration algorithm
of Iyer and Narasimha terminates for decisive Markov chains and can thus be
used to solve the approximate quantitative safety problem. A modified variant
of this algorithm solves the approximate quantitative liveness problem. 3.
Finally, we show that the exact probability of (repeatedly) reaching F cannot
be effectively expressed (in a uniform way) in Tarski-algebra for either PLCS,
PVASS or (P)NTM.Comment: 32 pages, 0 figure
Model Checking Probabilistic Pushdown Automata
We consider the model checking problem for probabilistic pushdown automata
(pPDA) and properties expressible in various probabilistic logics. We start
with properties that can be formulated as instances of a generalized random
walk problem. We prove that both qualitative and quantitative model checking
for this class of properties and pPDA is decidable. Then we show that model
checking for the qualitative fragment of the logic PCTL and pPDA is also
decidable. Moreover, we develop an error-tolerant model checking algorithm for
PCTL and the subclass of stateless pPDA. Finally, we consider the class of
omega-regular properties and show that both qualitative and quantitative model
checking for pPDA is decidable
Approximation Techniques for Stochastic Analysis of Biological Systems
There has been an increasing demand for formal methods in the design process
of safety-critical synthetic genetic circuits. Probabilistic model checking
techniques have demonstrated significant potential in analyzing the intrinsic
probabilistic behaviors of complex genetic circuit designs. However, its
inability to scale limits its applicability in practice. This chapter addresses
the scalability problem by presenting a state-space approximation method to
remove unlikely states resulting in a reduced, finite state representation of
the infinite-state continuous-time Markov chain that is amenable to
probabilistic model checking. The proposed method is evaluated on a design of a
genetic toggle switch. Comparisons with another state-of-art tool demonstrates
both accuracy and efficiency of the presented method
Finite-State Abstractions for Probabilistic Computation Tree Logic
Probabilistic Computation Tree Logic (PCTL) is the established temporal
logic for probabilistic verification of discrete-time Markov chains. Probabilistic
model checking is a technique that verifies or refutes whether a property
specified in this logic holds in a Markov chain. But Markov chains are often
infinite or too large for this technique to apply. A standard solution to
this problem is to convert the Markov chain to an abstract model and to
model check that abstract model. The problem this thesis therefore studies
is whether or when such finite abstractions of Markov chains for model
checking PCTL exist.
This thesis makes the following contributions. We identify a sizeable fragment
of PCTL for which 3-valued Markov chains can serve as finite abstractions;
this fragment is maximal for those abstractions and subsumes many
practically relevant specifications including, e.g., reachability. We also develop
game-theoretic foundations for the semantics of PCTL over Markov
chains by capturing the standard PCTL semantics via a two-player games.
These games, finally, inspire a notion of p-automata, which accept entire
Markov chains. We show that p-automata subsume PCTL and Markov
chains; that their languages of Markov chains have pleasant closure properties;
and that the complexity of deciding acceptance matches that of probabilistic
model checking for p-automata representing PCTL formulae. In addition,
we offer a simulation between p-automata that under-approximates
language containment. These results then allow us to show that p-automata
comprise a solution to the problem studied in this thesis