Computing Quantiles in Markov Reward Models

Probabilistic model checking mainly concentrates on techniques for reasoning about the probabilities of certain path properties or expected values of certain random variables. For the quantitative system analysis, however, there is also another type of interesting performance measure, namely quantiles. A typical quantile query takes as input a lower probability bound p and a reachability property. The task is then to compute the minimal reward bound r such that with probability at least p the target set will be reached before the accumulated reward exceeds r. Quantiles are well-known from mathematical statistics, but to the best of our knowledge they have not been addressed by the model checking community so far. In this paper, we study the complexity of quantile queries for until properties in discrete-time finite-state Markov decision processes with non-negative rewards on states. We show that qualitative quantile queries can be evaluated in polynomial time and present an exponential algorithm for the evaluation of quantitative quantile queries. For the special case of Markov chains, we show that quantitative quantile queries can be evaluated in time polynomial in the size of the chain and the maximum reward.Comment: 17 pages, 1 figure; typo in example correcte

Qualitative Logics and Equivalences for Probabilistic Systems

We investigate logics and equivalence relations that capture the qualitative behavior of Markov Decision Processes (MDPs). We present Qualitative Randomized CTL (QRCTL): formulas of this logic can express the fact that certain temporal properties hold over all paths, or with probability 0 or 1, but they do not distinguish among intermediate probability values. We present a symbolic, polynomial time model-checking algorithm for QRCTL on MDPs. The logic QRCTL induces an equivalence relation over states of an MDP that we call qualitative equivalence: informally, two states are qualitatively equivalent if the sets of formulas that hold with probability 0 or 1 at the two states are the same. We show that for finite alternating MDPs, where nondeterministic and probabilistic choices occur in different states, qualitative equivalence coincides with alternating bisimulation, and can thus be computed via efficient partition-refinement algorithms. On the other hand, in non-alternating MDPs the equivalence relations cannot be computed via partition-refinement algorithms, but rather, they require non-local computation. Finally, we consider QRCTL*, that extends QRCTL with nested temporal operators in the same manner in which CTL* extends CTL. We show that QRCTL and QRCTL* induce the same qualitative equivalence on alternating MDPs, while on non-alternating MDPs, the equivalence arising from QRCTL* can be strictly finer. We also provide a full characterization of the relation between qualitative equivalence, bisimulation, and alternating bisimulation, according to whether the MDPs are finite, and to whether their transition relations are finitely-branching.Comment: The paper is accepted for LMC

A formal language towards the unification of model checking and performance evaluation

In computer science, model checking refers to a computation process that, given a formal structure, checks whether the structure satisfies a logic formula which encodes certain properties. If the structure is a discrete state system and the interested properties depend only on which states to be reached, not on the time or probability to reach them, traditional temporal logics such as linear temporal logic (LTL) and computation tree logic (CTL) are powerful mathematical formalisms that can express properties such as \u27\u27no collision shall occur in a traffic light control system\u27\u27, or \u27\u27eventually, a service is completed\u27\u27. To express performance-dependability related properties over discrete state stochastic systems, these logics have evolved into quantitative model checking logics such as probabilistic linear temporal logic (PLTL), probabilistic computation tree logic (PCTL), and computation tree stochastic logic (CSL), etc., and can express properties such as ``with probability at least 0.98, the system will not reach a deadlock state before time 100\u27\u27. While these logics and their model checking algorithms are powerful, they are inadequate in expressing complex performance measures, either because they are limited to producing only true/false responses (although in practice, a real valued response can sometimes be obtained for the outer-most path quantifier), or the computational complexity is too expensive to be practical. To address these limitations, for this PhD work, we propose a novel mechanism with the following research aims: 1) Define general specification formalisms to express performance queries in real values while retaining the ability to express temporal properties. 2) Develop efficient mathematical algorithms for the proposed formalisms. 3)Implement the approach in tools and experiment on large-scaled Markov models for the analysis of example queries

Markovian Processes for Quantitative Information Leakage

Quantification of information leakage is a successful approach for evaluating the security of a system. It models the system to be analyzed as a channel with the secret as the input and an output as observable by the attacker as the output, and applies information theory to quantify the amount of information transmitted through such channel, thus effectively quantifying how many bits of the secret can be inferred by the attacker by analyzing the system’s output.Channels are usually encoded as matrices of conditional probabilities, known as channel matrices. Such matrices grow exponentially in the size of the secret and observables, are cumbersome to compute and store, encode both the behavior of the system and assumptions about the attacker, and assume an input-output behavior of the system. For these reasons we propose to model the system-attacker scenario with Markovian models.We show that such models are more compact and treatable than channel matrices. Also, they clearly separate the behavior of the system from the assumptions about the attacker, and can represent even non-terminating behavior in a finite model. We provide techniques and algorithms to model and analyze both deterministic and randomized processes with Markovian models and to compute their informationleakage for a very general model of attacker. We present the QUAIL tool that automates such analysis and is able to compute the information leakage of an imperative WHILE language. Finally, we show how to use QUAIL to analyze some interesting cases of secret-dependent protocols

Markovian Processes for Quantitative Information Leakage

Quantification of information leakage is a successful approach for evaluating the security of a system. It models the system to be analyzed as a channel with the secret as the input and an output as observable by the attacker as the output, and applies information theory to quantify the amount of information transmitted through such channel, thus effectively quantifying how many bits of the secret can be inferred by the attacker by analyzing the system’s output.Channels are usually encoded as matrices of conditional probabilities, known as channel matrices. Such matrices grow exponentially in the size of the secret and observables, are cumbersome to compute and store, encode both the behavior of the system and assumptions about the attacker, and assume an input-output behavior of the system. For these reasons we propose to model the system-attacker scenario with Markovian models.We show that such models are more compact and treatable than channel matrices. Also, they clearly separate the behavior of the system from the assumptions about the attacker, and can represent even non-terminating behavior in a finite model. We provide techniques and algorithms to model and analyze both deterministic and randomized processes with Markovian models and to compute their informationleakage for a very general model of attacker. We present the QUAIL tool that automates such analysis and is able to compute the information leakage of an imperative WHILE language. Finally, we show how to use QUAIL to analyze some interesting cases of secret-dependent protocols