Extending the Logic IM-SPDL with Impulse and State Rewards

This report presents the logic SDRL (Stochastic Dynamic Reward Logic), an extension of the stochastic logic IM-SPDL, which supports the specication of complex performance and dependability requirements. SDRL extends IM-SPDL with the possibility to express impulse- and state reward measures.\ud The logic is interpreted over extended action-based Markov reward model (EMRM), i.e. transition systems containing both immediate and Markovian transitions, where additionally the states and transitions can be enriched with rewards.\ud We define ne the syntax and semantics of the new logic and show that SDRL provides powerful means to specify path-based properties with timing and reward-based restrictions.\ud In general, paths can be characterised by regular expressions, also called programs, where the executability of a program may depend on the validity of test formulae. For the model checking of SDRL time- and reward-bounded path formulae, a deterministic program automaton is constructed from the requirement. Afterwards the product transition\ud system between this automaton and the EMRM is built and subsequently transformed into a continuous time Markov reward model (MRM) on which numerical\ud analysis is performed.\u

Computing Quantiles in Markov Reward Models

Probabilistic model checking mainly concentrates on techniques for reasoning about the probabilities of certain path properties or expected values of certain random variables. For the quantitative system analysis, however, there is also another type of interesting performance measure, namely quantiles. A typical quantile query takes as input a lower probability bound p and a reachability property. The task is then to compute the minimal reward bound r such that with probability at least p the target set will be reached before the accumulated reward exceeds r. Quantiles are well-known from mathematical statistics, but to the best of our knowledge they have not been addressed by the model checking community so far. In this paper, we study the complexity of quantile queries for until properties in discrete-time finite-state Markov decision processes with non-negative rewards on states. We show that qualitative quantile queries can be evaluated in polynomial time and present an exponential algorithm for the evaluation of quantitative quantile queries. For the special case of Markov chains, we show that quantitative quantile queries can be evaluated in time polynomial in the size of the chain and the maximum reward.Comment: 17 pages, 1 figure; typo in example correcte

Formal Dependability Engineering with MIOA

In this paper, we introduce MIOA, a stochastic process algebra-like specification language with datatypes, as well as a logic intSPDL, and its model checking algorithms. MIOA, which stands for Markovian input/output automata language, is an extension of Lynch's input/automata with Markovian timed transitions.MIOA can serve both as a fully fledged ``stand-alone'' specification language and the semantic model for the architectural dependability modelling and evaluation language Arcade. The logic intSPDL is an extension of the stochastic logic SPDL, to deal with the specialties of MIOA. intSPDL in the context of Arcade can be seen as the semantic model of abstract and complex dependability measures that can be defined in the Arcade framework. We define syntax and semantics of both MIOA and intSPDL, and show examples of applying MIOA and intSPDL in the realm of dependability modelling with Arcade

Transient Reward Approximation for Continuous-Time Markov Chains

We are interested in the analysis of very large continuous-time Markov chains (CTMCs) with many distinct rates. Such models arise naturally in the context of reliability analysis, e.g., of computer network performability analysis, of power grids, of computer virus vulnerability, and in the study of crowd dynamics. We use abstraction techniques together with novel algorithms for the computation of bounds on the expected final and accumulated rewards in continuous-time Markov decision processes (CTMDPs). These ingredients are combined in a partly symbolic and partly explicit (symblicit) analysis approach. In particular, we circumvent the use of multi-terminal decision diagrams, because the latter do not work well if facing a large number of different rates. We demonstrate the practical applicability and efficiency of the approach on two case studies.Comment: Accepted for publication in IEEE Transactions on Reliabilit

Trend-based analysis of a population model of the AKAP scaffold protein

We formalise a continuous-time Markov chain with multi-dimensional discrete state space model of the AKAP scaffold protein as a crosstalk mediator between two biochemical signalling pathways. The analysis by temporal properties of the AKAP model requires reasoning about whether the counts of individuals of the same type (species) are increasing or decreasing. For this purpose we propose the concept of stochastic trends based on formulating the probabilities of transitions that increase (resp. decrease) the counts of individuals of the same type, and express these probabilities as formulae such that the state space of the model is not altered. We define a number of stochastic trend formulae (e.g. weakly increasing, strictly increasing, weakly decreasing, etc.) and use them to extend the set of state formulae of Continuous Stochastic Logic. We show how stochastic trends can be implemented in a guarded-command style specification language for transition systems. We illustrate the application of stochastic trends with numerous small examples and then we analyse the AKAP model in order to characterise and show causality and pulsating behaviours in this biochemical system

Analysis of signalling pathways using continuous time Markov chains

We describe a quantitative modelling and analysis approach for signal transduction networks. We illustrate the approach with an example, the RKIP inhibited ERK pathway [CSK+03]. Our models are high level descriptions of continuous time Markov chains: proteins are modelled by synchronous processes and reactions by transitions. Concentrations are modelled by discrete, abstract quantities. The main advantage of our approach is that using a (continuous time) stochastic logic and the PRISM model checker, we can perform quantitative analysis such as what is the probability that if a concentration reaches a certain level, it will remain at that level thereafter? or how does varying a given reaction rate affect that probability? We also perform standard simulations and compare our results with a traditional ordinary differential equation model. An interesting result is that for the example pathway, only a small number of discrete data values is required to render the simulations practically indistinguishable