476 research outputs found
Recommended from our members
Numerical solution of Markov Chains
This project deals with techniques to solve Markov Chains numerically
A survey of generalized inverses and their use in stochastic modelling
In many stochastic models, in particular Markov chains in discrete or continuous time and Markov
renewal processes, a Markov chain is present either directly or indirectly through some form of
embedding. The analysis of many problems of interest associated with these models, eg. stationary
distributions, moments of first passage time distributions and moments of occupation time random
variables, often concerns the solution of a system of linear equations involving I ā P, where P is the
transition matrix of a finite, irreducible, discrete time Markov chain.
Generalized inverses play an important role in the solution of such singular sets of equations. In this
paper we survey the application of generalized inverses to the aforementioned problems. The
presentation will include results concerning the analysis of perturbed systems and the characterization of
types of generalized inverses associated with Markovian kernels
Asymptotic Expansions for Stationary Distributions of Perturbed Semi-Markov Processes
New algorithms for computing of asymptotic expansions for stationary
distributions of nonlinearly perturbed semi-Markov processes are presented. The
algorithms are based on special techniques of sequential phase space reduction,
which can be applied to processes with asymptotically coupled and uncoupled
finite phase spaces.Comment: 83 page
Using the probabilistic evaluation tool for the analytical solution of large Markov models
Stochastic Petri net-based Markov modeling is a potentially very powerful and generic approach for evaluating the performance and dependability of many different systems, such as computer systems, communication networks, manufacturing systems, etc. As a consequence of their general applicability, SPN-based Markov models form the basic solution approach for several software packages that have been developed for the analytic solution of performance and dependability models. In these tools, stochastic Petri nets are used to conveniently specify complicated models, after which an automatic mapping can be carried out to an underlying Markov reward model. Subsequently, this Markov reward model is solved by specialized solution algorithms, appropriately selected for the measure of interest. One of the major aspects that hampers the use of SPN-based Markov models for the analytic solution of performance and dependability results is the size of the state space. Although typically models of up to a few hundred thousand states can conveniently be solved on modern-day work-stations, often even larger models are required to represent all the desired detail of the system. Our tool PET (probabilistic evaluation tool) circumvents problems of large state spaces when the desired performance and dependability measure are transient measures. It does so by an approach named probabilistic evaluatio
Computing lifetimes for battery-powered devices
The battery lifetime of mobile devices depends on the usage pattern of the battery, next to the discharge rate and the battery capacity. Therefore, it is important to include the usage pattern in battery lifetime computations. We do this by combining a stochastic workload, modeled as a continuous-time Markov model, with a well-known battery model. For this combined model, we provide new algorithms to efficiently compute the expected lifetime and the distribution and expected value of the delivered charge
Explicit Model Checking of Very Large MDP using Partitioning and Secondary Storage
The applicability of model checking is hindered by the state space explosion
problem in combination with limited amounts of main memory. To extend its
reach, the large available capacities of secondary storage such as hard disks
can be exploited. Due to the specific performance characteristics of secondary
storage technologies, specialised algorithms are required. In this paper, we
present a technique to use secondary storage for probabilistic model checking
of Markov decision processes. It combines state space exploration based on
partitioning with a block-iterative variant of value iteration over the same
partitions for the analysis of probabilistic reachability and expected-reward
properties. A sparse matrix-like representation is used to store partitions on
secondary storage in a compact format. All file accesses are sequential, and
compression can be used without affecting runtime. The technique has been
implemented within the Modest Toolset. We evaluate its performance on several
benchmark models of up to 3.5 billion states. In the analysis of time-bounded
properties on real-time models, our method neutralises the state space
explosion induced by the time bound in its entirety.Comment: The final publication is available at Springer via
http://dx.doi.org/10.1007/978-3-319-24953-7_1
Efficient CSL Model Checking Using Stratification
For continuous-time Markov chains, the model-checking problem with respect to
continuous-time stochastic logic (CSL) has been introduced and shown to be
decidable by Aziz, Sanwal, Singhal and Brayton in 1996. Their proof can be
turned into an approximation algorithm with worse than exponential complexity.
In 2000, Baier, Haverkort, Hermanns and Katoen presented an efficient
polynomial-time approximation algorithm for the sublogic in which only binary
until is allowed. In this paper, we propose such an efficient polynomial-time
approximation algorithm for full CSL. The key to our method is the notion of
stratified CTMCs with respect to the CSL property to be checked. On a
stratified CTMC, the probability to satisfy a CSL path formula can be
approximated by a transient analysis in polynomial time (using uniformization).
We present a measure-preserving, linear-time and -space transformation of any
CTMC into an equivalent, stratified one. This makes the present work the
centerpiece of a broadly applicable full CSL model checker. Recently, the
decision algorithm by Aziz et al. was shown to work only for stratified CTMCs.
As an additional contribution, our measure-preserving transformation can be
used to ensure the decidability for general CTMCs.Comment: 18 pages, preprint for LMCS. An extended abstract appeared in ICALP
201
Modelling the influence of RKIP on the ERK signalling pathway using the stochastic process algebra PEPA
This paper examines the influence of the Raf Kinase Inhibitor Protein (RKIP) on the Extracellular signal Regulated Kinase (ERK) signalling pathway [5] through modelling in a Markovian process algebra, PEPA [11]. Two models of the system are presented, a reagent-centric view and a pathway-centric view. The models capture functionality at the level of subpathway, rather than at a molecular level. Each model affords a different perspective of the pathway and analysis. We demonstrate the two models to be formally equivalent using the timing-aware bisimulation defined over PEPA models and discuss the biological significance
- ā¦