Probabilistic Model Checking for Continuous-Time Markov Chains via Sequential Bayesian Inference

Probabilistic model checking for systems with large or unbounded state space is a challenging computational problem in formal modelling and its applications. Numerical algorithms require an explicit representation of the state space, while statistical approaches require a large number of samples to estimate the desired properties with high confidence. Here, we show how model checking of time-bounded path properties can be recast exactly as a Bayesian inference problem. In this novel formulation the problem can be efficiently approximated using techniques from machine learning. Our approach is inspired by a recent result in statistical physics which derived closed form differential equations for the first-passage time distribution of stochastic processes. We show on a number of non-trivial case studies that our method achieves both high accuracy and significant computational gains compared to statistical model checking

Geometric fluid approximation for general continuous-time Markov chains

Fluid approximations have seen great success in approximating the macro-scale behaviour of Markov systems with a large number of discrete states. However, these methods rely on the continuous-time Markov chain (CTMC) having a particular population structure which suggests a natural continuous state-space endowed with a dynamics for the approximating process. We construct here a general method based on spectral analysis of the transition matrix of the CTMC, without the need for a population structure. Specifically, we use the popular manifold learning method of diffusion maps to analyse the transition matrix as the operator of a hidden continuous process. An embedding of states in a continuous space is recovered, and the space is endowed with a drift vector field inferred via Gaussian process regression. In this manner, we construct an ODE whose solution approximates the evolution of the CTMC mean, mapped onto the continuous space (known as the fluid limit)

The exit time finite state projection scheme: bounding exit distributions and occupation measures of continuous-time Markov chains

We introduce the exit time finite state projection (ETFSP) scheme, a truncation- based method that yields approximations to the exit distribution and occupation measure associated with the time of exit from a domain (i.e., the time of first passage to the complement of the domain) of time-homogeneous continuous-time Markov chains. We prove that: (i) the computed approximations bound the measures from below; (ii) the total variation distances between the approximations and the measures decrease monotonically as states are added to the truncation; and (iii) the scheme converges, in the sense that, as the truncation tends to the entire state space, the total variation distances tend to zero. Furthermore, we give a computable bound on the total variation distance between the exit distribution and its approximation, and we delineate the cases in which the bound is sharp. We also revisit the related finite state projection scheme and give a comprehensive account of its theoretical properties. We demonstrate the use of the ETFSP scheme by applying it to two biological examples: the computation of the first passage time associated with the expression of a gene, and the fixation times of competing species subject to demographic noise

ControCity: An Autonomous Approach for Controlling Elasticity Using Buffer Management in Cloud Computing Environment

Cloud computing has been one of the most popular distributed computing paradigms. Elasticity is a crucial feature that distinguishes cloud computing from other distributed computing models. It considers the resource provisioning and allocation processes can be implemented automatically and dynamically. Elasticity feature allows cloud platforms to handle different loads efficiently without disrupting the normal behavior of the application. Therefore, providing a resource elasticity analytical model can play a significant role in cloud resource management. This paper presents Controlling Elasticity (ControCity) framework for controlling resources elasticity through using “buffer management” and “elasticity management”. In the proposed framework, there are two essential components called buffer manager and elasticity manager in the application layer and middleware layer, respectively. The buffer management controls the input queue of the user’s request and the elasticity management controls the elasticity of the cloud platform using learning automata technique. In the application layer, applications are received by cloud applications and, then, placed in the control of the buffer. Buffer manager controls the queue of requests, and elasticity manager of the middleware layer using the learning automata provides a solution for controlling the elasticity of the cloud platform. The experimental results indicate that ControCity reduces the response time by up to 3.7%, and increases the resource utilization and elasticity by up to 8.4% and 5.4%, respectively, compared with the other approaches