Uncoupled Analysis of Stochastic Reaction Networks in Fluctuating Environments

The dynamics of stochastic reaction networks within cells are inevitably modulated by factors considered extrinsic to the network such as for instance the fluctuations in ribsome copy numbers for a gene regulatory network. While several recent studies demonstrate the importance of accounting for such extrinsic components, the resulting models are typically hard to analyze. In this work we develop a general mathematical framework that allows to uncouple the network from its dynamic environment by incorporating only the environment's effect onto the network into a new model. More technically, we show how such fluctuating extrinsic components (e.g., chemical species) can be marginalized in order to obtain this decoupled model. We derive its corresponding process- and master equations and show how stochastic simulations can be performed. Using several case studies, we demonstrate the significance of the approach. For instance, we exemplarily formulate and solve a marginal master equation describing the protein translation and degradation in a fluctuating environment.Comment: 7 pages, 4 figures, Appendix attached as SI.pdf, under submissio

Sojourn times in fluid queues with independent and dependent input and output processes

Markov Fluid Queues (MFQs) are the continuous counterparts of quasi birth–death processes, where infinitesimally small jobs (fluid drops) are arriving and are being served according to rates modulated by a continuous time Markov chain. The fluid drops are served according to the First-Come–First-Served (FCFS) discipline. The queue length process of MFQs can be analyzed by efficient numerical methods developed for Markovian fluid models. In this paper, however, we are focusing on the sojourn time distribution of the fluid drops. In the first part of the paper we derive the phase-type representation of the sojourn time when the input and output processes of the queue are dependent. In the second part we investigate the case when the input and output processes are independent. Based on the age process analysis of the fluid drops, we provide smaller phase-type representations for the sojourn time than the one for dependent input and output processes

Dynamic Service Rate Control for a Single Server Queue with Markov Modulated Arrivals

We consider the problem of service rate control of a single server queueing system with a finite-state Markov-modulated Poisson arrival process. We show that the optimal service rate is non-decreasing in the number of customers in the system; higher congestion rates warrant higher service rates. On the contrary, however, we show that the optimal service rate is not necessarily monotone in the current arrival rate. If the modulating process satisfies a stochastic monotonicity property the monotonicity is recovered. We examine several heuristics and show where heuristics are reasonable substitutes for the optimal control. None of the heuristics perform well in all the regimes. Secondly, we discuss when the Markov-modulated Poisson process with service rate control can act as a heuristic itself to approximate the control of a system with a periodic non-homogeneous Poisson arrival process. Not only is the current model of interest in the control of Internet or mobile networks with bursty traffic, but it is also useful in providing a tractable alternative for the control of service centers with non-stationary arrival rates.Comment: 32 Pages, 7 Figure

Error bounds for last-column-block-augmented truncations of block-structured Markov chains

This paper discusses the error estimation of the last-column-block-augmented northwest-corner truncation (LC-block-augmented truncation, for short) of block-structured Markov chains (BSMCs) in continuous time. We first derive upper bounds for the absolute difference between the time-averaged functionals of a BSMC and its LC-block-augmented truncation, under the assumption that the BSMC satisfies the general $f$-modulated drift condition. We then establish computable bounds for a special case where the BSMC is exponentially ergodic. To derive such computable bounds for the general case, we propose a method that reduces BSMCs to be exponentially ergodic. We also apply the obtained bounds to level-dependent quasi-birth-and-death processes (LD-QBDs), and discuss the properties of the bounds through the numerical results on an M/M/$s$ retrial queue, which is a representative example of LD-QBDs. Finally, we present computable perturbation bounds for the stationary distribution vectors of BSMCs.Comment: This version has fixed the bugs for the positions of Figures 1 through

Statistical inverse problems for population processes

Population processes are stochastic processes that record the dynamics of the number of individuals in a population, and have many different applications in a broad range of areas. Population processes are often modelled as Markov processes, and have the important feature that transitions correspond either to an increase or a decrease in the population size. These two types of transitions are often referred to as births and deaths. A specific class of population processes is the class of birth-death processes, where transitions can only increase or decrease the population by one at a time. In many real-life situations the dynamics of a population is affected by exogenous, often unobservable, factors. Therefore, this thesis considers population processes of which the parameters are affected by an underlying stochastic process, referred to as the background process. The aim is to find reliable inference techniques to estimate the parameters, including those related to the background process, from discrete-time observations of the population size. The statistical inference is complicated severely by the fact that a substantial part of the process is unobserved. First, the underlying background process is not observed. Second, only the population size is observed, which is the net effect of all the transitions in the dynamics of the population. Last, the population size is observed in discrete time, hence the transitions in between two consecutive observations are not observed. In this thesis we show a collection of techniques to overcome these complications for a variety of population processes. The aspects in which the models differ, ask for specific inference techniques. For a certain class of Markov-modulated population processes, we show how the well-known EM algorithm can be used to estimate the model parameters. In these models, the background process is a finite, continuous-time Markov chain and the parameters of the population process switch between distinct values at the jump times of this Markov chain. An algorithm is presented that iteratively maximizes the likelihood function and at the same time updates the parameter estimates. A generalization of the conventional birth-death process, involving a background process, is the quasi birth-death process. We use the Erlangization technique to evaluate the likelihood function for this kind of processes, which can then be maximized numerically to obtain maximum likelihood estimates. A specific model in the class of quasi birth-death processes is a birth-death process of which the births follow a hypoexponential distribution with L phases and are controlled by an on/off mechanism. We call this the on/off-seq-L process, and use it to model the dynamics of populations of mRNA molecules in single living cells. Numerical complications related to the likelihood maximization are analyzed and solutions are presented. Based on real-life data, we illustrate the estimation method, and perform a model selection procedure on the number of phases and on the on/off mechanism. Last, we consider a class of discrete-time multivariate population processes under Markov-modulation. In these models, the population process is defined on a network with finitely many nodes. In addition to the births and deaths that can occur at each of the nodes, the individuals follow a probabilistic route through the network. We introduce the saddlepoint technique and show how it can be used to evaluate the likelihood function based on observations of the network population vector. The likelihood function can again be maximized numerically to obtain maximum likelihood estimates. Throughout the thesis, the accuracy of the inference methods is investigated by extensive simulation studies