Bounding Mean First Passage Times in Population Continuous-Time Markov Chains

We consider the problem of bounding mean first passage times and reachability probabilities for the class of population continuous-time Markov chains, which capture stochastic interactions between groups of identical agents. The quantitative analysis of such models is notoriously difficult since typically neither state-based numerical approaches nor methods based on stochastic sampling give efficient and accurate results. Here, we propose a novel approach that leverages techniques from martingale theory and stochastic processes to generate constraints on the statistical moments of first passage time distributions. These constraints induce a semi-definite program that can be used to compute exact bounds on reachability probabilities and mean first passage times without numerically solving the transient probability distribution of the process or sampling from it. We showcase the method on some test examples and tailor it to models exhibiting multimodality, a class of particularly challenging scenarios from biology

The exit time finite state projection scheme: bounding exit distributions and occupation measures of continuous-time Markov chains

We introduce the exit time finite state projection (ETFSP) scheme, a truncation- based method that yields approximations to the exit distribution and occupation measure associated with the time of exit from a domain (i.e., the time of first passage to the complement of the domain) of time-homogeneous continuous-time Markov chains. We prove that: (i) the computed approximations bound the measures from below; (ii) the total variation distances between the approximations and the measures decrease monotonically as states are added to the truncation; and (iii) the scheme converges, in the sense that, as the truncation tends to the entire state space, the total variation distances tend to zero. Furthermore, we give a computable bound on the total variation distance between the exit distribution and its approximation, and we delineate the cases in which the bound is sharp. We also revisit the related finite state projection scheme and give a comprehensive account of its theoretical properties. We demonstrate the use of the ETFSP scheme by applying it to two biological examples: the computation of the first passage time associated with the expression of a gene, and the fixation times of competing species subject to demographic noise

Queues in a random environment

Exponential single server queues with state dependent arrival and service rates are considered which evolve under influences of external environments. The transitions of the queues are influenced by the environment's state and the movements of the environment depend on the status of the queues (bi-directional interaction). The structure of the environment is constructed in a way to encompass various models from the recent Operation Research literature, where a queue is coupled e.g. with an inventory or with reliability issues. With a Markovian joint queueing-environment process we prove separability for a large class of such interactive systems, i.e. the steady state distribution is of product form and explicitly given: The queue and the environment processes decouple asymptotically and in steady state. For non-separable systems we develop ergodicity criteria via Lyapunov functions. By examples we show principles for bounding throughputs of non-separable systems by throughputs of two separable systems as upper and lower bound

Last passage percolation and traveling fronts

We consider a system of N particles with a stochastic dynamics introduced by Brunet and Derrida. The particles can be interpreted as last passage times in directed percolation on {1,...,N} of mean-field type. The particles remain grouped and move like a traveling wave, subject to discretization and driven by a random noise. As N increases, we obtain estimates for the speed of the front and its profile, for different laws of the driving noise. The Gumbel distribution plays a central role for the particle jumps, and we show that the scaling limit is a L\'evy process in this case. The case of bounded jumps yields a completely different behavior