Imprecise continuous-time Markov chains : efficient computational methods with guaranteed error bounds

Imprecise continuous-time Markov chains are a robust type of continuous-time Markov chains that allow for partially specified time-dependent parameters. Computing inferences for them requires the solution of a non-linear differential equation. As there is no general analytical expression for this solution, efficient numerical approximation methods are essential to the applicability of this model. We here improve the uniform approximation method of Krak et al. (2016) in two ways and propose a novel and more efficient adaptive approximation method. For ergodic chains, we also provide a method that allows us to approximate stationary distributions up to any desired maximal error

Geometric ergodicity of the Random Walk Metropolis with position-dependent proposal covariance

We consider a Metropolis-Hastings method with proposal kernel $\mathcal{N}(x,hG^{-1}(x))$, where $x$ is the current state. After discussing specific cases from the literature, we analyse the ergodicity properties of the resulting Markov chains. In one dimension we find that suitable choice of $G^{-1}(x)$ can change the ergodicity properties compared to the Random Walk Metropolis case $\mathcal{N}(x,h\Sigma)$, either for the better or worse. In higher dimensions we use a specific example to show that judicious choice of $G^{-1}(x)$ can produce a chain which will converge at a geometric rate to its limiting distribution when probability concentrates on an ever narrower ridge as $|x|$ grows, something which is not true for the Random Walk Metropolis.Comment: 15 pages + appendices, 4 figure

On the long time behavior of the TCP window size process

The TCP window size process appears in the modeling of the famous Transmission Control Protocol used for data transmission over the Internet. This continuous time Markov process takes its values in $[0,\infty)$, is ergodic and irreversible. It belongs to the Additive Increase Multiplicative Decrease class of processes. The sample paths are piecewise linear deterministic and the whole randomness of the dynamics comes from the jump mechanism. Several aspects of this process have already been investigated in the literature. In the present paper, we mainly get quantitative estimates for the convergence to equilibrium, in terms of the $W_1$ Wasserstein coupling distance, for the process and also for its embedded chain.Comment: Correction

Large deviation asymptotics and control variates for simulating large functions

Consider the normalized partial sums of a real-valued function $F$ of a Markov chain, $$\phi_n:=n^{-1}\sum_{k=0}^{n-1}F(\Phi(k)),\qquad n\ge1.$$ The chain $\{\Phi(k):k\ge0\}$ takes values in a general state space $\mathsf {X}$, with transition kernel $P$, and it is assumed that the Lyapunov drift condition holds: $PV\le V-W+b\mathbb{I}_C$ where $V:\mathsf {X}\to(0,\infty)$, $W:\mathsf {X}\to[1,\infty)$, the set $C$ is small and $W$ dominates $F$. Under these assumptions, the following conclusions are obtained: 1. It is known that this drift condition is equivalent to the existence of a unique invariant distribution $\pi$ satisfying $\pi(W)<\infty$, and the law of large numbers holds for any function $F$ dominated by $W$: $$\phi_n\to\phi:=\pi(F),\qquad{a.s.}, n\to\infty.$$ 2. The lower error probability defined by $\mathsf {P}\{\phi_n\le c\}$, for $c<\phi$, $n\ge1$, satisfies a large deviation limit theorem when the function $F$ satisfies a monotonicity condition. Under additional minor conditions an exact large deviations expansion is obtained. 3. If $W$ is near-monotone, then control-variates are constructed based on the Lyapunov function $V$, providing a pair of estimators that together satisfy nontrivial large asymptotics for the lower and upper error probabilities. In an application to simulation of queues it is shown that exact large deviation asymptotics are possible even when the estimator does not satisfy a central limit theorem.Comment: Published at http://dx.doi.org/10.1214/105051605000000737 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org