Control Variates for Reversible MCMC Samplers

A general methodology is introduced for the construction and effective application of control variates to estimation problems involving data from reversible MCMC samplers. We propose the use of a specific class of functions as control variates, and we introduce a new, consistent estimator for the values of the coefficients of the optimal linear combination of these functions. The form and proposed construction of the control variates is derived from our solution of the Poisson equation associated with a specific MCMC scenario. The new estimator, which can be applied to the same MCMC sample, is derived from a novel, finite-dimensional, explicit representation for the optimal coefficients. The resulting variance-reduction methodology is primarily applicable when the simulated data are generated by a conjugate random-scan Gibbs sampler. MCMC examples of Bayesian inference problems demonstrate that the corresponding reduction in the estimation variance is significant, and that in some cases it can be quite dramatic. Extensions of this methodology in several directions are given, including certain families of Metropolis-Hastings samplers and hybrid Metropolis-within-Gibbs algorithms. Corresponding simulation examples are presented illustrating the utility of the proposed methods. All methodological and asymptotic arguments are rigorously justified under easily verifiable and essentially minimal conditions.Comment: 44 pages; 6 figures; 5 table

Coupling Control Variates for Markov Chain Monte Carlo

We show that Markov couplings can be used to improve the accuracy of Markov chain Monte Carlo calculations in some situations where the steady-state probability distribution is not explicitly known. The technique generalizes the notion of control variates from classical Monte Carlo integration. We illustrate it using two models of nonequilibrium transport

Regression estimators in simulation

sampling;simulation;queuing systems;operations research

Large deviation asymptotics and control variates for simulating large functions

Consider the normalized partial sums of a real-valued function $F$ of a Markov chain, $$\phi_n:=n^{-1}\sum_{k=0}^{n-1}F(\Phi(k)),\qquad n\ge1.$$ The chain $\{\Phi(k):k\ge0\}$ takes values in a general state space $\mathsf {X}$, with transition kernel $P$, and it is assumed that the Lyapunov drift condition holds: $PV\le V-W+b\mathbb{I}_C$ where $V:\mathsf {X}\to(0,\infty)$, $W:\mathsf {X}\to[1,\infty)$, the set $C$ is small and $W$ dominates $F$. Under these assumptions, the following conclusions are obtained: 1. It is known that this drift condition is equivalent to the existence of a unique invariant distribution $\pi$ satisfying $\pi(W)<\infty$, and the law of large numbers holds for any function $F$ dominated by $W$: $$\phi_n\to\phi:=\pi(F),\qquad{a.s.}, n\to\infty.$$ 2. The lower error probability defined by $\mathsf {P}\{\phi_n\le c\}$, for $c<\phi$, $n\ge1$, satisfies a large deviation limit theorem when the function $F$ satisfies a monotonicity condition. Under additional minor conditions an exact large deviations expansion is obtained. 3. If $W$ is near-monotone, then control-variates are constructed based on the Lyapunov function $V$, providing a pair of estimators that together satisfy nontrivial large asymptotics for the lower and upper error probabilities. In an application to simulation of queues it is shown that exact large deviation asymptotics are possible even when the estimator does not satisfy a central limit theorem.Comment: Published at http://dx.doi.org/10.1214/105051605000000737 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org