How Do Heuristics Expedite Markov Chain Search? Hitting-time Analysis of the Independence Metropolis Sampler

Solving vision problems often entails searching a solution space for optimal states that have maximum Bayesian posterior probability or minimum energy. When the volume of the space is huge, exhaustive search becomes infeasible. Generic stochastic search (e.g. Markov chain Monte Carlo) could be even worse than exhaustive search as it may visit a state repeatedly. To expedite the Markov chain search, one may use heuristics as proposal probability to guide the search in promising portions of the space. Empirically the recent data-driven Markov chain Monte Carlo (DDMCMC) scheme[14,12,2] achieves fast search in a number of vision tasks, attributed by two observations: (i). The posterior probabilities in vision tasks often have very low entropy and thus are narrowly focused on a tight portion of the state space; (ii). The proposal probability computed in bottom-up methods can approximate the posterior well. In this paper we study an independent Metropolis sampler which is a simple case used as components in designing complex MCMC algorithms. We obtain an analytic formula for the expected time to first hit a certain state ("first hitting-time"), as well as very tight lower and upper bounds which depend on the total variation between the target posterior and the heuristic probabilities. These results show, though in humble cases, that one can indeed reach optimal solutions in very few steps with good proposal probabilities regardless of the size of the original search space. This result is different from previous analysis on the Markov chain convergence rate which is bounded by the second largest eigen-value (modulus) and often corresponds to the worst case in the entire search space. In comparison our analysis bears more relevance to the optimization tasks in vision

How Do Heuristics Expedite Markov Chain Search? Hitting-time Analysis of the Independence Metropolis Sampler

Solving vision problems often entails searching a solution space for optimal states that have maximum Bayesian posterior probability or minimum energy. When the volume of the space is huge, exhaustive search becomes infeasible. Generic stochastic search (e.g. Markov chain Monte Carlo) could be even worse than exhaustive search as it may visit a state repeatedly. To expedite the Markov chain search, one may use heuristics as proposal probability to guide the search in promising portions of the space. Empirically the recent data-driven Markov chain Monte Carlo (DDMCMC) scheme[14,12,2] achieves fast search in a number of vision tasks, attributed by two observations: (i). The posterior probabilities in vision tasks often have very low entropy and thus are narrowly focused on a tight portion of the state space; (ii). The proposal probability computed in bottom-up methods can approximate the posterior well. In this paper we study an independent Metropolis sampler which is a simple case used as components in designing complex MCMC algorithms. We obtain an analytic formula for the expected time to first hit a certain state ("first hitting-time"), as well as very tight lower and upper bounds which depend on the total variation between the target posterior and the heuristic probabilities. These results show, though in humble cases, that one can indeed reach optimal solutions in very few steps with good proposal probabilities regardless of the size of the original search space. This result is different from previous analysis on the Markov chain convergence rate which is bounded by the second largest eigen-value (modulus) and often corresponds to the worst case in the entire search space. In comparison our analysis bears more relevance to the optimization tasks in vision

(To appear in the Journal for Theoretical Probability) First Hitting Time Analysis of the Independence Metropolis Sampler

In this paper, we study a special case of the Metropolis algorithm, the Independence Metropolis Sampler (IMS), in the finite state space case. The IMS is often used in designing components of more complex Markov Chain Monte Carlo algorithms. We present new results related to the first hitting time of individual states for the IMS. These results are expressed mostly in terms of the eigenvalues of the transition kernel. We derive a simple form formula for the mean first hitting time and we show tight lower and upper bounds on the mean first hitting time with the upper bound being the product of two factors: a ”local ” factor corresponding to the target state and a ”global ” factor, common to all the states, which is expressed in terms of the total variation distance between the target and the proposal probabilities. We also briefly discuss properties of the distribution of the first hitting time for the IMS and analyze its variance. We conclude by showing how some non-independence Metropolis-Hastings algorithms can perform better than the IMS and deriving general lower and upper bounds for the mean first hitting times of a Metropolis-Hastings algorithm

Lupus nephritis: induction therapy in severe lupus nephritis--should MMF be considered the drug of choice?

link_to_OA_fulltex