On the fastest finite Markov processes

International audienceConsider a finite irreducible Markov chain with invariant probability π. Define its inverse communication speed as the expectation to go from x to y, when x, y are sampled independently according to π. In the discrete time setting and when π is the uniform distribution υ, Litvak and Ejov have shown that the permutation matrices associated to Hamiltonian cycles are the fastest Markov chains. Here we prove (A) that the above optimality is with respect to all processes compatible with a fixed graph of permitted transitions (assuming that it does contain a Hamiltonian cycle), not only the Markov chains, and, (B) that this result admits a natural extension in both discrete and continuous time when π is close to υ: the fastest Markov chains/processes are those moving successively on the points of a Hamiltonian cycle, with transition probabilities/jump rates dictated by π. Nevertheless, the claim is no longer true when π is significantly different from υ

Hamiltonian cycles and singularly perturbed Markov chains

We consider the Hamiltonian cycle problem embedded in a singularly perturbed Markov decision process. We also consider a functional on the space of deterministic policies of the process that consist of the (1,1)-entry of the fundamental matrices of the Markov chains induced by the same policies. We show that when the perturbation parameter, e, is less than or equal to 1/N2, the Hamiltonian cycles of the directed graph are precisely the minimizers of our functional over the space of deterministic policies. In the process, we derive analytical expressions for the possible N distinct values of the functional over the, typically, much larger space of deterministic policies

Markov chains and optimality of the Hamiltonian cycle

We consider the Hamiltonian cycle problem (HCP) embedded in a controlled Markov decision process. In this setting, HCP reduces to an optimization problem on a set of Markov chains corresponding to a given graph. We prove that Hamiltonian cycles are minimizers for the trace of the fundamental matrix on a set of all stochastic transition matrices. In case of doubly stochastic matrices with symmetric linear perturbation, we show that Hamiltonian cycles minimize a diagonal element of a fundamental matrix for all admissible values of the perturbation parameter. In contrast to the previous work on this topic, our arguments are primarily based on probabilistic rather than algebraic methods

Hamiltonian cycles and subsets of discounted occupational measures

We study a certain polytope arising from embedding the Hamiltonian cycle problem in a discounted Markov decision process. The Hamiltonian cycle problem can be reduced to finding particular extreme points of a certain polytope associated with the input graph. This polytope is a subset of the space of discounted occupational measures. We characterize the feasible bases of the polytope for a general input graph $G$, and determine the expected numbers of different types of feasible bases when the underlying graph is random. We utilize these results to demonstrate that augmenting certain additional constraints to reduce the polyhedral domain can eliminate a large number of feasible bases that do not correspond to Hamiltonian cycles. Finally, we develop a random walk algorithm on the feasible bases of the reduced polytope and present some numerical results. We conclude with a conjecture on the feasible bases of the reduced polytope.Comment: revised based on referees comment

Quantitative Small Subgraph Conditioning

We revisit the method of small subgraph conditioning, used to establish that random regular graphs are Hamiltonian a.a.s. We refine this method using new technical machinery for random $d$-regular graphs on $n$ vertices that hold not just asymptotically, but for any values of $d$ and $n$. This lets us estimate how quickly the probability of containing a Hamiltonian cycle converges to 1, and it produces quantitative contiguity results between different models of random regular graphs. These results hold with $d$ held fixed or growing to infinity with $n$. As additional applications, we establish the distributional convergence of the number of Hamiltonian cycles when $d$ grows slowly to infinity, and we prove that the number of Hamiltonian cycles can be approximately computed from the graph's eigenvalues for almost all regular graphs.Comment: 59 pages, 5 figures; minor changes for clarit

Quantum speedup of classical mixing processes

Most approximation algorithms for #P-complete problems (e.g., evaluating the permanent of a matrix or the volume of a polytope) work by reduction to the problem of approximate sampling from a distribution $\pi$ over a large set $\S$. This problem is solved using the {\em Markov chain Monte Carlo} method: a sparse, reversible Markov chain $P$ on $\S$ with stationary distribution $\pi$ is run to near equilibrium. The running time of this random walk algorithm, the so-called {\em mixing time} of $P$, is $O(\delta^{-1} \log 1/\pi_*)$ as shown by Aldous, where $\delta$ is the spectral gap of $P$ and $\pi_*$ is the minimum value of $\pi$. A natural question is whether a speedup of this classical method to $O(\sqrt{\delta^{-1}} \log 1/\pi_*)$, the diameter of the graph underlying $P$, is possible using {\em quantum walks}. We provide evidence for this possibility using quantum walks that {\em decohere} under repeated randomized measurements. We show: (a) decoherent quantum walks always mix, just like their classical counterparts, (b) the mixing time is a robust quantity, essentially invariant under any smooth form of decoherence, and (c) the mixing time of the decoherent quantum walk on a periodic lattice $\Z_n^d$ is $O(n d \log d)$, which is indeed $O(\sqrt{\delta^{-1}} \log 1/\pi_*)$ and is asymptotically no worse than the diameter of $\Z_n^d$ (the obvious lower bound) up to at most a logarithmic factor.Comment: 13 pages; v2 revised several part