A Numerical Analyst Looks at the "Cutoff Phenomenon" in Card Shuffling and Other Markov Chains

Abstract

Diaconis and others have shown that certain Markov chains exhibit a "cutoff phenomenon" in which, after an initial period of seemingly little progress, convergence to the steady state occurs suddenly. Since Markov chains are just powers of matrices, how can such effects be explained in the language of applied linear algebra? We attempt to do this, focusing on two examples: random walk on a hypercube, which is essentially the same as the problem of Ehrenfest urns, and the celebrated case of riffle shuffling of a deck of cards. As is typical with transient phenomena in matrix processes, the reason for the cutoff is not readily apparent from an examination of eigenvalues or eigenvectors, but it is reflected strongly in pseudosprectra - provided they are measured in the 1-norm, not the 2-norm. We illustrate and explain the cutoff phenomenon with Matlab computations based in part on a new explicit formula for the entries of the $n \times n$ "riffle shuffle matrix", and note that while the normwise cutoff may occur at one point, such as $\frac{3}{2} \log_{2} n$ for the riffle shuffle, weak convergence may occur at an equally precise earlier point such as $\log_{2} n$