Mixing Time of Markov Chains, Dynamical Systems and Evolution

In this paper we study the mixing time of evolutionary Markov chains over populations of a fixed size (N) in which each individual can be one of m types. These Markov chains have the property that they are guided by a dynamical system from the m-dimensional probability simplex to itself. Roughly, given the current state of the Markov chain, which can be viewed as a probability distribution over the m types, the next state is generated by applying this dynamical system to this distribution, and then sampling from it N times. Many processes in nature, from biology to sociology, are evolutionary and such chains can be used to model them. In this study, the mixing time is of particular interest as it determines the speed of evolution and whether the statistics of the steady state can be efficiently computed. In a recent result [Panageas, Srivastava, Vishnoi, Soda, 2016], it was suggested that the mixing time of such Markov chains is connected to the geometry of this guiding dynamical system. In particular, when the dynamical system has a fixed point which is a global attractor, then the mixing is fast. The limit sets of dynamical systems, however, can exhibit more complex behavior: they could have multiple fixed points that are not necessarily stable, periodic orbits, or even chaos. Such behavior arises in important evolutionary settings such as the dynamics of sexual evolution and that of grammar acquisition. In this paper we prove that the geometry of the dynamical system can also give tight mixing time bounds when the dynamical system has multiple fixed points and periodic orbits. We show that the mixing time continues to remain small in the presence of several unstable fixed points and is exponential in N when there are two or more stable fixed points. As a consequence of our results, we obtain a phase transition result for the mixing time of the sexual/grammar model mentioned above. We arrive at the conclusion that in the interesting parameter regime for these models, i.e., when there are multiple stable fixed points, the mixing is slow. Our techniques strengthen the connections between Markov chains and dynamical systems and we expect that the tools developed in this paper should have a wider applicability

Evolutionary Dynamics in Finite Populations Mix Rapidly

In this paper we prove that the mixing time of a broad class of evolutionary dynamics in finite, unstructured populations is roughly logarithmic in the size of the state space. An important special case of such a stochastic process is the Wright-Fisher model from evolutionary biology (with selection and mutation) on a population of size N over m genotypes. Our main result implies that the mixing time of this process is O(log N) for all mutation rates and fitness landscapes, and solves the main open problem from [4]. In particular, it significantly extends the main result in [18] who proved this for m = 2. Biologically, such models have been used to study the evolution of viral populations with applications to drug design strategies countering them. Here the time it takes for the population to reach a steady state is important both for the estimation of the steady-state structure of the population as well in the modeling of the treatment strength and duration. Our result, that such populations exhibit rapid mixing, makes both of these approaches sound. Technically, we make a novel connection between Markov chains arising in evolutionary dynamics and dynamical systems on the probability simplex. This allows us to use the local and global stability properties of the fixed points of such dynamical systems to construct a contractive coupling in a fairly general setting. We expect that our mixing time result would be useful beyond the evolutionary biology setting, and the techniques used here would find applications in bounding the mixing times of Markov chains which have a natural underlying dynamical system

Consistency of maximum likelihood estimation for some dynamical systems

We consider the asymptotic consistency of maximum likelihood parameter estimation for dynamical systems observed with noise. Under suitable conditions on the dynamical systems and the observations, we show that maximum likelihood parameter estimation is consistent. Our proof involves ideas from both information theory and dynamical systems. Furthermore, we show how some well-studied properties of dynamical systems imply the general statistical properties related to maximum likelihood estimation. Finally, we exhibit classical families of dynamical systems for which maximum likelihood estimation is consistent. Examples include shifts of finite type with Gibbs measures and Axiom A attractors with SRB measures.Comment: Published in at http://dx.doi.org/10.1214/14-AOS1259 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Irreversible Markov chains in spin models: Topological excitations

We analyze the convergence of the irreversible event-chain Monte Carlo algorithm for continuous spin models in the presence of topological excitations. In the two-dimensional XY model, we show that the local nature of the Markov-chain dynamics leads to slow decay of vortex-antivortex correlations while spin waves decorrelate very quickly. Using a Frechet description of the maximum vortex-antivortex distance, we quantify the contributions of topological excitations to the equilibrium correlations, and show that they vary from a dynamical critical exponent z \sim 2 at the critical temperature to z \sim 0 in the limit of zero temperature. We confirm the event-chain algorithm's fast relaxation (corresponding to z = 0) of spin waves in the harmonic approximation to the XY model. Mixing times (describing the approach towards equilibrium from the least favorable initial state) however remain much larger than equilibrium correlation times at low temperatures. We also describe the respective influence of topological monopole-antimonopole excitations and of spin waves on the event-chain dynamics in the three-dimensional Heisenberg model.Comment: 5 pages, 5 figure

A local fluctuation theorem

A mechanism for the validity of a local version of the fluctuation theorem, uniform in the system size, is discussed for a reversible chain of weakly coupled Anosov systems.Comment: plain TeX, 1 figur

The ensemble of random Markov matrices

The ensemble of random Markov matrices is introduced as a set of Markov or stochastic matrices with the maximal Shannon entropy. The statistical properties of the stationary distribution pi, the average entropy growth rate $h$ and the second largest eigenvalue nu across the ensemble are studied. It is shown and heuristically proven that the entropy growth-rate and second largest eigenvalue of Markov matrices scale in average with dimension of matrices d as h ~ log(O(d)) and nu ~ d^(-1/2), respectively, yielding the asymptotic relation h tau_c ~ 1/2 between entropy h and correlation decay time tau_c = -1/log|nu| . Additionally, the correlation between h and and tau_c is analysed and is decreasing with increasing dimension d.Comment: 12 pages, 6 figur

Agent Based Models and Opinion Dynamics as Markov Chains

This paper introduces a Markov chain approach that allows a rigorous analysis of agent based opinion dynamics as well as other related agent based models (ABM). By viewing the ABM dynamics as a micro description of the process, we show how the corresponding macro description is obtained by a projection construction. Then, well known conditions for lumpability make it possible to establish the cases where the macro model is still Markov. In this case we obtain a complete picture of the dynamics including the transient stage, the most interesting phase in applications. For such a purpose a crucial role is played by the type of probability distribution used to implement the stochastic part of the model which defines the updating rule and governs the dynamics. In addition, we show how restrictions in communication leading to the co-existence of different opinions follow from the emergence of new absorbing states. We describe our analysis in detail with some specific models of opinion dynamics. Generalizations concerning different opinion representations as well as opinion models with other interaction mechanisms are also discussed. We find that our method may be an attractive alternative to mean-field approaches and that this approach provides new perspectives on the modeling of opinion exchange dynamics, and more generally of other ABM.Comment: 26 pages, 12 figure