A mathematical formalization of the parallel replica dynamics

The purpose of this article is to lay the mathematical foundations of a well known numerical approach in computational statistical physics and molecular dynamics, namely the parallel replica dynamics introduced by A.F. Voter. The aim of the approach is to efficiently generate a coarse-grained evolution (in terms of state-to-state dynamics) of a given stochastic process. The approach formally consists in concurrently considering several realizations of the stochastic process, and tracking among the realizations that which, the soonest, undergoes an important transition. Using specific properties of the dynamics generated, a computational speed-up is obtained. In the best cases, this speed-up approaches the number of realizations considered. By drawing connections with the theory of Markov processes and, in particular, exploiting the notion of quasi-stationary distribution, we provide a mathematical setting appropriate for assessing theoretically the performance of the approach, and possibly improving it

The parallel replica method for simulating long trajectories of Markov chains

The parallel replica dynamics, originally developed by A.F. Voter, efficiently simulates very long trajectories of metastable Langevin dynamics. We present an analogous algorithm for discrete time Markov processes. Such Markov processes naturally arise, for example, from the time discretization of a continuous time stochastic dynamics. Appealing to properties of quasistationary distributions, we show that our algorithm reproduces exactly (in some limiting regime) the law of the original trajectory, coarsened over the metastable states.Comment: 13 pages, 6 figure

Two mathematical tools to analyze metastable stochastic processes

We present how entropy estimates and logarithmic Sobolev inequalities on the one hand, and the notion of quasi-stationary distribution on the other hand, are useful tools to analyze metastable overdamped Langevin dynamics, in particular to quantify the degree of metastability. We discuss the interest of these approaches to estimate the efficiency of some classical algorithms used to speed up the sampling, and to evaluate the error introduced by some coarse-graining procedures. This paper is a summary of a plenary talk given by the author at the ENUMATH 2011 conference

A walk in the statistical mechanical formulation of neural networks

Neural networks are nowadays both powerful operational tools (e.g., for pattern recognition, data mining, error correction codes) and complex theoretical models on the focus of scientific investigation. As for the research branch, neural networks are handled and studied by psychologists, neurobiologists, engineers, mathematicians and theoretical physicists. In particular, in theoretical physics, the key instrument for the quantitative analysis of neural networks is statistical mechanics. From this perspective, here, we first review attractor networks: starting from ferromagnets and spin-glass models, we discuss the underlying philosophy and we recover the strand paved by Hopfield, Amit-Gutfreund-Sompolinky. One step forward, we highlight the structural equivalence between Hopfield networks (modeling retrieval) and Boltzmann machines (modeling learning), hence realizing a deep bridge linking two inseparable aspects of biological and robotic spontaneous cognition. As a sideline, in this walk we derive two alternative (with respect to the original Hebb proposal) ways to recover the Hebbian paradigm, stemming from ferromagnets and from spin-glasses, respectively. Further, as these notes are thought of for an Engineering audience, we highlight also the mappings between ferromagnets and operational amplifiers and between antiferromagnets and flip-flops (as neural networks -built by op-amp and flip-flops- are particular spin-glasses and the latter are indeed combinations of ferromagnets and antiferromagnets), hoping that such a bridge plays as a concrete prescription to capture the beauty of robotics from the statistical mechanical perspective.Comment: Contribute to the proceeding of the conference: NCTA 2014. Contains 12 pages,7 figure

The Relativistic Hopfield network: rigorous results

The relativistic Hopfield model constitutes a generalization of the standard Hopfield model that is derived by the formal analogy between the statistical-mechanic framework embedding neural networks and the Lagrangian mechanics describing a fictitious single-particle motion in the space of the tuneable parameters of the network itself. In this analogy the cost-function of the Hopfield model plays as the standard kinetic-energy term and its related Mattis overlap (naturally bounded by one) plays as the velocity. The Hamiltonian of the relativisitc model, once Taylor-expanded, results in a P-spin series with alternate signs: the attractive contributions enhance the information-storage capabilities of the network, while the repulsive contributions allow for an easier unlearning of spurious states, conferring overall more robustness to the system as a whole. Here we do not deepen the information processing skills of this generalized Hopfield network, rather we focus on its statistical mechanical foundation. In particular, relying on Guerra's interpolation techniques, we prove the existence of the infinite volume limit for the model free-energy and we give its explicit expression in terms of the Mattis overlaps. By extremizing the free energy over the latter we get the generalized self-consistent equations for these overlaps, as well as a picture of criticality that is further corroborated by a fluctuation analysis. These findings are in full agreement with the available previous results.Comment: 11 pages, 1 figur

Free energies of Boltzmann Machines: self-averaging, annealed and replica symmetric approximations in the thermodynamic limit

Restricted Boltzmann machines (RBMs) constitute one of the main models for machine statistical inference and they are widely employed in Artificial Intelligence as powerful tools for (deep) learning. However, in contrast with countless remarkable practical successes, their mathematical formalization has been largely elusive: from a statistical-mechanics perspective these systems display the same (random) Gibbs measure of bi-partite spin-glasses, whose rigorous treatment is notoriously difficult. In this work, beyond providing a brief review on RBMs from both the learning and the retrieval perspectives, we aim to contribute to their analytical investigation, by considering two distinct realizations of their weights (i.e., Boolean and Gaussian) and studying the properties of their related free energies. More precisely, focusing on a RBM characterized by digital couplings, we first extend the Pastur-Shcherbina-Tirozzi method (originally developed for the Hopfield model) to prove the self-averaging property for the free energy, over its quenched expectation, in the infinite volume limit, then we explicitly calculate its simplest approximation, namely its annealed bound. Next, focusing on a RBM characterized by analogical weights, we extend Guerra's interpolating scheme to obtain a control of the quenched free-energy under the assumption of replica symmetry: we get self-consistencies for the order parameters (in full agreement with the existing Literature) as well as the critical line for ergodicity breaking that turns out to be the same obtained in AGS theory. As we discuss, this analogy stems from the slow-noise universality. Finally, glancing beyond replica symmetry, we analyze the fluctuations of the overlaps for an estimate of the (slow) noise affecting the retrieval of the signal, and by a stability analysis we recover the Aizenman-Contucci identities typical of glassy systems.Comment: 21 pages, 1 figur

Anergy in self-directed B lymphocytes from a statistical mechanics perspective

The ability of the adaptive immune system to discriminate between self and non-self mainly stems from the ontogenic clonal-deletion of lymphocytes expressing strong binding affinity with self-peptides. However, some self-directed lymphocytes may evade selection and still be harmless due to a mechanism called clonal anergy. As for B lymphocytes, two major explanations for anergy developed over three decades: according to "Varela theory", it stems from a proper orchestration of the whole B-repertoire, in such a way that self-reactive clones, due to intensive interactions and feed-back from other clones, display more inertia to mount a response. On the other hand, according to the `two-signal model", which has prevailed nowadays, self-reacting cells are not stimulated by helper lymphocytes and the absence of such signaling yields anergy. The first result we present, achieved through disordered statistical mechanics, shows that helper cells do not prompt the activation and proliferation of a certain sub-group of B cells, which turn out to be just those broadly interacting, hence it merges the two approaches as a whole (in particular, Varela theory is then contained into the two-signal model). As a second result, we outline a minimal topological architecture for the B-world, where highly connected clones are self-directed as a natural consequence of an ontogenetic learning; this provides a mathematical framework to Varela perspective. As a consequence of these two achievements, clonal deletion and clonal anergy can be seen as two inter-playing aspects of the same phenomenon too