Statistical Physics and Representations in Real and Artificial Neural Networks

This document presents the material of two lectures on statistical physics and neural representations, delivered by one of us (R.M.) at the Fundamental Problems in Statistical Physics XIV summer school in July 2017. In a first part, we consider the neural representations of space (maps) in the hippocampus. We introduce an extension of the Hopfield model, able to store multiple spatial maps as continuous, finite-dimensional attractors. The phase diagram and dynamical properties of the model are analyzed. We then show how spatial representations can be dynamically decoded using an effective Ising model capturing the correlation structure in the neural data, and compare applications to data obtained from hippocampal multi-electrode recordings and by (sub)sampling our attractor model. In a second part, we focus on the problem of learning data representations in machine learning, in particular with artificial neural networks. We start by introducing data representations through some illustrations. We then analyze two important algorithms, Principal Component Analysis and Restricted Boltzmann Machines, with tools from statistical physics

A Class of P,TP,T-Invariant Topological Phases of Interacting Electrons

We describe a class of parity- and time-reversal-invariant topological states of matter which can arise in correlated electron systems in 2+1-dimensions. These states are characterized by particle-like excitations exhibiting exotic braiding statistics. $P$ and $T$ invariance are maintained by a `doubling' of the low-energy degrees of freedom which occurs naturally without doubling the underlying microscopic degrees of freedom. The simplest examples have been the subject of considerable interest as proposed mechanisms for high-$T_c$ superconductivity. One is the `doubled' version (i.e. two opposite-chirality copies) of the U(1) chiral spin liquid. The second example corresponds to $Z_2$ gauge theory, which describes a scenario for spin-charge separation. Our main concern, with an eye towards applications to quantum computation, are richer models which support non-Abelian statistics. All of these models, richer or poorer, lie in a tightly-organized discrete family. The physical inference is that a material manifesting the $Z_2$ gauge theory or a doubled chiral spin liquid might be easily altered to one capable of universal quantum computation. These phases of matter have a field-theoretic description in terms of gauge theories which, in their infrared limits, are topological field theories. We motivate these gauge theories using a parton model or slave-fermion construction and show how they can be solved exactly. The structure of the resulting Hilbert spaces can be understood in purely combinatorial terms. The highly-constrained nature of this combinatorial construction, phrased in the language of the topology of curves on surfaces, lays the groundwork for a strategy for constructing microscopic lattice models which give rise to these phases.Comment: Typos fixed, references adde

Conformal field theories in anti-de Sitter space

In this paper we discuss the dynamics of conformal field theories on anti-de Sitter space, focussing on the special case of the N=4 supersymmetric Yang-Mills theory on AdS_4. We argue that the choice of boundary conditions, in particular for the gauge field, has a large effect on the dynamics. For example, for weak coupling, one of two natural choices of boundary conditions for the gauge field leads to a large N deconfinement phase transition as a function of the temperature, while the other does not. For boundary conditions that preserve supersymmetry, the strong coupling dynamics can be analyzed using S-duality (relevant for g_{YM} >> 1), utilizing results of Gaiotto and Witten, as well as by using the AdS/CFT correspondence (relevant for large N and large 't Hooft coupling). We argue that some very specific choices of boundary conditions lead to a simple dual gravitational description for this theory, while for most choices the gravitational dual is not known. In the cases where the gravitational dual is known, we discuss the phase structure at large 't Hooft coupling.Comment: 57 pages, 1 figure. v2: fixed typo

Aspects of Type IIB Theory on ALE Spaces

D-brane technology and strong/weak coupling duality supplement traditional orbifold techniques by making certain background geometries more accessible. In this spirit, we consider some of the geometric properties of the type IIB theory on R^6 \times M where M is an `Asymptotically Locally Euclidean (ALE)' gravitational instanton. Given the self-duality of the theory, we can extract the geometry (both singular and resolved) seen by the weakly coupled IIB string by studying the physics of a D1-brane probe. The construction is both amusing and instructive, as the physics of the probe completely captures the mathematics of the construction of ALE instantons via `HyperKahler Quotients', as presented by Kronheimer. This relation has been noted by Douglas and Moore for the A-series. We extend the explicit construction to the case of the D- and E-series -- uncovering a quite beautiful structure -- and highlight how all of the elements of the mathematical construction find their counterparts in the physics of the type IIB D-string. We discuss the explicit ALE metrics which may be obtained using these techniques, and comment on the role duality plays in relating gauged linear sigma models to conformal field theories.Comment: 27 pages, three figures. Uses harvmac.tex and epsf.tex (sentences corrected on pages 13+14, reference added, small addition to final remarks

Free particles from Brauer algebras in complex matrix models

The gauge invariant degrees of freedom of matrix models based on an N x N complex matrix, with U(N) gauge symmetry, contain hidden free particle structures. These are exhibited using triangular matrix variables via the Schur decomposition. The Brauer algebra basis for complex matrix models developed earlier is useful in projecting to a sector which matches the state counting of N free fermions on a circle. The Brauer algebra projection is characterized by the vanishing of a scale invariant laplacian constructed from the complex matrix. The special case of N=2 is studied in detail: the ring of gauge invariant functions as well as a ring of scale and gauge invariant differential operators are characterized completely. The orthonormal basis of wavefunctions in this special case is completely characterized by a set of five commuting Hamiltonians, which display free particle structures. Applications to the reduced matrix quantum mechanics coming from radial quantization in N=4 SYM are described. We propose that the string dual of the complex matrix harmonic oscillator quantum mechanics has an interpretation in terms of strings and branes in 2+1 dimensions.Comment: 64 pages, v2: Exposition improved, minor corrections; v3: Typos corrected, published versio

Deep clustering: Discriminative embeddings for segmentation and separation

We address the problem of acoustic source separation in a deep learning framework we call "deep clustering." Rather than directly estimating signals or masking functions, we train a deep network to produce spectrogram embeddings that are discriminative for partition labels given in training data. Previous deep network approaches provide great advantages in terms of learning power and speed, but previously it has been unclear how to use them to separate signals in a class-independent way. In contrast, spectral clustering approaches are flexible with respect to the classes and number of items to be segmented, but it has been unclear how to leverage the learning power and speed of deep networks. To obtain the best of both worlds, we use an objective function that to train embeddings that yield a low-rank approximation to an ideal pairwise affinity matrix, in a class-independent way. This avoids the high cost of spectral factorization and instead produces compact clusters that are amenable to simple clustering methods. The segmentations are therefore implicitly encoded in the embeddings, and can be "decoded" by clustering. Preliminary experiments show that the proposed method can separate speech: when trained on spectrogram features containing mixtures of two speakers, and tested on mixtures of a held-out set of speakers, it can infer masking functions that improve signal quality by around 6dB. We show that the model can generalize to three-speaker mixtures despite training only on two-speaker mixtures. The framework can be used without class labels, and therefore has the potential to be trained on a diverse set of sound types, and to generalize to novel sources. We hope that future work will lead to segmentation of arbitrary sounds, with extensions to microphone array methods as well as image segmentation and other domains.Comment: Originally submitted on June 5, 201