Overlap synchronisation in multipartite random energy models

In a multipartite random energy model, made of a number of coupled GREMs, we determine the joint law of the overlaps in terms of the ones of the single GREMs. This provides the simplest example of the so-called overlap synchronisation.Comment: 6 page

Legendre Duality of Spherical and Gaussian Spin Glasses

The classical result of concentration of the Gaussian measure on the sphere in the limit of large dimension induces a natural duality between Gaussian and spherical models of spin glass. We analyse the Legendre variational structure linking the free energies of these two systems, in the spirit of the equivalence of ensembles of statistical mechanics. Our analysis, combined with the previous work [4], shows that such models are replica symmetric. Lastly, we briefly discuss an application of our result to the study of the Gaussian Hopfield model

Inference of the Kinetic Ising Model with Heterogeneous Missing Data

We consider the problem of inferring a causality structure from multiple binary time series by using the Kinetic Ising Model in datasets where a fraction of observations is missing. We take our steps from a recent work on Mean Field methods for the inference of the model with hidden spins and develop a pseudo-Expectation-Maximization algorithm that is able to work even in conditions of severe data sparsity. The methodology relies on the Martin-Siggia-Rose path integral method with second order saddle-point solution to make it possible to calculate the log-likelihood in polynomial time, giving as output a maximum likelihood estimate of the couplings matrix and of the missing observations. We also propose a recursive version of the algorithm, where at every iteration some missing values are substituted by their maximum likelihood estimate, showing that the method can be used together with sparsification schemes like LASSO regularization or decimation. We test the performance of the algorithm on synthetic data and find interesting properties when it comes to the dependency on heterogeneity of the observation frequency of spins and when some of the hypotheses that are necessary to the saddle-point approximation are violated, such as the small couplings limit and the assumption of statistical independence between couplings

Non-Convex Multi-species Hopfield models

In this work we introduce a multi-species generalization of the Hopfield model for associative memory, where neurons are divided into groups and both inter-groups and intra-groups pair-wise interactions are considered, with different intensities. Thus, this system contains two of the main ingredients of modern Deep neural network architectures: Hebbian interactions to store patterns of information and multiple layers coding different levels of correlations. The model is completely solvable in the low-load regime with a suitable generalization of the Hamilton-Jacobi technique, despite the Hamiltonian can be a non-definite quadratic form of the magnetizations. The family of multi-species Hopfield model includes, as special cases, the 3-layers Restricted Boltzmann Machine (RBM) with Gaussian hidden layer and the Bidirectional Associative Memory (BAM) model.Comment: This is a pre-print of an article published in J. Stat. Phy

Centrality metrics and localization in core-periphery networks

Two concepts of centrality have been defined in complex networks. The first considers the centrality of a node and many different metrics for it has been defined (e.g. eigenvector centrality, PageRank, non-backtracking centrality, etc). The second is related to a large scale organization of the network, the core-periphery structure, composed by a dense core plus an outlying and loosely-connected periphery. In this paper we investigate the relation between these two concepts. We consider networks generated via the Stochastic Block Model, or its degree corrected version, with a strong core-periphery structure and we investigate the centrality properties of the core nodes and the ability of several centrality metrics to identify them. We find that the three measures with the best performance are marginals obtained with belief propagation, PageRank, and degree centrality, while non-backtracking and eigenvector centrality (or MINRES}, showed to be equivalent to the latter in the large network limit) perform worse in the investigated networks.Comment: 15 pages, 8 figure

Detectability of Macroscopic Structures in Directed Asymmetric Stochastic Block Model

We study the problem of identifying macroscopic structures in networks, characterizing the impact of introducing link directions on the detectability phase transition. To this end, building on the stochastic block model, we construct a class of hardly detectable directed networks. We find closed form solutions by using belief propagation method showing how the transition line depends on the assortativity and the asymmetry of the network. Finally, we numerically identify the existence of a hard phase for detection close to the transition point.Comment: 9 pages, 7 figure

Mean field spin glasses treated with PDE techniques

Following an original idea of F. Guerra, in this notes we analyze the Sherrington-Kirkpatrick model from different perspectives, all sharing the underlying approach which consists in linking the resolution of the statistical mechanics of the model (e.g. solving for the free energy) to well-known partial differential equation (PDE) problems (in suitable spaces). The plan is then to solve the related PDE using techniques involved in their native field and lastly bringing back the solution in the proper statistical mechanics framework. Within this strand, after a streamlined test-case on the Curie-Weiss model to highlight the methods more than the physics behind, we solve the SK both at the replica symmetric and at the 1-RSB level, obtaining the correct expression for the free energy via an analogy to a Fourier equation and for the self-consistencies with an analogy to a Burger equation, whose shock wave develops exactly at critical noise level (triggering the phase transition). Our approach, beyond acting as a new alternative method (with respect to the standard routes) for tackling the complexity of spin glasses, links symmetries in PDE theory with constraints in statistical mechanics and, as a novel result from the theoretical physics perspective, we obtain a new class of polynomial identities (namely of Aizenman-Contucci type but merged within the Guerra's broken replica measures), whose interest lies in understanding, via the recent Panchenko breakthroughs, how to force the overlap organization to the ultrametric tree predicted by Parisi

Neural Networks retrieving Boolean patterns in a sea of Gaussian ones

Restricted Boltzmann Machines are key tools in Machine Learning and are described by the energy function of bipartite spin-glasses. From a statistical mechanical perspective, they share the same Gibbs measure of Hopfield networks for associative memory. In this equivalence, weights in the former play as patterns in the latter. As Boltzmann machines usually require real weights to be trained with gradient descent like methods, while Hopfield networks typically store binary patterns to be able to retrieve, the investigation of a mixed Hebbian network, equipped with both real (e.g., Gaussian) and discrete (e.g., Boolean) patterns naturally arises. We prove that, in the challenging regime of a high storage of real patterns, where retrieval is forbidden, an extra load of Boolean patterns can still be retrieved, as long as the ratio among the overall load and the network size does not exceed a critical threshold, that turns out to be the same of the standard Amit-Gutfreund-Sompolinsky theory. Assuming replica symmetry, we study the case of a low load of Boolean patterns combining the stochastic stability and Hamilton-Jacobi interpolating techniques. The result can be extended to the high load by a non rigorous but standard replica computation argument.Comment: 16 pages, 1 figur

Extensive load in multitasking associative networks

We use belief-propagation techniques to study the equilibrium behavior of a bipartite spin-glass, with interactions between two sets of $N$ and $P = \alpha N$ spins. Each spin has a finite degree, i.e.\ number of interaction partners in the opposite set; an equivalent view is then of a system of $N$ neurons storing $P$ diluted patterns. We show that in a large part of the parameter space of noise, dilution and storage load, delimited by a critical surface, the network behaves as an extensive parallel processor, retrieving all $P$ patterns {\it in parallel} without falling into spurious states due to pattern cross-talk and typical of the structural glassiness built into the network. Our approach allows us to consider effects beyond those studied in replica theory so far, including pattern asymmetry and heterogeneous dilution. Parallel extensive retrieval is more robust for homogeneous degree distributions, and is not disrupted by biases in the distributions of the spin-glass links