First-passage phenomena in hierarchical networks

In this paper we study Markov processes and related first passage problems on a class of weighted, modular graphs which generalize the Dyson hierarchical model. In these networks, the coupling strength between two nodes depends on their distance and is modulated by a parameter $\sigma$. We find that, in the thermodynamic limit, ergodicity is lost and the "distant" nodes can not be reached. Moreover, for finite-sized systems, there exists a threshold value for $\sigma$ such that, when $\sigma$ is relatively large, the inhomogeneity of the coupling pattern prevails and "distant" nodes are hardly reached. The same analysis is carried on also for generic hierarchical graphs, where interactions are meant to involve $p$-plets ($p>2$) of nodes, finding that ergodicity is still broken in the thermodynamic limit, but no threshold value for $\sigma$ is evidenced, ultimately due to a slow growth of the network diameter with the size

Dreaming neural networks: forgetting spurious memories and reinforcing pure ones

The standard Hopfield model for associative neural networks accounts for biological Hebbian learning and acts as the harmonic oscillator for pattern recognition, however its maximal storage capacity is $\alpha \sim 0.14$, far from the theoretical bound for symmetric networks, i.e. $\alpha =1$. Inspired by sleeping and dreaming mechanisms in mammal brains, we propose an extension of this model displaying the standard on-line (awake) learning mechanism (that allows the storage of external information in terms of patterns) and an off-line (sleep) unlearning$\&$consolidating mechanism (that allows spurious-pattern removal and pure-pattern reinforcement): this obtained daily prescription is able to saturate the theoretical bound $\alpha=1$, remaining also extremely robust against thermal noise. Both neural and synaptic features are analyzed both analytically and numerically. In particular, beyond obtaining a phase diagram for neural dynamics, we focus on synaptic plasticity and we give explicit prescriptions on the temporal evolution of the synaptic matrix. We analytically prove that our algorithm makes the Hebbian kernel converge with high probability to the projection matrix built over the pure stored patterns. Furthermore, we obtain a sharp and explicit estimate for the "sleep rate" in order to ensure such a convergence. Finally, we run extensive numerical simulations (mainly Monte Carlo sampling) to check the approximations underlying the analytical investigations (e.g., we developed the whole theory at the so called replica-symmetric level, as standard in the Amit-Gutfreund-Sompolinsky reference framework) and possible finite-size effects, finding overall full agreement with the theory.Comment: 31 pages, 12 figure

Free energies of Boltzmann Machines: self-averaging, annealed and replica symmetric approximations in the thermodynamic limit

Restricted Boltzmann machines (RBMs) constitute one of the main models for machine statistical inference and they are widely employed in Artificial Intelligence as powerful tools for (deep) learning. However, in contrast with countless remarkable practical successes, their mathematical formalization has been largely elusive: from a statistical-mechanics perspective these systems display the same (random) Gibbs measure of bi-partite spin-glasses, whose rigorous treatment is notoriously difficult. In this work, beyond providing a brief review on RBMs from both the learning and the retrieval perspectives, we aim to contribute to their analytical investigation, by considering two distinct realizations of their weights (i.e., Boolean and Gaussian) and studying the properties of their related free energies. More precisely, focusing on a RBM characterized by digital couplings, we first extend the Pastur-Shcherbina-Tirozzi method (originally developed for the Hopfield model) to prove the self-averaging property for the free energy, over its quenched expectation, in the infinite volume limit, then we explicitly calculate its simplest approximation, namely its annealed bound. Next, focusing on a RBM characterized by analogical weights, we extend Guerra's interpolating scheme to obtain a control of the quenched free-energy under the assumption of replica symmetry: we get self-consistencies for the order parameters (in full agreement with the existing Literature) as well as the critical line for ergodicity breaking that turns out to be the same obtained in AGS theory. As we discuss, this analogy stems from the slow-noise universality. Finally, glancing beyond replica symmetry, we analyze the fluctuations of the overlaps for an estimate of the (slow) noise affecting the retrieval of the signal, and by a stability analysis we recover the Aizenman-Contucci identities typical of glassy systems.Comment: 21 pages, 1 figur

Slow Encounters of Particle Pairs in Branched Structures

On infinite homogeneous structures, two random walkers meet with certainty if and only if the structure is recurrent, i.e., a single random walker returns to its starting point with probability 1. However, on general inhomogeneous structures this property does not hold and, although a single random walker will certainly return to its starting point, two moving particles may never meet. This striking property has been shown to hold, for instance, on infinite combs. Due to the huge variety of natural phenomena which can be modeled in terms of encounters between two (or more) particles diffusing in comb-like structures, it is fundamental to investigate if and, if so, to what extent similar effects may take place in finite structures. By means of numerical simulations we evidence that, indeed, even on finite structures, the topological inhomogeneity can qualitatively affect the two-particle problem. In particular, the mean encounter time can be polynomially larger than the time expected from the related one particle problem.Comment: 8 pages, 12 figures; accepted for publication in Physical Review

A Diffusive Strategic Dynamics for Social Systems

We propose a model for the dynamics of a social system, which includes diffusive effects and a biased rule for spin-flips, reproducing the effect of strategic choices. This model is able to mimic some phenomena taking place during marketing or political campaigns. Using a cost function based on the Ising model defined on the typical quenched interaction environments for social systems (Erdos-Renyi graph, small-world and scale-free networks), we find, by numerical simulations, that a stable stationary state is reached, and we compare the final state to the one obtained with standard dynamics, by means of total magnetization and magnetic susceptibility. Our results show that the diffusive strategic dynamics features a critical interaction parameter strictly lower than the standard one. We discuss the relevance of our findings in social systems.Comment: Major revisions; to appear on the Journal of Statistical Physic

Non-Convex Multi-species Hopfield models

In this work we introduce a multi-species generalization of the Hopfield model for associative memory, where neurons are divided into groups and both inter-groups and intra-groups pair-wise interactions are considered, with different intensities. Thus, this system contains two of the main ingredients of modern Deep neural network architectures: Hebbian interactions to store patterns of information and multiple layers coding different levels of correlations. The model is completely solvable in the low-load regime with a suitable generalization of the Hamilton-Jacobi technique, despite the Hamiltonian can be a non-definite quadratic form of the magnetizations. The family of multi-species Hopfield model includes, as special cases, the 3-layers Restricted Boltzmann Machine (RBM) with Gaussian hidden layer and the Bidirectional Associative Memory (BAM) model.Comment: This is a pre-print of an article published in J. Stat. Phy

Acquaintance role for decision making and exchanges in social networks

We model a social network by a random graph whose nodes represent agents and links between two of them stand for a reciprocal interaction; each agent is also associated to a binary variable which represents a dichotomic opinion or attribute. We consider both the case of pair-wise (p=2) and multiple (p>2) interactions among agents and we study the behavior of the resulting system by means of the energy-entropy scheme, typical of statistical mechanics methods. We show, analytically and numerically, that the connectivity of the social network plays a non-trivial role: while for pair-wise interactions (p=2) the connectivity weights linearly, when interactions involve contemporary a number of agents larger than two (p>2), its weight gets more and more important. As a result, when p is large, a full consensus within the system, can be reached at relatively small critical couplings with respect to the p=2 case usually analyzed, or, otherwise stated, relatively small coupling strengths among agents are sufficient to orient most of the system.Comment: 7 pages, 1 figur

Neural Networks retrieving Boolean patterns in a sea of Gaussian ones

Restricted Boltzmann Machines are key tools in Machine Learning and are described by the energy function of bipartite spin-glasses. From a statistical mechanical perspective, they share the same Gibbs measure of Hopfield networks for associative memory. In this equivalence, weights in the former play as patterns in the latter. As Boltzmann machines usually require real weights to be trained with gradient descent like methods, while Hopfield networks typically store binary patterns to be able to retrieve, the investigation of a mixed Hebbian network, equipped with both real (e.g., Gaussian) and discrete (e.g., Boolean) patterns naturally arises. We prove that, in the challenging regime of a high storage of real patterns, where retrieval is forbidden, an extra load of Boolean patterns can still be retrieved, as long as the ratio among the overall load and the network size does not exceed a critical threshold, that turns out to be the same of the standard Amit-Gutfreund-Sompolinsky theory. Assuming replica symmetry, we study the case of a low load of Boolean patterns combining the stochastic stability and Hamilton-Jacobi interpolating techniques. The result can be extended to the high load by a non rigorous but standard replica computation argument.Comment: 16 pages, 1 figur

Organization and evolution of synthetic idiotypic networks

We introduce a class of weighted graphs whose properties are meant to mimic the topological features of idiotypic networks, namely the interaction networks involving the B-core of the immune system. Each node is endowed with a bit-string representing the idiotypic specificity of the corresponding B cell and a proper distance between any couple of bit-strings provides the coupling strength between the two nodes. We show that a biased distribution of the entries in bit-strings can yield fringes in the (weighted) degree distribution, small-worlds features, and scaling laws, in agreement with experimental findings. We also investigate the role of ageing, thought of as a progressive increase in the degree of bias in bit-strings, and we show that it can possibly induce mild percolation phenomena, which are investigated too.Comment: 13 page