Techniques of replica symmetry breaking and the storage problem of the McCulloch-Pitts neuron

In this article the framework for Parisi's spontaneous replica symmetry breaking is reviewed, and subsequently applied to the example of the statistical mechanical description of the storage properties of a McCulloch-Pitts neuron. The technical details are reviewed extensively, with regard to the wide range of systems where the method may be applied. Parisi's partial differential equation and related differential equations are discussed, and a Green function technique introduced for the calculation of replica averages, the key to determining the averages of physical quantities. The ensuing graph rules involve only tree graphs, as appropriate for a mean-field-like model. The lowest order Ward-Takahashi identity is recovered analytically and is shown to lead to the Goldstone modes in continuous replica symmetry breaking phases. The need for a replica symmetry breaking theory in the storage problem of the neuron has arisen due to the thermodynamical instability of formerly given solutions. Variational forms for the neuron's free energy are derived in terms of the order parameter function x(q), for different prior distribution of synapses. Analytically in the high temperature limit and numerically in generic cases various phases are identified, among them one similar to the Parisi phase in the Sherrington-Kirkpatrick model. Extensive quantities like the error per pattern change slightly with respect to the known unstable solutions, but there is a significant difference in the distribution of non-extensive quantities like the synaptic overlaps and the pattern storage stability parameter. A simulation result is also reviewed and compared to the prediction of the theory.Comment: 103 Latex pages (with REVTeX 3.0), including 15 figures (ps, epsi, eepic), accepted for Physics Report

Techniques of replica symmetry breaking and the storage problem of the McCulloch-Pitts neuron

In this article the framework for Parisi's spontaneous replica symmetry breaking is reviewed, and subsequently applied to the example of the statistical mechanical description of the storage properties of a McCulloch-Pitts neuron. The technical details are reviewed extensively, with regard to the wide range of systems where the method may be applied. Parisi's partial differential equation and related differential equations are discussed, and a Green function technique introduced for the calculation of replica averages, the key to determining the averages of physical quantities. The ensuing graph rules involve only tree graphs, as appropriate for a mean-field-like model. The lowest order Ward-Takahashi identity is recovered analytically and is shown to lead to the Goldstone modes in continuous replica symmetry breaking phases. The need for a replica symmetry breaking theory in the storage problem of the neuron has arisen due to the thermodynamical instability of formerly given solutions. Variational forms for the neuron's free energy are derived in terms of the order parameter function x(q), for different prior distribution of synapses. Analytically in the high temperature limit and numerically in generic cases various phases are identified, among them one similar to the Parisi phase in the Sherrington-Kirkpatrick model. Extensive quantities like the error per pattern change slightly with respect to the known unstable solutions, but there is a significant difference in the distribution of non-extensive quantities like the synaptic overlaps and the pattern storage stability parameter. A simulation result is also reviewed and compared to the prediction of the theory.Comment: 103 Latex pages (with REVTeX 3.0), including 15 figures (ps, epsi, eepic), accepted for Physics Report

Entropic repulsion for the occupation-time field of random interlacements conditioned on disconnection

We investigate percolation of the vacant set of random interlacements on $\mathbb{Z}^d$, $d\geq 3$, in the strongly percolative regime. We consider the event that the interlacement set at level $u$ disconnects the discrete blow-up of a compact set $A\subseteq \mathbb{R}^d$ from the boundary of an enclosing box. We derive asymptotic large deviation upper bounds on the probability that the local averages of the occupation times deviate from a specific function depending on the harmonic potential of $A$, when disconnection occurs. If certain critical levels coincide, which is plausible but open at the moment, these bounds imply that conditionally on disconnection, the occupation-time profile undergoes an entropic push governed by a specific function depending on $A$. Similar entropic repulsion phenomena conditioned on disconnection by level-sets of the discrete Gaussian free field on $\mathbb{Z}^d$, $d \geq 3$, have been obtained by the authors in arxiv:1808.09947. Our proofs rely crucially on the `solidification estimates' developed in arXiv:1706.07229 by A.-S. Sznitman and the second author.Comment: 35 pages, 2 figures, accepted in the Annals of Probabilit

Weak Chaos from Tsallis Entropy

We present a geometric, model-independent, argument that aims to explain why the Tsallis entropy describes systems exhibiting "weak chaos", namely systems whose underlying dynamics has vanishing largest Lyapunov exponent. Our argument relies on properties of a deformation map of the reals induced by the Tsallis entropy, and its conclusion agrees with all currently known results.Comment: 19 pages, Standard LaTeX2e, v2: addition of the last paragraph in Section 4. Three additional refs. To be published in QScience Connec

Artificial Sequences and Complexity Measures

In this paper we exploit concepts of information theory to address the fundamental problem of identifying and defining the most suitable tools to extract, in a automatic and agnostic way, information from a generic string of characters. We introduce in particular a class of methods which use in a crucial way data compression techniques in order to define a measure of remoteness and distance between pairs of sequences of characters (e.g. texts) based on their relative information content. We also discuss in detail how specific features of data compression techniques could be used to introduce the notion of dictionary of a given sequence and of Artificial Text and we show how these new tools can be used for information extraction purposes. We point out the versatility and generality of our method that applies to any kind of corpora of character strings independently of the type of coding behind them. We consider as a case study linguistic motivated problems and we present results for automatic language recognition, authorship attribution and self consistent-classification.Comment: Revised version, with major changes, of previous "Data Compression approach to Information Extraction and Classification" by A. Baronchelli and V. Loreto. 15 pages; 5 figure