On Cavity Approximations for Graphical Models

We reformulate the Cavity Approximation (CA), a class of algorithms recently introduced for improving the Bethe approximation estimates of marginals in graphical models. In our new formulation, which allows for the treatment of multivalued variables, a further generalization to factor graphs with arbitrary order of interaction factors is explicitly carried out, and a message passing algorithm that implements the first order correction to the Bethe approximation is described. Furthermore we investigate an implementation of the CA for pairwise interactions. In all cases considered we could confirm that CA[k] with increasing $k$ provides a sequence of approximations of markedly increasing precision. Furthermore in some cases we could also confirm the general expectation that the approximation of order $k$, whose computational complexity is $O(N^{k+1})$ has an error that scales as $1/N^{k+1}$ with the size of the system. We discuss the relation between this approach and some recent developments in the field.Comment: Extension to factor graphs and comments on related work adde

Slowly evolving random graphs II: Adaptive geometry in finite-connectivity Hopfield models

We present an analytically solvable random graph model in which the connections between the nodes can evolve in time, adiabatically slowly compared to the dynamics of the nodes. We apply the formalism to finite connectivity attractor neural network (Hopfield) models and we show that due to the minimisation of the frustration effects the retrieval region of the phase diagram can be significantly enlarged. Moreover, the fraction of misaligned spins is reduced by this effect, and is smaller than in the infinite connectivity regime. The main cause of this difference is found to be the non-zero fraction of sites with vanishing local field when the connectivity is finite.Comment: 17 pages, 8 figure

Spin-glass behaviour on random lattices

The ground-state phase diagram of an Ising spin-glass model on a random graph with an arbitrary fraction $w$ of ferromagnetic interactions is analysed in the presence of an external field. Using the replica method, and performing an analysis of stability of the replica-symmetric solution, it is shown that $w=1/2$, correponding to an unbiased spin glass, is a singular point in the phase diagram, separating a region with a spin-glass phase ($w<1/2$) from a region with spin-glass, ferromagnetic, mixed, and paramagnetic phases ($w>1/2$)

Slowly evolving geometry in recurrent neural networks I: extreme dilution regime

We study extremely diluted spin models of neural networks in which the connectivity evolves in time, although adiabatically slowly compared to the neurons, according to stochastic equations which on average aim to reduce frustration. The (fast) neurons and (slow) connectivity variables equilibrate separately, but at different temperatures. Our model is exactly solvable in equilibrium. We obtain phase diagrams upon making the condensed ansatz (i.e. recall of one pattern). These show that, as the connectivity temperature is lowered, the volume of the retrieval phase diverges and the fraction of mis-aligned spins is reduced. Still one always retains a region in the retrieval phase where recall states other than the one corresponding to the `condensed' pattern are locally stable, so the associative memory character of our model is preserved.Comment: 18 pages, 6 figure

Analogue neural networks on correlated random graphs

We consider a generalization of the Hopfield model, where the entries of patterns are Gaussian and diluted. We focus on the high-storage regime and we investigate analytically the topological properties of the emergent network, as well as the thermodynamic properties of the model. We find that, by properly tuning the dilution in the pattern entries, the network can recover different topological regimes characterized by peculiar scalings of the average coordination number with respect to the system size. The structure is also shown to exhibit a large degree of cliquishness, even when very sparse. Moreover, we obtain explicitly the replica symmetric free energy and the self-consistency equations for the overlaps (order parameters of the theory), which turn out to be classical weighted sums of 'sub-overlaps' defined on all possible sub-graphs. Finally, a study of criticality is performed through a small-overlap expansion of the self-consistencies and through a whole fluctuation theory developed for their rescaled correlations: Both approaches show that the net effect of dilution in pattern entries is to rescale the critical noise level at which ergodicity breaks down.Comment: 34 pages, 3 figure

Equilibrium statistical mechanics on correlated random graphs

Biological and social networks have recently attracted enormous attention between physicists. Among several, two main aspects may be stressed: A non trivial topology of the graph describing the mutual interactions between agents exists and/or, typically, such interactions are essentially (weighted) imitative. Despite such aspects are widely accepted and empirically confirmed, the schemes currently exploited in order to generate the expected topology are based on a-priori assumptions and in most cases still implement constant intensities for links. Here we propose a simple shift in the definition of patterns in an Hopfield model to convert frustration into dilution: By varying the bias of the pattern distribution, the network topology -which is generated by the reciprocal affinities among agents - crosses various well known regimes (fully connected, linearly diverging connectivity, extreme dilution scenario, no network), coupled with small world properties, which, in this context, are emergent and no longer imposed a-priori. The model is investigated at first focusing on these topological properties of the emergent network, then its thermodynamics is analytically solved (at a replica symmetric level) by extending the double stochastic stability technique, and presented together with its fluctuation theory for a picture of criticality. At least at equilibrium, dilution simply decreases the strength of the coupling felt by the spins, but leaves the paramagnetic/ferromagnetic flavors unchanged. The main difference with respect to previous investigations and a naive picture is that within our approach replicas do not appear: instead of (multi)-overlaps as order parameters, we introduce a class of magnetizations on all the possible sub-graphs belonging to the main one investigated: As a consequence, for these objects a closure for a self-consistent relation is achieved.Comment: 30 pages, 4 figure

The cavity method for large deviations

A method is introduced for studying large deviations in the context of statistical physics of disordered systems. The approach, based on an extension of the cavity method to atypical realizations of the quenched disorder, allows us to compute exponentially small probabilities (rate functions) over different classes of random graphs. It is illustrated with two combinatorial optimization problems, the vertex-cover and coloring problems, for which the presence of replica symmetry breaking phases is taken into account. Applications include the analysis of models on adaptive graph structures.Comment: 18 pages, 7 figure

Phase variance of squeezed vacuum states

We consider the problem of estimating the phase of squeezed vacuum states within a Bayesian framework. We derive bounds on the average Holevo variance for an arbitrary number $N$ of uncorrelated copies. We find that it scales with the mean photon number, $n$, as dictated by the Heisenberg limit, i.e., as $n^{-2}$, only for $N>4$. For $N\leq 4$ this fundamental scaling breaks down and it becomes $n^{-N/2}$. Thus, a single squeezed vacuum state performs worse than a single coherent state with the same energy. We find the optimal splitting of a fixed given energy among various copies. We also compute the variance for repeated individual measurements (without classical communication or adaptivity) and find that the standard Heisenberg-limited scaling $n^{-2}$ is recovered for large samples.Comment: Minor changes, version to appear in PRA, 8 pages, 2 figure

Statistical Mechanics of Dilute Batch Minority Games with Random External Information

We study the dynamics and statics of a dilute batch minority game with random external information. We focus on the case in which the number of connections per agent is infinite in the thermodynamic limit. The dynamical scenario of ergodicity breaking in this model is different from the phase transition in the standard minority game and is characterised by the onset of long-term memory at finite integrated response. We demonstrate that finite memory appears at the AT-line obtained from the corresponding replica calculation, and compare the behaviour of the dilute model with the minority game with market impact correction, which is known to exhibit similar features.Comment: 22 pages, 6 figures, text modified, references updated and added, figure added, typos correcte