A perturbative approach to non-linearities in the information carried by a two layer neural network

We evaluate the mutual information between the input and the output of a two layer network in the case of a noisy and non-linear analogue channel. In the case where the non-linearity is small with respect to the variability in the noise, we derive an exact expression for the contribution to the mutual information given by the non-linear term in first order of perturbation theory. Finally we show how the calculation can be simplified by means of a diagrammatic expansion. Our results suggest that the use of perturbation theories applied to neural systems might give an insight on the contribution of non-linearities to the information transmission and in general to the neuronal dynamics.Comment: Accepted as a preprint of ICTP, Triest

Time evolution of the extremely diluted Blume-Emery-Griffiths neural network

The time evolution of the extremely diluted Blume-Emery-Griffiths neural network model is studied, and a detailed equilibrium phase diagram is obtained exhibiting pattern retrieval, fluctuation retrieval and self-sustained activity phases. It is shown that saddle-point solutions associated with fluctuation overlaps slow down considerably the flow of the network states towards the retrieval fixed points. A comparison of the performance with other three-state networks is also presented.Comment: 8 pages, 5 figure

Statistical mechanics of the multi-constraint continuous knapsack problem

We apply the replica analysis established by Gardner to the multi-constraint continuous knapsack problem,which is one of the linear programming problems and a most fundamental problem in the field of operations research (OR). For a large problem size, we analyse the space of solution and its volume, and estimate the optimal number of items to go into the knapsack as a function of the number of constraints. We study the stability of the replica symmetric (RS) solution and find that the RS calculation cannot estimate the optimal number of items in knapsack correctly if many constraints are required.In order to obtain a consistent solution in the RS region,we try the zero entropy approximation for this continuous solution space and get a stable solution within the RS ansatz.On the other hand, in replica symmetry breaking (RSB) region, the one step RSB solution is found by Parisi's scheme. It turns out that this problem is closely related to the problem of optimal storage capacity and of generalization by maximum stability rule of a spherical perceptron.Comment: Latex 13 pages using IOP style file, 5 figure

Bump formation in a binary attractor neural network

This paper investigates the conditions for the formation of local bumps in the activity of binary attractor neural networks with spatially dependent connectivity. We show that these formations are observed when asymmetry between the activity during the retrieval and learning is imposed. Analytical approximation for the order parameters is derived. The corresponding phase diagram shows a relatively large and stable region, where this effect is observed, although the critical storage and the information capacities drastically decrease inside that region. We demonstrate that the stability of the network, when starting from the bump formation, is larger than the stability when starting even from the whole pattern. Finally, we show a very good agreement between the analytical results and the simulations performed for different topologies of the network.Comment: about 14 page

The mutual information of a stochastic binary channel: validity of the Replica Symmetry Ansatz

We calculate the mutual information (MI) of a two-layered neural network with noiseless, continuous inputs and binary, stochastic outputs under several assumptions on the synaptic efficiencies. The interesting regime corresponds to the limit where the number of both input and output units is large but their ratio is kept fixed at a value $\alpha$. We first present a solution for the MI using the replica technique with a replica symmetric (RS) ansatz. Then we find an exact solution for this quantity valid in a neighborhood of $\alpha = 0$. An analysis of this solution shows that the system must have a phase transition at some finite value of $\alpha$. This transition shows a singularity in the third derivative of the MI. As the RS solution turns out to be infinitely differentiable, it could be regarded as a smooth approximation to the MI. This is checked numerically in the validity domain of the exact solution.Comment: Latex, 29 pages, 2 Encapsulated Post Script figures. To appear in Journal of Physics

Power spectra of self-organized critical sandpiles

We analyze the power spectra of avalanches in two classes of self-organized critical sandpile models, the Bak-Tang-Wiesenfeld model and the Manna model. We show that these decay with a $1/f^\alpha$ power law, where the exponent value $\alpha$ is significantly smaller than 2 and equals the scaling exponent relating the avalanche size to its duration. We discuss the basic ingredients behind this result, such as the scaling of the average avalanche shape.Comment: 7 pages, 3 figures, submitted to JSTA

System size resonance in an attractor neural network

We study the response of an attractor neural network, in the ferromagnetic phase, to an external, time-dependent stimulus, which drives the system periodically two different attractors. We demonstrate a non-trivial dependance of the system via a system size resonance, by showing a signal amplification maximum at a certain finite size.Comment: 7 pages, 9 figures, submitted to Europhys. Let

Explicit Renormalization Group for D=2 random bond Ising model with long-range correlated disorder

We investigate the explicit renormalization group for fermionic field theoretic representation of two-dimensional random bond Ising model with long-range correlated disorder. We show that a new fixed point appears by introducing a long-range correlated disorder. Such as the one has been observed in previous works for the bosonic ($\phi^4$) description. We have calculated the correlation length exponent and the anomalous scaling dimension of fermionic fields at this fixed point. Our results are in agreement with the extended Harris criterion derived by Weinrib and Halperin.Comment: 5 page