Diffusion, localization and dispersion relations on ``small-world'' lattices

The spectral properties of the Laplacian operator on ``small-world'' lattices, that is mixtures of unidimensional chains and random graphs structures are investigated numerically and analytically. A transfer matrix formalism including a self-consistent potential a la Edwards is introduced. In the extended region of the spectrum, an effective medium calculation provides the density of states and pseudo relations of dispersion for the eigenmodes in close agreement with the simulations. Localization effects, which are due to connectivity fluctuations of the sites are shown to be quantitatively described by the single defect approximation recently introduced for random graphs.Comment: 17 revtex pages, 16 eps figures + 2 table

Emergence of Compositional Representations in Restricted Boltzmann Machines

Extracting automatically the complex set of features composing real high-dimensional data is crucial for achieving high performance in machine--learning tasks. Restricted Boltzmann Machines (RBM) are empirically known to be efficient for this purpose, and to be able to generate distributed and graded representations of the data. We characterize the structural conditions (sparsity of the weights, low effective temperature, nonlinearities in the activation functions of hidden units, and adaptation of fields maintaining the activity in the visible layer) allowing RBM to operate in such a compositional phase. Evidence is provided by the replica analysis of an adequate statistical ensemble of random RBMs and by RBM trained on the handwritten digits dataset MNIST.Comment: Supplementary material available at the authors' webpag

Fast Inference of Interactions in Assemblies of Stochastic Integrate-and-Fire Neurons from Spike Recordings

We present two Bayesian procedures to infer the interactions and external currents in an assembly of stochastic integrate-and-fire neurons from the recording of their spiking activity. The first procedure is based on the exact calculation of the most likely time courses of the neuron membrane potentials conditioned by the recorded spikes, and is exact for a vanishing noise variance and for an instantaneous synaptic integration. The second procedure takes into account the presence of fluctuations around the most likely time courses of the potentials, and can deal with moderate noise levels. The running time of both procedures is proportional to the number S of spikes multiplied by the squared number N of neurons. The algorithms are validated on synthetic data generated by networks with known couplings and currents. We also reanalyze previously published recordings of the activity of the salamander retina (including from 32 to 40 neurons, and from 65,000 to 170,000 spikes). We study the dependence of the inferred interactions on the membrane leaking time; the differences and similarities with the classical cross-correlation analysis are discussed.Comment: Accepted for publication in J. Comput. Neurosci. (dec 2010

The Entropy of the K-Satisfiability Problem

The threshold behaviour of the K-Satisfiability problem is studied in the framework of the statistical mechanics of random diluted systems. We find that at the transition the entropy is finite and hence that the transition itself is due to the abrupt appearance of logical contradictions in all solutions and not to the progressive decreasing of the number of these solutions down to zero. A physical interpretation is given for the different cases $K=1$, $K=2$ and $K \geq 3$.Comment: revtex, 11 pages + 1 figur

Weight Space Structure and Internal Representations: a Direct Approach to Learning and Generalization in Multilayer Neural Network

We analytically derive the geometrical structure of the weight space in multilayer neural networks (MLN), in terms of the volumes of couplings associated to the internal representations of the training set. Focusing on the parity and committee machines, we deduce their learning and generalization capabilities both reinterpreting some known properties and finding new exact results. The relationship between our approach and information theory as well as the Mitchison--Durbin calculation is established. Our results are exact in the limit of a large number of hidden units, showing that MLN are a class of exactly solvable models with a simple interpretation of replica symmetry breaking.Comment: 12 pages, 1 compressed ps figure (uufile), RevTeX fil

Theoretical study of collective modes in DNA at ambient temperature

The instantaneous normal modes corresponding to base pair vibrations (radial modes) and twist angle fluctuations (angular modes) of a DNA molecule model at ambient temperature are theoretically investigated. Due to thermal disorder, normal modes are not plane waves with a single wave number q but have a finite and frequency dependent damping width. The density of modes rho(nu), the average dispersion relation nu(q) as well as the coherence length xi(nu) are analytically calculated. The Gibbs averaged resolvent is computed using a replicated transfer matrix formalism and variational wave functions for the ground and first excited state. Our results for the density of modes are compared to Raman spectroscopy measurements of the collective modes for DNA in solution and show a good agreement with experimental data in the low frequency regime nu < 150 cm^{-1}. Radial modes extend over frequencies ranging from 50 cm^{-1} to 110 cm^{-1}. Angular modes, related to helical axis vibrations are limited to nu < 25 cm^{-1}. Normal modes are highly disordered and coherent over a few base pairs only (xi < 2 nm) in good agreement with neutron scattering experiments.Comment: 20 pages + 13 ps figure