80,015 research outputs found
Statistical Mechanics of Recurrent Neural Networks I. Statics
A lecture notes style review of the equilibrium statistical mechanics of
recurrent neural networks with discrete and continuous neurons (e.g. Ising,
coupled-oscillators). To be published in the Handbook of Biological Physics
(North-Holland). Accompanied by a similar review (part II) dealing with the
dynamics.Comment: 49 pages, LaTe
Topological properties of hierarchical networks
Hierarchical networks are attracting a renewal interest for modelling the
organization of a number of biological systems and for tackling the complexity
of statistical mechanical models beyond mean-field limitations. Here we
consider the Dyson hierarchical construction for ferromagnets, neural networks
and spin-glasses, recently analyzed from a statistical-mechanics perspective,
and we focus on the topological properties of the underlying structures. In
particular, we find that such structures are weighted graphs that exhibit high
degree of clustering and of modularity, with small spectral gap; the robustness
of such features with respect to link removal is also studied. These outcomes
are then discussed and related to the statistical mechanics scenario in full
consistency. Lastly, we look at these weighted graphs as Markov chains and we
show that in the limit of infinite size, the emergence of ergodicity breakdown
for the stochastic process mirrors the emergence of meta-stabilities in the
corresponding statistical mechanical analysis
Multitasking associative networks
We introduce a bipartite, diluted and frustrated, network as a sparse
restricted Boltzman machine and we show its thermodynamical equivalence to an
associative working memory able to retrieve multiple patterns in parallel
without falling into spurious states typical of classical neural networks. We
focus on systems processing in parallel a finite (up to logarithmic growth in
the volume) amount of patterns, mirroring the low-level storage of standard
Amit-Gutfreund-Sompolinsky theory. Results obtained trough statistical
mechanics, signal-to-noise technique and Monte Carlo simulations are overall in
perfect agreement and carry interesting biological insights. Indeed, these
associative networks pave new perspectives in the understanding of multitasking
features expressed by complex systems, e.g. neural and immune networks.Comment: to appear on Phys.Rev.Let
A walk in the statistical mechanical formulation of neural networks
Neural networks are nowadays both powerful operational tools (e.g., for
pattern recognition, data mining, error correction codes) and complex
theoretical models on the focus of scientific investigation. As for the
research branch, neural networks are handled and studied by psychologists,
neurobiologists, engineers, mathematicians and theoretical physicists. In
particular, in theoretical physics, the key instrument for the quantitative
analysis of neural networks is statistical mechanics. From this perspective,
here, we first review attractor networks: starting from ferromagnets and
spin-glass models, we discuss the underlying philosophy and we recover the
strand paved by Hopfield, Amit-Gutfreund-Sompolinky. One step forward, we
highlight the structural equivalence between Hopfield networks (modeling
retrieval) and Boltzmann machines (modeling learning), hence realizing a deep
bridge linking two inseparable aspects of biological and robotic spontaneous
cognition. As a sideline, in this walk we derive two alternative (with respect
to the original Hebb proposal) ways to recover the Hebbian paradigm, stemming
from ferromagnets and from spin-glasses, respectively. Further, as these notes
are thought of for an Engineering audience, we highlight also the mappings
between ferromagnets and operational amplifiers and between antiferromagnets
and flip-flops (as neural networks -built by op-amp and flip-flops- are
particular spin-glasses and the latter are indeed combinations of ferromagnets
and antiferromagnets), hoping that such a bridge plays as a concrete
prescription to capture the beauty of robotics from the statistical mechanical
perspective.Comment: Contribute to the proceeding of the conference: NCTA 2014. Contains
12 pages,7 figure
- …