The Hopfield model and its role in the development of synthetic biology

Neural network models make extensive use of concepts coming from physics and engineering. How do scientists justify the use of these concepts in the representation of biological systems? How is evidence for or against the use of these concepts produced in the application and manipulation of the models? It will be shown in this article that neural network models are evaluated differently depending on the scientific context and its modeling practice. In the case of the Hopfield model, the different modeling practices related to theoretical physics and neurobiology played a central role for how the model was received and used in the different scientific communities. In theoretical physics, where the Hopfield model has its roots, mathematical modeling is much more common and established than in neurobiology which is strongly experiment driven. These differences in modeling practice contributed to the development of the new field of synthetic biology which introduced a third type of model which combines mathematical modeling and experimenting on biological systems and by doing so mediates between the different modeling practices

Modeling Quantum Mechanical Observers via Neural-Glial Networks

We investigate the theory of observers in the quantum mechanical world by using a novel model of the human brain which incorporates the glial network into the Hopfield model of the neural network. Our model is based on a microscopic construction of a quantum Hamiltonian of the synaptic junctions. Using the Eguchi-Kawai large N reduction, we show that, when the number of neurons and astrocytes is exponentially large, the degrees of freedom of the dynamics of the neural and glial networks can be completely removed and, consequently, that the retention time of the superposition of the wave functions in the brain is as long as that of the microscopic quantum system of pre-synaptics sites. Based on this model, the classical information entropy of the neural-glial network is introduced. Using this quantity, we propose a criterion for the brain to be a quantum mechanical observer.Comment: 24 pages, published versio

Statistical Physics and Representations in Real and Artificial Neural Networks

This document presents the material of two lectures on statistical physics and neural representations, delivered by one of us (R.M.) at the Fundamental Problems in Statistical Physics XIV summer school in July 2017. In a first part, we consider the neural representations of space (maps) in the hippocampus. We introduce an extension of the Hopfield model, able to store multiple spatial maps as continuous, finite-dimensional attractors. The phase diagram and dynamical properties of the model are analyzed. We then show how spatial representations can be dynamically decoded using an effective Ising model capturing the correlation structure in the neural data, and compare applications to data obtained from hippocampal multi-electrode recordings and by (sub)sampling our attractor model. In a second part, we focus on the problem of learning data representations in machine learning, in particular with artificial neural networks. We start by introducing data representations through some illustrations. We then analyze two important algorithms, Principal Component Analysis and Restricted Boltzmann Machines, with tools from statistical physics

Synthesis of neural networks for spatio-temporal spike pattern recognition and processing

The advent of large scale neural computational platforms has highlighted the lack of algorithms for synthesis of neural structures to perform predefined cognitive tasks. The Neural Engineering Framework offers one such synthesis, but it is most effective for a spike rate representation of neural information, and it requires a large number of neurons to implement simple functions. We describe a neural network synthesis method that generates synaptic connectivity for neurons which process time-encoded neural signals, and which makes very sparse use of neurons. The method allows the user to specify, arbitrarily, neuronal characteristics such as axonal and dendritic delays, and synaptic transfer functions, and then solves for the optimal input-output relationship using computed dendritic weights. The method may be used for batch or online learning and has an extremely fast optimization process. We demonstrate its use in generating a network to recognize speech which is sparsely encoded as spike times.Comment: In submission to Frontiers in Neuromorphic Engineerin

Free energies of Boltzmann Machines: self-averaging, annealed and replica symmetric approximations in the thermodynamic limit

Restricted Boltzmann machines (RBMs) constitute one of the main models for machine statistical inference and they are widely employed in Artificial Intelligence as powerful tools for (deep) learning. However, in contrast with countless remarkable practical successes, their mathematical formalization has been largely elusive: from a statistical-mechanics perspective these systems display the same (random) Gibbs measure of bi-partite spin-glasses, whose rigorous treatment is notoriously difficult. In this work, beyond providing a brief review on RBMs from both the learning and the retrieval perspectives, we aim to contribute to their analytical investigation, by considering two distinct realizations of their weights (i.e., Boolean and Gaussian) and studying the properties of their related free energies. More precisely, focusing on a RBM characterized by digital couplings, we first extend the Pastur-Shcherbina-Tirozzi method (originally developed for the Hopfield model) to prove the self-averaging property for the free energy, over its quenched expectation, in the infinite volume limit, then we explicitly calculate its simplest approximation, namely its annealed bound. Next, focusing on a RBM characterized by analogical weights, we extend Guerra's interpolating scheme to obtain a control of the quenched free-energy under the assumption of replica symmetry: we get self-consistencies for the order parameters (in full agreement with the existing Literature) as well as the critical line for ergodicity breaking that turns out to be the same obtained in AGS theory. As we discuss, this analogy stems from the slow-noise universality. Finally, glancing beyond replica symmetry, we analyze the fluctuations of the overlaps for an estimate of the (slow) noise affecting the retrieval of the signal, and by a stability analysis we recover the Aizenman-Contucci identities typical of glassy systems.Comment: 21 pages, 1 figur