49 research outputs found
The Relativistic Hopfield network: rigorous results
The relativistic Hopfield model constitutes a generalization of the standard
Hopfield model that is derived by the formal analogy between the
statistical-mechanic framework embedding neural networks and the Lagrangian
mechanics describing a fictitious single-particle motion in the space of the
tuneable parameters of the network itself. In this analogy the cost-function of
the Hopfield model plays as the standard kinetic-energy term and its related
Mattis overlap (naturally bounded by one) plays as the velocity. The
Hamiltonian of the relativisitc model, once Taylor-expanded, results in a
P-spin series with alternate signs: the attractive contributions enhance the
information-storage capabilities of the network, while the repulsive
contributions allow for an easier unlearning of spurious states, conferring
overall more robustness to the system as a whole. Here we do not deepen the
information processing skills of this generalized Hopfield network, rather we
focus on its statistical mechanical foundation. In particular, relying on
Guerra's interpolation techniques, we prove the existence of the infinite
volume limit for the model free-energy and we give its explicit expression in
terms of the Mattis overlaps. By extremizing the free energy over the latter we
get the generalized self-consistent equations for these overlaps, as well as a
picture of criticality that is further corroborated by a fluctuation analysis.
These findings are in full agreement with the available previous results.Comment: 11 pages, 1 figur
How spiking neurons give rise to a temporal-feature map
A temporal-feature map is a topographic neuronal representation of temporal attributes of phenomena or objects that occur in the outside world. We explain the evolution of such maps by means of a spike-based Hebbian learning rule in conjunction with a presynaptically unspecific contribution in that, if a synapse changes, then all other synapses connected to the same axon change by a small fraction as well. The learning equation is solved for the case of an array of Poisson neurons. We discuss the evolution of a temporal-feature map and the synchronization of the single cellsβ synaptic structures, in dependence upon the strength of presynaptic unspecific learning. We also give an upper bound for the magnitude of the presynaptic interaction by estimating its impact on the noise level of synaptic growth. Finally, we compare the results with those obtained from a learning equation for nonlinear neurons and show that synaptic structure formation may profit
from the nonlinearity
Coverage, Continuity and Visual Cortical Architecture
The primary visual cortex of many mammals contains a continuous
representation of visual space, with a roughly repetitive aperiodic map of
orientation preferences superimposed. It was recently found that orientation
preference maps (OPMs) obey statistical laws which are apparently invariant
among species widely separated in eutherian evolution. Here, we examine whether
one of the most prominent models for the optimization of cortical maps, the
elastic net (EN) model, can reproduce this common design. The EN model
generates representations which optimally trade of stimulus space coverage and
map continuity. While this model has been used in numerous studies, no
analytical results about the precise layout of the predicted OPMs have been
obtained so far. We present a mathematical approach to analytically calculate
the cortical representations predicted by the EN model for the joint mapping of
stimulus position and orientation. We find that in all previously studied
regimes, predicted OPM layouts are perfectly periodic. An unbiased search
through the EN parameter space identifies a novel regime of aperiodic OPMs with
pinwheel densities lower than found in experiments. In an extreme limit,
aperiodic OPMs quantitatively resembling experimental observations emerge.
Stabilization of these layouts results from strong nonlocal interactions rather
than from a coverage-continuity-compromise. Our results demonstrate that
optimization models for stimulus representations dominated by nonlocal
suppressive interactions are in principle capable of correctly predicting the
common OPM design. They question that visual cortical feature representations
can be explained by a coverage-continuity-compromise.Comment: 100 pages, including an Appendix, 21 + 7 figure
Refinement and Pattern Formation in Neural Circuits by the Interaction of Traveling Waves with Spike-Timing Dependent Plasticity
Traveling waves in the developing brain are a prominent source of highly correlated spiking activity that may instruct the refinement of neural circuits. A candidate mechanism for mediating such refinement is spike-timing dependent plasticity (STDP), which translates correlated activity patterns into changes in synaptic strength. To assess the potential of these phenomena to build useful structure in developing neural circuits, we examined the interaction of wave activity with STDP rules in simple, biologically plausible models of spiking neurons. We derive an expression for the synaptic strength dynamics showing that, by mapping the time dependence of STDP into spatial interactions, traveling waves can build periodic synaptic connectivity patterns into feedforward circuits with a broad class of experimentally observed STDP rules. The spatial scale of the connectivity patterns increases with wave speed and STDP time constants. We verify these results with simulations and demonstrate their robustness to likely sources of noise. We show how this pattern formation ability, which is analogous to solutions of reaction-diffusion systems that have been widely applied to biological pattern formation, can be harnessed to instruct the refinement of postsynaptic receptive fields. Our results hold for rich, complex wave patterns in two dimensions and over several orders of magnitude in wave speeds and STDP time constants, and they provide predictions that can be tested under existing experimental paradigms. Our model generalizes across brain areas and STDP rules, allowing broad application to the ubiquitous occurrence of traveling waves and to wave-like activity patterns induced by moving stimuli
Natural Image Coding in V1: How Much Use is Orientation Selectivity?
Orientation selectivity is the most striking feature of simple cell coding in
V1 which has been shown to emerge from the reduction of higher-order
correlations in natural images in a large variety of statistical image models.
The most parsimonious one among these models is linear Independent Component
Analysis (ICA), whereas second-order decorrelation transformations such as
Principal Component Analysis (PCA) do not yield oriented filters. Because of
this finding it has been suggested that the emergence of orientation
selectivity may be explained by higher-order redundancy reduction. In order to
assess the tenability of this hypothesis, it is an important empirical question
how much more redundancies can be removed with ICA in comparison to PCA, or
other second-order decorrelation methods. This question has not yet been
settled, as over the last ten years contradicting results have been reported
ranging from less than five to more than hundred percent extra gain for ICA.
Here, we aim at resolving this conflict by presenting a very careful and
comprehensive analysis using three evaluation criteria related to redundancy
reduction: In addition to the multi-information and the average log-loss we
compute, for the first time, complete rate-distortion curves for ICA in
comparison with PCA. Without exception, we find that the advantage of the ICA
filters is surprisingly small. Furthermore, we show that a simple spherically
symmetric distribution with only two parameters can fit the data even better
than the probabilistic model underlying ICA. Since spherically symmetric models
are agnostic with respect to the specific filter shapes, we conlude that
orientation selectivity is unlikely to play a critical role for redundancy
reduction
On the Origin of the Functional Architecture of the Cortex
The basic structure of receptive fields and functional maps in primary visual cortex is established without exposure to normal sensory experience and before the onset of the critical period. How the brain wires these circuits in the early stages of development remains unknown. Possible explanations include activity-dependent mechanisms driven by spontaneous activity in the retina and thalamus, and molecular guidance orchestrating thalamo-cortical connections on a fine spatial scale. Here I propose an alternative hypothesis: the blueprint for receptive fields, feature maps, and their inter-relationships may reside in the layout of the retinal ganglion cell mosaics along with a simple statistical connectivity scheme dictating the wiring between thalamus and cortex. The model is shown to account for a number of experimental findings, including the relationship between retinotopy, orientation maps, spatial frequency maps and cytochrome oxidase patches. The theory's simplicity, explanatory and predictive power makes it a serious candidate for the origin of the functional architecture of primary visual cortex