10,430 research outputs found
Neural Distributed Autoassociative Memories: A Survey
Introduction. Neural network models of autoassociative, distributed memory
allow storage and retrieval of many items (vectors) where the number of stored
items can exceed the vector dimension (the number of neurons in the network).
This opens the possibility of a sublinear time search (in the number of stored
items) for approximate nearest neighbors among vectors of high dimension. The
purpose of this paper is to review models of autoassociative, distributed
memory that can be naturally implemented by neural networks (mainly with local
learning rules and iterative dynamics based on information locally available to
neurons). Scope. The survey is focused mainly on the networks of Hopfield,
Willshaw and Potts, that have connections between pairs of neurons and operate
on sparse binary vectors. We discuss not only autoassociative memory, but also
the generalization properties of these networks. We also consider neural
networks with higher-order connections and networks with a bipartite graph
structure for non-binary data with linear constraints. Conclusions. In
conclusion we discuss the relations to similarity search, advantages and
drawbacks of these techniques, and topics for further research. An interesting
and still not completely resolved question is whether neural autoassociative
memories can search for approximate nearest neighbors faster than other index
structures for similarity search, in particular for the case of very high
dimensional vectors.Comment: 31 page
On the number of spurious memories in the Hopfield model
The outer-product method for programming the Hopfield model is discussed. The method can result in many spurious stable states-exponential in the number of vectors that are to be stored-even in the case when the vectors are orthogonal
An associative network with spatially organized connectivity
We investigate the properties of an autoassociative network of
threshold-linear units whose synaptic connectivity is spatially structured and
asymmetric. Since the methods of equilibrium statistical mechanics cannot be
applied to such a network due to the lack of a Hamiltonian, we approach the
problem through a signal-to-noise analysis, that we adapt to spatially
organized networks. The conditions are analyzed for the appearance of stable,
spatially non-uniform profiles of activity with large overlaps with one of the
stored patterns. It is also shown, with simulations and analytic results, that
the storage capacity does not decrease much when the connectivity of the
network becomes short range. In addition, the method used here enables us to
calculate exactly the storage capacity of a randomly connected network with
arbitrary degree of dilution.Comment: 27 pages, 6 figures; Accepted for publication in JSTA
An analog feedback associative memory
A method for the storage of analog vectors, i.e., vectors whose components are real-valued, is developed for the Hopfield continuous-time network. An important requirement is that each memory vector has to be an asymptotically stable (i.e. attractive) equilibrium of the network. Some of the limitations imposed by the continuous Hopfield model on the set of vectors that can be stored are pointed out. These limitations can be relieved by choosing a network containing visible as well as hidden units. An architecture consisting of several hidden layers and a visible layer, connected in a circular fashion, is considered. It is proved that the two-layer case is guaranteed to store any number of given analog vectors provided their number does not exceed 1 + the number of neurons in the hidden layer. A learning algorithm that correctly adjusts the locations of the equilibria and guarantees their asymptotic stability is developed. Simulation results confirm the effectiveness of the approach
Localized activity profiles and storage capacity of rate-based autoassociative networks
We study analytically the effect of metrically structured connectivity on the
behavior of autoassociative networks. We focus on three simple rate-based model
neurons: threshold-linear, binary or smoothly saturating units. For a
connectivity which is short range enough the threshold-linear network shows
localized retrieval states. The saturating and binary models also exhibit
spatially modulated retrieval states if the highest activity level that they
can achieve is above the maximum activity of the units in the stored patterns.
In the zero quenched noise limit, we derive an analytical formula for the
critical value of the connectivity width below which one observes spatially
non-uniform retrieval states. Localization reduces storage capacity, but only
by a factor of 2~3. The approach that we present here is generic in the sense
that there are no specific assumptions on the single unit input-output function
nor on the exact connectivity structure.Comment: 4 pages, 4 figure
Adiabatic Quantum Optimization for Associative Memory Recall
Hopfield networks are a variant of associative memory that recall information
stored in the couplings of an Ising model. Stored memories are fixed points for
the network dynamics that correspond to energetic minima of the spin state. We
formulate the recall of memories stored in a Hopfield network using energy
minimization by adiabatic quantum optimization (AQO). Numerical simulations of
the quantum dynamics allow us to quantify the AQO recall accuracy with respect
to the number of stored memories and the noise in the input key. We also
investigate AQO performance with respect to how memories are stored in the
Ising model using different learning rules. Our results indicate that AQO
performance varies strongly with learning rule due to the changes in the energy
landscape. Consequently, learning rules offer indirect methods for
investigating change to the computational complexity of the recall task and the
computational efficiency of AQO.Comment: 22 pages, 11 figures. Updated for clarity and figures, to appear in
Frontiers of Physic
- …