Connection Strategies in Associative Memory Models

“The original publication is available at www.springerlink.com”. Copyright Springer.The problem we address in this paper is that of finding effective and parsimonious patterns of connectivity in sparse associative memories. This problem must be addressed in real neuronal systems, so results in artificial systems could throw light on real systems. We show that there are efficient patterns of connectivity and that these patterns are effective in models with either spiking or non-spiking neurons. This suggests that there may be some underlying general principles governing good connectivity in such networks.Peer reviewe

Delay-independent stability in bidirectional associative memory networks

It is shown that if the neuronal gains are small compared with the synaptic connection weights, then a bidirectional associative memory network with axonal signal transmission delays converges to the equilibria associated with exogenous inputs to the network. Both discrete and continuously distributed delays are considered; the asymptotic stability is global in the state space of neuronal activations and also is independent of the delays

Preservation of long-term memory and synaptic plasticity despite short-term impairments in the Tc1 mouse model of Down syndrome

Down syndrome (DS) is a genetic disorder arising from the presence of a third copy of the human chromosome 21 (Hsa21). Recently, O’Doherty and colleagues in an earlier study generated a new genetic mouse model of DS (Tc1) that carries an almost complete Hsa21. Since DS is the most common genetic cause of mental retardation, we have undertaken a detailed analysis of cognitive function and synaptic plasticity in Tc1 mice. Here we show that Tc1 mice have impaired spatial working memory (WM) but spared long-term spatial reference memory (RM) in the Morris watermaze. Similarly, Tc1 mice are selectively impaired in short-term memory (STM) but have intact long-term memory (LTM) in the novel object recognition task. The pattern of impaired STM and normal LTM is paralleled by a corresponding phenotype in long-term potentiation (LTP). Freely-moving Tc1 mice exhibit reduced LTP 1 h after induction but normal maintenance over days in the dentate gyrus of the hippocampal formation. Biochemical analysis revealed a reduction in membrane surface expression of the AMPAR (α-amino-3-hydroxy-5-methyl-4-propionic acid receptor) subunit GluR1 in the hippocampus of Tc1 mice, suggesting a potential mechanism for the impairment in early LTP. Our observations also provide further evidence that STM and LTM for hippocampus-dependent tasks are subserved by parallel processing streams

Statistical mechanics of temporal association in neural networks with transmission delays

We study the representation of static patterns and temporal sequences in neural networks with signal delays and a stochastic parallel dynamics. For a wide class of delay distributions, the asymptotic network behavior can be described by a generalized Gibbs distribution, generated by a novel Lyapunov functional for the determination dynamics. We extend techniques of equilibrium statistical mechanics so as to deal with time-dependent phenomena, derive analytic results for both retrieval quality and storage capacity, and compare them with numerical simulations

Capacity, Fidelity, and Noise Tolerance of Associative Spatial-Temporal Memories Based on Memristive Neuromorphic Network

We have calculated the key characteristics of associative (content-addressable) spatial-temporal memories based on neuromorphic networks with restricted connectivity - "CrossNets". Such networks may be naturally implemented in nanoelectronic hardware using hybrid CMOS/memristor circuits, which may feature extremely high energy efficiency, approaching that of biological cortical circuits, at much higher operation speed. Our numerical simulations, in some cases confirmed by analytical calculations, have shown that the characteristics depend substantially on the method of information recording into the memory. Of the four methods we have explored, two look especially promising - one based on the quadratic programming, and the other one being a specific discrete version of the gradient descent. The latter method provides a slightly lower memory capacity (at the same fidelity) then the former one, but it allows local recording, which may be more readily implemented in nanoelectronic hardware. Most importantly, at the synchronous retrieval, both methods provide a capacity higher than that of the well-known Ternary Content-Addressable Memories with the same number of nonvolatile memory cells (e.g., memristors), though the input noise immunity of the CrossNet memories is somewhat lower

Adaptive Timing, Attention, and Movement Control

Examples of how LTP and LTD can control adaptively timed learning that modulates attention and motor control are given. It is also suggested that LTP/LTD can also play a role in storing memories. The distinction between match-based and mismatch-based learning may help to clarity the difference.Defense Advanced Research Projects Agency; Office of Naval Research (N00014-92-J-1209, N00014-95-1-0409, N00014-95-1-0657

The hippocampus and cerebellum in adaptively timed learning, recognition, and movement

The concepts of declarative memory and procedural memory have been used to distinguish two basic types of learning. A neural network model suggests how such memory processes work together as recognition learning, reinforcement learning, and sensory-motor learning take place during adaptive behaviors. To coordinate these processes, the hippocampal formation and cerebellum each contain circuits that learn to adaptively time their outputs. Within the model, hippocampal timing helps to maintain attention on motivationally salient goal objects during variable task-related delays, and cerebellar timing controls the release of conditioned responses. This property is part of the model's description of how cognitive-emotional interactions focus attention on motivationally valued cues, and how this process breaks down due to hippocampal ablation. The model suggests that the hippocampal mechanisms that help to rapidly draw attention to salient cues could prematurely release motor commands were not the release of these commands adaptively timed by the cerebellum. The model hippocampal system modulates cortical recognition learning without actually encoding the representational information that the cortex encodes. These properties avoid the difficulties faced by several models that propose a direct hippocampal role in recognition learning. Learning within the model hippocampal system controls adaptive timing and spatial orientation. Model properties hereby clarify how hippocampal ablations cause amnesic symptoms and difficulties with tasks which combine task delays, novelty detection, and attention towards goal objects amid distractions. When these model recognition, reinforcement, sensory-motor, and timing processes work together, they suggest how the brain can accomplish conditioning of multiple sensory events to delayed rewards, as during serial compound conditioning.Air Force Office of Scientific Research (F49620-92-J-0225, F49620-86-C-0037, 90-0128); Advanced Research Projects Agency (ONR N00014-92-J-4015); Office of Naval Research (N00014-91-J-4100, N00014-92-J-1309, N00014-92-J-1904); National Institute of Mental Health (MH-42900