Neural Distributed Autoassociative Memories: A Survey

Introduction. Neural network models of autoassociative, distributed memory allow storage and retrieval of many items (vectors) where the number of stored items can exceed the vector dimension (the number of neurons in the network). This opens the possibility of a sublinear time search (in the number of stored items) for approximate nearest neighbors among vectors of high dimension. The purpose of this paper is to review models of autoassociative, distributed memory that can be naturally implemented by neural networks (mainly with local learning rules and iterative dynamics based on information locally available to neurons). Scope. The survey is focused mainly on the networks of Hopfield, Willshaw and Potts, that have connections between pairs of neurons and operate on sparse binary vectors. We discuss not only autoassociative memory, but also the generalization properties of these networks. We also consider neural networks with higher-order connections and networks with a bipartite graph structure for non-binary data with linear constraints. Conclusions. In conclusion we discuss the relations to similarity search, advantages and drawbacks of these techniques, and topics for further research. An interesting and still not completely resolved question is whether neural autoassociative memories can search for approximate nearest neighbors faster than other index structures for similarity search, in particular for the case of very high dimensional vectors.Comment: 31 page

NASA JSC neural network survey results

A survey of Artificial Neural Systems in support of NASA's (Johnson Space Center) Automatic Perception for Mission Planning and Flight Control Research Program was conducted. Several of the world's leading researchers contributed papers containing their most recent results on artificial neural systems. These papers were broken into categories and descriptive accounts of the results make up a large part of this report. Also included is material on sources of information on artificial neural systems such as books, technical reports, software tools, etc

Brain mechanisms of successful recognition through retrieval of semantic context

Episodic memory is associated with the encoding and retrieval of context information and with a subjective sense of reexperiencing past events. The neural correlates of episodic retrieval have been extensively studied using fMRI, leading to the identification of a "general recollection network" including medial temporal, parietal, and prefrontal regions. However, in these studies, it is difficult to disentangle the effects of context retrieval from recollection. In this study, we used fMRI to determine the extent to which the recruitment of regions in the recollection network is contingent on context reinstatement. Participants were scanned during a cued recognition test for target words from encoded sentences. Studied target words were preceded by either a cue word studied in the same sentence (thus congruent with encoding context) or a cue word studied in a different sentence (thus incongruent with encoding context). Converging fMRI results from independently defined ROIs and whole-brain analysis showed regional specificity in the recollection network. Activity in hippocampus and parahippocampal cortex was specifically increased during successful retrieval following congruent context cues, whereas parietal and prefrontal components of the general recollection network were associated with confident retrieval irrespective of contextual congruency. Our findings implicate medial temporal regions in the retrieval of semantic context, contributing to, but dissociable from, recollective experience

Anticipatory Semantic Processes

Why anticipatory processes correspond to cognitive abilities of living systems? To be adapted to an environment, behaviors need at least i) internal representations of events occurring in the external environment; and ii) internal anticipations of possible events to occur in the external environment. Interactions of these two opposite but complementary cognitive properties lead to various patterns of experimental data on semantic processing. How to investigate dynamic semantic processes? Experimental studies in cognitive psychology offer several interests such as: i) the control of the semantic environment such as words embedded in sentences; ii) the methodological tools allowing the observation of anticipations and adapted oculomotor behavior during reading; and iii) the analyze of different anticipatory processes within the theoretical framework of semantic processing. What are the different types of semantic anticipations? Experimental data show that semantic anticipatory processes involve i) the coding in memory of sequences of words occurring in textual environments; ii) the anticipation of possible future words from currently perceived words; and iii) the selection of anticipated words as a function of the sequences of perceived words, achieved by anticipatory activations and inhibitory selection processes. How to modelize anticipatory semantic processes? Localist or distributed neural networks models can account for some types of semantic processes, anticipatory or not. Attractor neural networks coding temporal sequences are presented as good candidate for modeling anticipatory semantic processes, according to specific properties of the human brain such as i) auto-associative memory; ii) learning and memorization of sequences of patterns; and iii) anticipation of memorized patterns from previously perceived patterns

The Role of Consciousness in Memory

Conscious events interact with memory systems in learning, rehearsal and retrieval (Ebbinghaus 1885/1964; Tulving 1985). Here we present hypotheses that arise from the IDA computional model (Franklin, Kelemen and McCauley 1998; Franklin 2001b) of global workspace theory (Baars 1988, 2002). Our primary tool for this exploration is a flexible cognitive cycle employed by the IDA computational model and hypothesized to be a basic element of human cognitive processing. Since cognitive cycles are hypothesized to occur five to ten times a second and include interaction between conscious contents and several of the memory systems, they provide the means for an exceptionally fine-grained analysis of various cognitive tasks. We apply this tool to the small effect size of subliminal learning compared to supraliminal learning, to process dissociation, to implicit learning, to recognition vs. recall, and to the availability heuristic in recall. The IDA model elucidates the role of consciousness in the updating of perceptual memory, transient episodic memory, and procedural memory. In most cases, memory is hypothesized to interact with conscious events for its normal functioning. The methodology of the paper is unusual in that the hypotheses and explanations presented are derived from an empirically based, but broad and qualitative computational model of human cognition

Brief targeted memory reactivation during the awake state enhances memory stability and benefits the weakest memories.

Reactivation of representations corresponding to recent experience is thought to be a critical mechanism supporting long-term memory stabilization. Targeted memory reactivation, or the re-exposure of recently learned cues, seeks to induce reactivation and has been shown to benefit later memory when it takes place during sleep. However, despite recent evidence for endogenous reactivation during post-encoding awake periods, less work has addressed whether awake targeted memory reactivation modulates memory. Here, we found that brief (50 ms) visual stimulus re-exposure during a repetitive foil task enhanced the stability of cued versus uncued associations in memory. The extent of external or task-oriented attention prior to re-exposure was inversely related to cueing benefits, suggesting that an internally-orientated state may be most permissible to reactivation. Critically, cueing-related memory benefits were greatest in participants without explicit recognition of cued items and remained reliable when only considering associations not recognized as cued, suggesting that explicit cue-triggered retrieval processes did not drive cueing benefits. Cueing benefits were strongest for associations and participants with the poorest initial learning. These findings expand our knowledge of the conditions under which targeted memory reactivation can benefit memory, and in doing so, support the notion that reactivation during awake time periods improves memory stabilization