583 research outputs found
Therapeutic applications of computer models of brain activity for Alzheimer disease.
THERAPEUTIC IMPLICATIONS OF COMPUTER MODELS OF BRAIN ACTIVITY FOR ALZHEIMER DISEASE
Organizing memories for generalization in complementary learning systems
Memorization and generalization are complementary cognitive processes that jointly promote adaptive behavior. For example, animals should memorize safe routes to specific water sources and generalize from these memories to discover environmental features that predict new ones. These functions depend on systems consolidation mechanisms that construct neocortical memory traces from hippocampal precursors, but why systems consolidation only applies to a subset of hippocampal memories is unclear. Here we introduce a new neural network formalization of systems consolidation that reveals an overlooked tensionâunregulated neocortical memory transfer can cause overfitting and harm generalization in an unpredictable world. We resolve this tension by postulating that memories only consolidate when it aids generalization. This framework accounts for partial hippocampalâcortical memory transfer and provides a normative principle for reconceptualizing numerous observations in the field. Generalization-optimized systems consolidation thus provides new insight into how adaptive behavior benefits from complementary learning systems specialized for memorization and generalization
Ăber die Selbstorganisation einer hierarchischen GedĂ€chtnisstruktur fĂŒr kompositionelle ObjektreprĂ€sentation im visuellen Kortex
At present, there is a huge lag between the artificial and the biological information processing systems in terms of their capability to learn. This lag could be certainly reduced by gaining more insight into the higher functions of the brain like learning and memory. For instance, primate visual cortex is thought to provide the long-term memory for the visual objects acquired by experience. The visual cortex handles effortlessly arbitrary complex objects by decomposing them rapidly into constituent components of much lower complexity along hierarchically organized visual pathways. How this processing architecture self-organizes into a memory domain that employs such compositional object representation by learning from experience remains to a large extent a riddle. The study presented here approaches this question by proposing a functional model of a self-organizing hierarchical memory network. The model is based on hypothetical neuronal mechanisms involved in cortical processing and adaptation. The network architecture comprises two consecutive layers of distributed, recurrently interconnected modules. Each module is identified with a localized cortical cluster of fine-scale excitatory subnetworks. A single module performs competitive unsupervised learning on the incoming afferent signals to form a suitable representation of the locally accessible input space. The network employs an operating scheme where ongoing processing is made of discrete successive fragments termed decision cycles, presumably identifiable with the fast gamma rhythms observed in the cortex. The cycles are synchronized across the distributed modules that produce highly sparse activity within each cycle by instantiating a local winner-take-all-like operation. Equipped with adaptive mechanisms of bidirectional synaptic plasticity and homeostatic activity regulation, the network is exposed to natural face images of different persons. The images are presented incrementally one per cycle to the lower network layer as a set of Gabor filter responses extracted from local facial landmarks. The images are presented without any person identity labels. In the course of unsupervised learning, the network creates simultaneously vocabularies of reusable local face appearance elements, captures relations between the elements by linking associatively those parts that encode the same face identity, develops the higher-order identity symbols for the memorized compositions and projects this information back onto the vocabularies in generative manner. This learning corresponds to the simultaneous formation of bottom-up, lateral and top-down synaptic connectivity within and between the network layers. In the mature connectivity state, the network holds thus full compositional description of the experienced faces in form of sparse memory traces that reside in the feed-forward and recurrent connectivity. Due to the generative nature of the established representation, the network is able to recreate the full compositional description of a memorized face in terms of all its constituent parts given only its higher-order identity symbol or a subset of its parts. In the test phase, the network successfully proves its ability to recognize identity and gender of the persons from alternative face views not shown before. An intriguing feature of the emerging memory network is its ability to self-generate activity spontaneously in absence of the external stimuli. In this sleep-like off-line mode, the network shows a self-sustaining replay of the memory content formed during the previous learning. Remarkably, the recognition performance is tremendously boosted after this off-line memory reprocessing. The performance boost is articulated stronger on those face views that deviate more from the original view shown during the learning. This indicates that the off-line memory reprocessing during the sleep-like state specifically improves the generalization capability of the memory network. The positive effect turns out to be surprisingly independent of synapse-specific plasticity, relying completely on the synapse-unspecific, homeostatic activity regulation across the memory network. The developed network demonstrates thus functionality not shown by any previous neuronal modeling approach. It forms and maintains a memory domain for compositional, generative object representation in unsupervised manner through experience with natural visual images, using both on- ("wake") and off-line ("sleep") learning regimes. This functionality offers a promising departure point for further studies, aiming for deeper insight into the learning mechanisms employed by the brain and their consequent implementation in the artificial adaptive systems for solving complex tasks not tractable so far.GegenwĂ€rtig besteht immer noch ein enormer Abstand zwischen der LernfĂ€higkeit von kĂŒnstlichen und biologischen Informationsverarbeitungssystemen. Dieser Abstand lieĂe sich durch eine bessere Einsicht in die höheren Funktionen des Gehirns wie Lernen und GedĂ€chtnis verringern. Im visuellen Kortex etwa werden die Objekte innerhalb kĂŒrzester Zeit entlang der hierarchischen Verarbeitungspfade in ihre Bestandteile zerlegt und so durch eine Komposition von Elementen niedrigerer KomplexitĂ€t dargestellt. Bereits bekannte Objekte werden so aus dem LangzeitgedĂ€chtnis abgerufen und wiedererkannt. Wie eine derartige kompositionell-hierarchische GedĂ€chtnisstruktur durch die visuelle Erfahrung zustande kommen kann, ist noch weitgehend ungeklĂ€rt. Um dieser Frage nachzugehen, wird hier ein funktionelles Modell eines lernfĂ€higen rekurrenten neuronalen Netzwerkes vorgestellt. Im Netzwerk werden neuronale Mechanismen implementiert, die der kortikalen Verarbeitung und PlastizitĂ€t zugrunde liegen. Die hierarchische Architektur des Netzwerkes besteht aus zwei nacheinander geschalteten Schichten, die jede eine Anzahl von verteilten, rekurrent vernetzten Modulen beherbergen. Ein Modul umfasst dabei mehrere funktionell separate Subnetzwerke. Jedes solches Modul ist imstande, aus den eintreffenden Signalen eine geeignete ReprĂ€sentation fĂŒr den lokalen Eingaberaum unĂŒberwacht zu lernen. Die fortlaufende Verarbeitung im Netzwerk setzt sich zusammen aus diskreten Fragmenten, genannt Entscheidungszyklen, die man mit den schnellen kortikalen Rhythmen im gamma-Frequenzbereich in Verbindung setzen kann. Die Zyklen sind synchronisiert zwischen den verteilten Modulen. Innerhalb eines Zyklus wird eine lokal umgrenzte winner-take-all-Ă€hnliche Operation in Modulen durchgefĂŒhrt. Die KompetitionsstĂ€rke wĂ€chst im Laufe des Zyklus an. Diese Operation aktiviert in AbhĂ€ngigkeit von den Eingabesignalen eine sehr kleine Anzahl von Einheiten und verstĂ€rkt sie auf Kosten der anderen, um den dargebotenen Reiz in der NetzwerkaktivitĂ€t abzubilden. Ausgestattet mit adaptiven Mechanismen der bidirektionalen synaptischen PlastizitĂ€t und der homöostatischen AktivitĂ€tsregulierung, erhĂ€lt das Netzwerk natĂŒrliche Gesichtsbilder von verschiedenen Personen dargeboten. Die Bilder werden der unteren Netzwerkschicht, je ein Bild pro Zyklus, als Ansammlung von Gaborfilterantworten aus lokalen Gesichtslandmarken zugefĂŒhrt, ohne Information ĂŒber die PersonenidentitĂ€t zur VerfĂŒgung zu stellen. Im Laufe der unĂŒberwachten Lernprozedur formt das Netzwerk die Verbindungsstruktur derart, dass die Gesichter aller dargebotenen Personen im Netzwerk in Form von dĂŒnn besiedelten GedĂ€chtnisspuren abgelegt werden. Hierzu werden gleichzeitig vorwĂ€rtsgerichtete (bottom-up) und rekurrente (lateral, top-down) synaptische Verbindungen innerhalb und zwischen den Schichten gelernt. Im reifen Verbindungszustand werden infolge dieses Lernens die einzelnen Gesichter als Komposition ihrer Bestandteile auf generative Art gespeichert. Dank der generativen Art der gelernten Struktur reichen schon allein das höhere IdentitĂ€tssymbol oder eine kleine Teilmenge von zugehörigen Gesichtselementen, um alle Bestandteile der gespeicherten Gesichter aus dem GedĂ€chtnis abzurufen. In der Testphase kann das Netzwerk erfolgreich sowohl die IdentitĂ€t als auch das Geschlecht von Personen aus vorher nicht gezeigten Gesichtsansichten erkennen. Eine bemerkenswerte Eigenschaft der entstandenen GedĂ€chtnisarchitektur ist ihre FĂ€higkeit, ohne Darbietung von externen Stimuli spontan AktivitĂ€tsmuster zu generieren und die im GedĂ€chtnis abgelegten Inhalte in diesem schlafĂ€hnlichen "off-line" Regime wiederzugeben. Interessanterweise ergibt sich aus der Schlafphase ein direkter Vorteil fĂŒr die GedĂ€chtnisfunktion. Dieser Vorteil macht sich durch eine drastisch verbesserte Erkennungsrate nach der Schlafphase bemerkbar, wenn das Netwerk mit den zuvor nicht dargebotenen Ansichten von den bereits bekannten Personen konfrontiert wird. Die Leistungsverbesserung nach der Schlafphase ist umso deutlicher, je stĂ€rker die Alternativansichten vom Original abweichen. Dieser positive Effekt ist zudem komplett unabhĂ€ngig von der synapsenspezifischen PlastizitĂ€t und kann allein durch die synapsenunspezifische, homöostatische Regulation der AktivitĂ€t im Netzwerk erklĂ€rt werden. Das entwickelte Netzwerk demonstriert so eine im Bereich der neuronalen Modellierung bisher nicht gezeigte FunktionalitĂ€t. Es kann unĂŒberwacht eine GedĂ€chtnisdomĂ€ne fĂŒr kompositionelle, generative ObjektreprĂ€sentation durch die Erfahrung mit natĂŒrlichen Bildern sowohl im reizgetriebenen, wachĂ€hnlichen Zustand als auch im reizabgekoppelten, schlafĂ€hnlichen Zustand formen und verwalten. Diese FunktionalitĂ€t bietet einen vielversprechenden Ausgangspunkt fĂŒr weitere Studien, die die neuronalen Lernmechanismen des Gehirns ins Visier nehmen und letztendlich deren konsequente Umsetzung in technischen, adaptiven Systemen anstreben
Recommended from our members
Memory in autism spectrum disorder: a meta-analysis of experimental studies
To address inconsistencies in the literature on memory in Autism Spectrum Disorder (ASD), we report the first ever meta-analysis of short-term (STM) and episodic long-term (LTM) memory in ASD, evaluating the effects of type of material, type of retrieval and the role of inter-item relations. Analysis of 64 studies comparing individuals with ASD and typical development (TD) showed greater difficulties in ASD compared to TD individuals in STM (Hedgesâ g=-0.53 [95%CI -0.90; -0.16], p=.005, IÂČ=96%) compared to LTM (g=-0.30 [95%CI -0.42; -0.17], p<.00001, IÂČ=24%), a small difficulty in verbal LTM (g=-0.21, p=.01), contrasting with a medium difficulty for visual LTM (g= -0.41, p=.0002) in ASD compared to TD individuals. We also found a general diminution in free recall compared to cued recall and recognition (LTM, free recall: g=-0.38, p<.00001, cued recall: g=-0.08, p=.58, recognition: g=-0.15, p=.16; STM, free recall: g=-0.59, p=.004, recognition: g=-0.33, p=.07). We discuss these results in terms of their relation to semantic memory. The limited diminution in verbal LTM and preserved overall recognition and cued recall (supported retrieval) may result from a greater overlap of these tasks with semantic long-term representations which are overall preserved in ASD. By contrast, difficulties in STM or free recall may result from less overlap with the semantic system or may involve additional cognitive operations and executive demands. These findings highlight the need to support STM functioning in ASD and acknowledge the potential benefit of using verbal materials at encoding and broader forms of memory support at retrieval to enhance performance
At the interface: Dynamic interactions of explicit and implicit language knowledge.
Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/139748/1/AttheInterface.pd
A Guide to Selected Resources on Memorization Techniques for Pianists: An Annotated Bibliography
This research project provides reviews of resources for pianists, performers, teachers and students, striving to deepen their knowledge of memorization techniques and explore different methods of memorizing piano music, and for scholars pursuing further research on this important topic. Readers will find resources that discuss the topic from the musical and psychological standpoints, but not a scientific approach to brain and memorization. Resources include books, journal articles, and theses/dissertations that are written in English from the 1890s through the first decade of the 21st century
Effective Augmentation of Creativity-Involving Productivity Consequent to Spontaneous Selectivity in Knowledge Acquisition
The results of many studies have suggested that we actively select information from the environment. However, the functional consequences of such selectivity in knowledge acquisition remain unclear, even though it is a vital factor in determining the characteristics of our future knowledge and cognition. We hypothesized that spontaneous selectivity in knowledge acquisition results in effective augmentation of productivity, especially in creativity-demanding task. To test this, we conducted experiments in which subjects acquired novel compositional words during their rapid presentation, evaluated memory confidence rates for the acquired words, and then produced essays based on these words. First, in experiment 1, we showed that the level of confidence in the recognition memory for the words positively related with the length of the essays (a measure of creativity-involving productivity in quantity). Additionally, we found that the semantic distance from the essay to the components of the compositional word (a measure of creative-productivity in quality) was farther for the word with higher memory confidence than for the word with lower memory confidence, suggesting creative leaps when writing the former. While this result supported our hypothesis, it might also reflect better memory that was independent of spontaneous selection. Thus, in a different subject group, we conducted a similar experiment (experiment 2) in which two of the 20 compositional words were presented more often (five times per block) to force memorization. Again, consistent with our hypothesis, essays based on spontaneously memorized words (presented once per block) were significantly longer than those produced using the forcedly memorized words. Therefore, better memory per se did not explain the higher productivity. Instead, these results suggested that the higher creativity-involving productivity was consequent to spontaneous selectivity in the knowledge acquisition. Additionally, we propose a possible mechanism for the observed results based on the results of a neural network simulation. In this simulation, we found that novel information that was assigned to locations more easily accessible to the entire network was better assimilated and therefore selectively acquired. Based on this simulation, we moderately suggest that spontaneously acquired knowledge effectively confers productivity because it effectively activates large parts of the neural networks
Recommended from our members
Synaptic plasticity and memory addressing in biological and artificial neural networks
Biological brains are composed of neurons, interconnected by synapses to create large complex networks. Learning and memory occur, in large part, due to synaptic plasticity -- modifications in the efficacy of information transmission through these synaptic connections. Artificial neural networks model these with neural "units" which communicate through synaptic weights. Models of learning and memory propose synaptic plasticity rules that describe and predict the weight modifications. An equally important but under-evaluated question is the selection of \textit{which} synapses should be updated in response to a memory event. In this work, we attempt to separate the questions of synaptic plasticity from that of memory addressing.
Chapter 1 provides an overview of the problem of memory addressing and a summary of the solutions that have been considered in computational neuroscience and artificial intelligence, as well as those that may exist in biology. Chapter 2 presents in detail a solution to memory addressing and synaptic plasticity in the context of familiarity detection, suggesting strong feedforward weights and anti-Hebbian plasticity as the respective mechanisms. Chapter 3 proposes a model of recall, with storage performed by addressing through local third factors and neo-Hebbian plasticity, and retrieval by content-based addressing. In Chapter 4, we consider the problem of concurrent memory consolidation and memorization. Both storage and retrieval are performed by content-based addressing, but the plasticity rule itself is implemented by gradient descent, modulated according to whether an item should be stored in a distributed manner or memorized verbatim. However, the classical method for computing gradients in recurrent neural networks, backpropagation through time, is generally considered unbiological. In Chapter 5 we suggest a more realistic implementation through an approximation of recurrent backpropagation.
Taken together, these results propose a number of potential mechanisms for memory storage and retrieval, each of which separates the mechanism of synaptic updating -- plasticity -- from that of synapse selection -- addressing. Explicit studies of memory addressing may find applications not only in artificial intelligence but also in biology. In artificial networks, for example, selectively updating memories in large language models can help improve user privacy and security. In biological ones, understanding memory addressing can help with health outcomes and treating memory-based illnesses such as Alzheimers or PTSD
Experience-driven formation of parts-based representations in a model of layered visual memory
Growing neuropsychological and neurophysiological evidence suggests that the
visual cortex uses parts-based representations to encode, store and retrieve
relevant objects. In such a scheme, objects are represented as a set of
spatially distributed local features, or parts, arranged in stereotypical
fashion. To encode the local appearance and to represent the relations between
the constituent parts, there has to be an appropriate memory structure formed
by previous experience with visual objects. Here, we propose a model how a
hierarchical memory structure supporting efficient storage and rapid recall of
parts-based representations can be established by an experience-driven process
of self-organization. The process is based on the collaboration of slow
bidirectional synaptic plasticity and homeostatic unit activity regulation,
both running at the top of fast activity dynamics with winner-take-all
character modulated by an oscillatory rhythm. These neural mechanisms lay down
the basis for cooperation and competition between the distributed units and
their synaptic connections. Choosing human face recognition as a test task, we
show that, under the condition of open-ended, unsupervised incremental
learning, the system is able to form memory traces for individual faces in a
parts-based fashion. On a lower memory layer the synaptic structure is
developed to represent local facial features and their interrelations, while
the identities of different persons are captured explicitly on a higher layer.
An additional property of the resulting representations is the sparseness of
both the activity during the recall and the synaptic patterns comprising the
memory traces.Comment: 34 pages, 12 Figures, 1 Table, published in Frontiers in
Computational Neuroscience (Special Issue on Complex Systems Science and
Brain Dynamics),
http://www.frontiersin.org/neuroscience/computationalneuroscience/paper/10.3389/neuro.10/015.2009
- âŠ