How the brain represents language and answers questions? Using an AI system to understand the underlying neurobiological mechanisms

To understand the computations that underlie high-level cognitive processes we propose a framework of mechanisms that could in principle implement START, an AI program that answers questions using natural language. START organizes a sentence into a series of triplets, each containing three elements (subject, verb, object). We propose that the brain similarly defines triplets and then chunks the three elements into a spatial pattern. A complete sentence can be represented using up to 7 triplets in a working memory buffer organized by theta and gamma oscillations. This buffer can transfer information into long-term memory networks where a second chunking operation converts the serial triplets into a single spatial pattern in a network, with each triplet (with corresponding elements) represented in specialized subregions. The triplets that define a sentence become synaptically linked, thereby encoding the sentence in synaptic weights. When a question is posed, there is a search for the closest stored memory (having the greatest number of shared triplets). We have devised a search process that does not require that the question and the stored memory have the same number of triplets or have triplets in the same order. Once the most similar memory is recalled and undergoes 2-level dechunking, the sought for information can be obtained by element-by-element comparison of the key triplet in the question to the corresponding triplet in the retrieved memory. This search may require a reordering to align corresponding triplets, the use of pointers that link different triplets, or the use of semantic memory. Our framework uses 12 network processes; existing models can implement many of these, but in other cases we can only suggest neural implementations. Overall, our scheme provides the first view of how language-based question answering could be implemented by the brain

Synaptic Homeostasis and Restructuring across the Sleep-Wake Cycle

Sleep is critical for hippocampus-dependent memory consolidation. However, the underlying mechanisms of synaptic plasticity are poorly understood. The central controversy is on whether long-term potentiation (LTP) takes a role during sleep and which would be its specific effect on memory. To address this question, we used immunohistochemistry to measure phosphorylation of Ca2+/calmodulin-dependent protein kinase II (pCaMKIIα) in the rat hippocampus immediately after specific sleep-wake states were interrupted. Control animals not exposed to novel objects during waking (WK) showed stable pCaMKIIα levels across the sleep-wake cycle, but animals exposed to novel objects showed a decrease during subsequent slow-wave sleep (SWS) followed by a rebound during rapid-eye-movement sleep (REM). The levels of pCaMKIIα during REM were proportional to cortical spindles near SWS/REM transitions. Based on these results, we modeled sleep-dependent LTP on a network of fully connected excitatory neurons fed with spikes recorded from the rat hippocampus across WK, SWS and REM. Sleep without LTP orderly rescaled synaptic weights to a narrow range of intermediate values. In contrast, LTP triggered near the SWS/REM transition led to marked swaps in synaptic weight ranking. To better understand the interaction between rescaling and restructuring during sleep, we implemented synaptic homeostasis and embossing in a detailed hippocampal-cortical model with both excitatory and inhibitory neurons. Synaptic homeostasis was implemented by weakening potentiation and strengthening depression, while synaptic embossing was simulated by evoking LTP on selected synapses. We observed that synaptic homeostasis facilitates controlled synaptic restructuring. The results imply a mechanism for a cognitive synergy between SWS and REM, and suggest that LTP at the SWS/REM transition critically influences the effect of sleep: Its lack determines synaptic homeostasis, its presence causes synaptic restructuring.: Support obtained from Financiadora de Estudos e Projetos (http://www.finep.gov.br/) Grant # 01.06.1092.00 to SR; Conselho Nacional de Desenvolvimento Científico e Tecnológico (http:// www.cnpq.br/): Grants 481506/2007-1, 481351/2011- 6 and 306604/2012-4 to SR, Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (http://www.capes.gov.br/) and Ciencias sem Fronteiras (http://www.cienciasemfronteiras.gov.br/ web/csf/home) to AT and CRC; Fundação de Amparo à Pesquisa do Rio Grande do Norte (http://wwwfapern.rn.gov.br/): Grant Pronem 003/2011 to SR; Fundação de Amparo à Pesquisa do Estado de São Paulo (http://www.fapesp.br/): Grant #2013/ 07699-0 - Center for Neuromathematics to SR; CMP and VRC supported by post-doctoral fellowships from Fundação de Amparo à Pesquisa do Rio Grande do Norte /CNPq. Additional support obtained from the Federal University of Rio Grande do Norte (www.ufrn. br); Ministry of Science, Technology and Innovation (http://www.mcti.gov.br/); Associação Alberto Santos Dumont de Apoio à Pesquisa (http://natalneuro.com/ associacao/index.asp); Pew Latin American Fellows Program (http://www.pewtrusts.org/en/projects/pewlatin-american-fellows/) to SR; Informatics Department of the Instituto Federal de Educação, Ciência e Tecnologia do Rio Grande do Norte (http:// portal.ifrn.edu.br/) to WB. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscrip

DinosaurVR: Using Virtual Reality to Enhance a Museum Exhibition

Museums featuring dinosaur fossils have always attracted the attention of the crowd. However, sustaining public interest in science becomes more challenging year after year, even for popular attractions, calling for changes in how exhibits appear in the face of new technologies. To enrich the user’s experience and enhance the exhibition’s attractiveness, we developed and evaluated an immersive and interactive Virtual Reality (VR) experience to integrate the Paleontology exhibition at the Câmara Cascudo Museum in Natal, Brazil. Experienced VR users were interviewed and reported, through software tests, that the system was stable and performed as intended without any noticeable issues that could affect the user experience. They also found the content adequate, except for the suggestion to include more information about Paleoichnology and the finding of fossil tracksites. The graphical quality received mixed reviews, with some participants suggesting improvements to the terrain, such as adding vegetation and enhancing lighting. They also noted that the experience would appeal to younger audiences, thanks to the novelty of VR technology and the accessibility offered by the museum. Improvements were made to the environment based on their feedback, including changes to the terrain and lighting. Additionally, we developed two alternative versions of the experience, one for multi-projection in a room and another without interactive elements. Our results indicate that the VR experience can be successfully integrated with the exhibition and has the potential to enhance museum visits. It can connect the audience with the actual fossils on display by using a dinosaur in the exhibition. It allows one to visualize how their region changed in the past million years. Finally, this experiment helped bridge the population from an unprivileged region with the science that is produced upon the fossils found in their context. Such knowledge broadens the public’s imagination and triggers a whole chain of development for the community around the museum. Our VR exhibition prototype showed great potential to amplify this mission

Combining virtual reality and tactile stimulation to investigate embodied finger-based numerical representations

Finger-based representation of numbers is a high-level cognitive strategy to assist numerical and arithmetic processing in children and adults. It is unclear whether this paradigm builds on simple perceptual features or comprises several attributes through embodiment. Here we describe the development and initial testing of an experimental setup to study embodiment during a finger-based numerical task using Virtual Reality (VR) and a low-cost tactile stimulator that is easy to build. Using VR allows us to create new ways to study finger-based numerical representation using a virtual hand that can be manipulated in ways our hand cannot, such as decoupling tactile and visual stimuli. The goal is to present a new methodology that can allow researchers to study embodiment through this new approach, maybe shedding new light on the cognitive strategy behind the finger-based representation of numbers. In this case, a critical methodological requirement is delivering precisely targeted sensory stimuli to specific effectors while simultaneously recording their behavior and engaging the participant in a simulated experience. We tested the device’s capability by stimulating users in different experimental configurations. Results indicate that our device delivers reliable tactile stimulation to all fingers of a participant’s hand without losing motion tracking quality during an ongoing task. This is reflected by an accuracy of over 95% in participants detecting stimulation of a single finger or multiple fingers in sequential stimulation as indicated by experiments with sixteen participants. We discuss possible application scenarios, explain how to apply our methodology to study the embodiment of finger-based numerical representations and other high-level cognitive functions, and discuss potential further developments of the device based on the data obtained in our testing

Nootropic effects of LSD: Behavioral, molecular and computational evidence

The therapeutic use of classical psychedelic substances such as d-lysergic acid diethylamide (LSD) surged in recent years. Studies in rodents suggest that these effects are produced by increased neural plasticity, including stimulation of the mTOR pathway, a key regulator of metabolism, plasticity, and aging. Could psychedelic-induced neural plasticity be harnessed to enhance cognition? Here we show that LSD treatment enhanced performance in a novel object recognition task in rats, and in a visuo-spatial memory task in humans. A proteomic analysis of human brain organoids showed that LSD affected metabolic pathways associated with neural plasticity, including mTOR. To gain insight into the relation of neural plasticity, aging and LSD-induced cognitive gains, we emulated the experiments in rats and humans with a neural network model of a cortico-hippocampal circuit. Using the baseline strength of plasticity as a proxy for age and assuming an increase in plasticity strength related to LSD dose, the simulations provided a good fit for the experimental data. Altogether, the results suggest that LSD has nootropic effects.This project was supported by the Beckley Foundation; Fundação de Amparo à Pesquisa do Estado do Rio de Janeiro, Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) – Finance Code 001, Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) (grants 308775/2015-5 and 408145/2016-1), São Paulo Research Foundation grants (2013/07699-0, 2014/10068-4, 2017/25588-1 and 2019/00098-7), intramural grants from D'Or Institute and Federal University of Rio Grande do Norte, and a Juan de la Cierva-Incorporación Scholarship (IJCI-2016-27864) from the Spanish Ministry of Science, Innovation and Universities, and a Newton International Fellowship from the Royal Society.Peer reviewe

Control of sound synthesis by acoustic and semantic analogy using bio-inspired computing

Orientadores: Fernando José Von Zuben, Jônatas ManzolliDissertação (mestrado) - Universidade Estadual de Campinas, Faculdade de Engenharia Elétrica e de ComputaçãoResumo: Este trabalho sugere novos paradigmas de controle de mecanismos de síntese sonora. Utilizando conceitos das ciências cognitivas, o processo gerativo é modelado como um sistema de conversões entre representações, da atuação subjetiva do usuário, passando pela descritiva e culminando no material sonoro. A partir do estudo da analogia descritiva, engendra-se a analogia acústica, representação por amostras sonoras, e a analogia semântica, representação por linguagem. Aplicadas à arquitetura modelada, essas analogias permitem que o processo de síntese sonora tenha um caráter mais intuitivo. São apresentadas duas implementações práticas, sendo que técnicas de computação bio-inspirada fornecem o maquinário computacional para a realização do mapeamento entre representações e controle do processo de sínteseAbstract: This work suggests novel control paradigms of sound synthesis mechanisms. Applying cognitive science concepts, the generative process is modeled as a system of conversions throughout representations: from user's insight, through descriptive, to the sound material. From descriptive analogy studies, the acoustic analogy (representation through sound) and the semantic analogy (representation through language) are engendered. Applied to the modeled architecture, these analogies allow the synthesis process to have a more intuitive nature. Two practical implementations are presented. Bio-inspired computing provides the computational machinery used to map different representations and to control the synthesis processMestradoMestre em Engenharia Elétric

The Hippocampus code : a computational study of the structure and function of the hippocampus

Actualment, no hi ha consens científic respecte a la informació representada en la activitat de les célules del hipocamp. D'una banda, experiments amb humans sostenen una visión de la funció de l'hipocamp com a un sistema per l'emmagatzematge de memóries episódiques, mentre que la recerca amb rodents enfatitza una visió com a sistema cognitiu espacial. Tot i que existeix abundant evidència experimental que indica una possible sobreposició d'ambdues teories, aquesta dissociació també es manté en part en base a dades fisiològiques aparentment incompatibles. Aquesta tèsi poposa que l'hippocamp té un rol funcional que s'hauría d'analitzar en termes de la seva estructura i funció, enlloc de mitjança estudis correlació entre activitat neuronal i comportament. La identificació d'un codi a l'hipocamp, es a dir, el conjunt de principis computacionals que conformen les transformacions d'entrada i sortida de l'activitat neuronal, hauría de proporcionar un explicació unificada de la seva funció. En aquesta tèsi presentem un model teòric que descriu quantitativament i que interpreta la selectivitat de certes regions de l'hipocamp en funció de variables espaials i no-espaials, tal i com observada en experiments amb rates. Aquest resultat suggereix que multiples aspectes de la memòria expressada en humans i rodents deriven d'uns mateixos principis. Per aquest motius, proposem nous principis per la memòria, l'auto-completat de patrons i plasticitat. A més, mitjançant aplicacions robòtiques, creem d'un nexe causal entre el circuit neural i el comportament amb el que demostrem la naturalesa conjuntiva de la selectivitat neuronal observada en el hipocamp es necessària per la solució de problemes pràctics comuns, com per example la cerca d'aliments. Tot plegat, aquests resultats avancen en l'idea general de que el codi de l'hipocamp es genèric i aplicable als diversos tipus de memòries estudiades en la literatura.There is no consensual understanding on what the activity of the hippocampus neurons represents. While experiments with humans foster a dominant view of an episodic memory system, experiments with rodents promote its role as a spatial cognitive system. Although there is abundant evidence pointing to an overlap between these two theories, the dissociation is sustained by conflicting physiological data. This thesis proposes that the functional role of the hippocampus should be analyzed in terms of its structure and function rather than by the correlation of neuronal activity and behavioral performance. The identification of the hippocampus code, i.e. the set of computational principles underlying the input-output transformations of neural activity, might ultimately provide a unifying understanding of its role. In this thesis we present a theoretical model that quantitatively describes and interprets the selectivity of regions of the hippocampus to spatial and non-spatial variables observed in experiments with rats. The results suggest that the multiple aspects of memory expressed in human and rodent data are derived form similar principles. This approach suggests new principles for memory, pattern completion and plasticity. In addition, by creating a causal tie between the neural circuitry and behavior through a robotic control framework we show that the conjunctive nature of neural selectivity observed in the hippocampus is needed for effective problem solving in real-world tasks such as foraging. Altogether, these results advance the concept that the hippocampal code is generic to the different aspects of memory highlighted in the literature

A wavelet based neural model to optimize and read out a temporal population code

It has been proposed that the dense excitatory local connectivity of the neo-cortex plays a specific role in the transformation of spatial stimulus information into a temporal representation or a temporal population code (TPC). TPC provides for a rapid, robust, and high-capacity encoding of salient stimulus features with respect to position, rotation, and distortion. The TPC hypothesis gives a functional interpretation to a core feature of the cortical anatomy: its dense local and sparse long-range connectivity. Thus far, the question of how the TPC encoding can be decoded in downstream areas has not been addressed. Here, we present a neural circuit that decodes the spectral properties of the TPC using a biologically plausible implementation of a Haar transform. We perform a systematic investigation of our model in a recognition task using a standardized stimulus set. We consider alternative implementations using either regular spiking or bursting neurons and a range of spectral bands. Our results show that our wavelet readout circuit provides for the robust decoding of the TPC and further compresses the code without loosing speed or quality of decoding. We show that in the TPC signal the relevant stimulus information is present in the frequencies around 100 Hz. Our results show that the TPC is constructed around a small number of coding components that can be well decoded by wavelet coefficients in a neuronal implementation. The solution to the TPC decoding problem proposed here suggests that cortical processing streams might well consist of sequential operations where spatio-temporal transformations at lower levels forming a compact stimulus encoding using TPC that are subsequently decoded back to a spatial representation using wavelet transforms. In addition, the results presented here show that different properties of the stimulus might be transmitted to further processing stages using different frequency components that are captured by appropriately tuned wavelet-based decoders.This work was supported by EU FP7 projects EFAA (FP7-ICT-270490) and GOAL-LEADERS (FP7-ICT-97732)

A wavelet based neural model to optimize and read out a temporal population code

It has been proposed that the dense excitatory local connectivity of the neo-cortex plays a specific role in the transformation of spatial stimulus information into a temporal representation or a temporal population code (TPC). TPC provides for a rapid, robust, and high-capacity encoding of salient stimulus features with respect to position, rotation, and distortion. The TPC hypothesis gives a functional interpretation to a core feature of the cortical anatomy: its dense local and sparse long-range connectivity. Thus far, the question of how the TPC encoding can be decoded in downstream areas has not been addressed. Here, we present a neural circuit that decodes the spectral properties of the TPC using a biologically plausible implementation of a Haar transform. We perform a systematic investigation of our model in a recognition task using a standardized stimulus set. We consider alternative implementations using either regular spiking or bursting neurons and a range of spectral bands. Our results show that our wavelet readout circuit provides for the robust decoding of the TPC and further compresses the code without loosing speed or quality of decoding. We show that in the TPC signal the relevant stimulus information is present in the frequencies around 100 Hz. Our results show that the TPC is constructed around a small number of coding components that can be well decoded by wavelet coefficients in a neuronal implementation. The solution to the TPC decoding problem proposed here suggests that cortical processing streams might well consist of sequential operations where spatio-temporal transformations at lower levels forming a compact stimulus encoding using TPC that are subsequently decoded back to a spatial representation using wavelet transforms. In addition, the results presented here show that different properties of the stimulus might be transmitted to further processing stages using different frequency components that are captured by appropriately tuned wavelet-based decoders.This work was supported by EU FP7 projects EFAA (FP7-ICT-270490) and GOAL-LEADERS (FP7-ICT-97732)