A compositional neural architecture for language

Hierarchical structure and compositionality imbue human language with unparalleled expressive power and set it apart from other perception–action systems. However, neither formal nor neurobiological models account for how these defining computational properties might arise in a physiological system. I attempt to reconcile hierarchy and compositionality with principles from cell assembly computation in neuroscience; the result is an emerging theory of how the brain could convert distributed perceptual representations into hierarchical structures across multiple timescales while representing interpretable incremental stages of (de) compositional meaning. The model's architecture—a multidimensional coordinate system based on neurophysiological models of sensory processing—proposes that a manifold of neural trajectories encodes sensory, motor, and abstract linguistic states. Gain modulation, including inhibition, tunes the path in the manifold in accordance with behavior and is how latent structure is inferred. As a consequence, predictive information about upcoming sensory input during production and comprehension is available without a separate operation. The proposed processing mechanism is synthesized from current models of neural entrainment to speech, concepts from systems neuroscience and category theory, and a symbolic-connectionist computational model that uses time and rhythm to structure information. I build on evidence from cognitive neuroscience and computational modeling that suggests a formal and mechanistic alignment between structure building and neural oscillations and moves toward unifying basic insights from linguistics and psycholinguistics with the currency of neural computation

Prolegomena to a neurocomputational architecture for human grammatical encoding and decoding

The study develops a neurocomputational architecture for grammatical processing in language production and language comprehension (grammatical encoding and decoding, respectively). It seeks to answer two questions. First, how is online syntactic structure formation of the complexity required by natural-language grammars possible in a fixed, preexisting neural network without the need for online creation of new connections or associations? Second, is it realistic to assume that the seemingly disparate instantiations of syntactic structure formation in grammatical encoding and grammatical decoding can run on the same neural infrastructure? This issue is prompted by accumulating experimental evidence for the hypothesis that the mechanisms for grammatical decoding overlap with those for grammatical encoding to a considerable extent, thus inviting the hypothesis of a single “grammatical coder.” The paper answers both questions by providing the blueprint for a syntactic structure formation mechanism that is entirely based on prewired circuitry (except for referential processing, which relies on the rapid learning capacity of the hippocampal complex), and can subserve decoding as well as encoding tasks. The model builds on the “Unification Space” model of syntactic parsing developed by Vosse & Kempen (2000, 2008, 2009). The design includes a neurocomputational mechanism for the treatment of an important class of grammatical movement phenomena

Prior knowledge contribution to declarative learning. A study in amnesia, aging and Alzheimer's disease

L'étude expérimentale de la mémoire humaine a connu deux moments historiques dans les soixante dernières années. 1957 marque la découverte du rôle du lobe temporal interne bilatéral dans l'apprentissage conscient, déclaratif. 1997 marque la découverte de deux systèmes de mémoire déclarative, épisodique et sémantique. Ces découvertes résultent d'études de cas en neuropsychologie. Cette thèse s'inscrit dans la tradition neuropsychologique: sa genèse doit tout à un patient souffrant d'une forme atypique d'amnésie développementale, le patient KA. Son point de départ est une étude de cas approfondie, avec deux résultats surprenants. Malgré une amnésie sévère, KA dispose de connaissances sémantiques exceptionnelles. Par ailleurs, il montre des capacités préservées d'apprentissage explicite, mais uniquement pour des stimuli concrets, pas abstraits. En conséquence, cette thèse a exploré deux pistes de recherche. Premièrement, nous avons caractérisé les processus préservés d'apprentissage déclaratif et l'anatomie cérébrale chez ce patient. Deuxièmement, nous avons étudié le rôle des connaissances préalables dans l'apprentissage: comment ce que l'on sait influence ce dont nous nous souvenons ? Une première série d'expériences montre chez ce patient une atteinte sévère et sélective de l'ensemble du système hippocampique, alors que les structures sous- hippocampiques (cortex entorhinal, périrhinal et parahippocampique) sont préservées. Malgré une amnésie épisodique sévère, nous montrons des connaissances sémantiques supranormales et des aptitudes d'apprentissage explicite rapide. Ces aptitudes sont toutefois restreintes aux stimuli associés à des connaissances préalables. Une seconde série d'expériences explore l'hypothèse selon laquelle les connaissances préalables facilitent l'apprentissage en mémoire déclarative, même dans les situations où le lobe temporal interne est fragilisé, comme dans le vieillissement, ou lésé, comme chez le patient KA ou dans la maladie d'Alzheimer. Nos résultats suggèrent l'existence de processus d'apprentissage rapide en mémoire déclarative, indépendants du système hippocampique et sensibles à la présence de représentations préexistantes. Ces processus semblent affectés par la maladie d'Alzheimer, et ce en lien avec un défaut d'activité des régions sous-hippocampiques antérieures. A l'inverse, les sujets âgés sains peuvent utiliser les connaissances préalables et pourraient ainsi compenser le déclin de la mémoire associative. Ce travail s'accorde avec les modèles postulant une dissociation fonctionnelle au sein du lobe temporal interne pour l'apprentissage déclaratif. Il soutient les propositions neurocognitives et computationnelles récentes, suggérant une voie d'apprentissage néocortical rapide mobilisable dans certaines circonstances. Il met en exergue la dynamique des apprentissages en mémoire déclarative et notamment l'intrication fondamentale entre "savoir" et "se souvenir". Ce que je sais a un impact profond sur ce dont je vais me souvenir. Cette thèse permet d'envisager de nouveaux outils cognitifs pour le diagnostic de la maladie d'Alzheimer. De plus, il semble que des lésions temporales internes auront un impact distinct sur l'apprentissage selon le statut des informations à mémoriser en mémoire à long terme, offrant un regard nouveau sur les effets stimulus-dépendants dans l'amnésie. Une considération approfondie des connaissances préalables associées au contenu de nos expériences, et leur caractérisation détaillée, est requise pour affiner les modèles de la mémoire déclarative. Ces résultats apportent de nouvelles pistes de recherche quant aux circonstances épargnant l'apprentissage, notamment associatif, lors du vieillissement. Plus généralement, ils contribuent à la compréhension des déterminants d'un apprentissage réussi, en mettant l'accent sur les recouvrements entre processus de récupération et d'acquisition. Des applications potentielles en découlent dans le domaine éducatif.The experimental study of human memory has had two historic moments in the last sixty years. 1957 marks the discovery of the role of the medial temporal lobes in conscious learning. 1997 marks the discovery of two systems of declarative memory, namely episodic and semantic memories. These major breakthroughs are owed to clinical case studies in neuropsychology. This thesis follows on from the neuropsychological tradition: its genesis owes everything to a patient suffering from an atypical form of developmental amnesia, the patient KA. The starting point of this work was a thorough neuropsychological study of this patient. Two striking findings shortly arose. First, despite lifelong amnesia, KA had acquired exceptional levels of knowledge about the world. Second, remaining explicit learning abilities were restricted to meaningful, not meaningless, memoranda. As a consequence, we have investigated two research pathways in that thesis. First, we aimed at better characterizing preserved learning abilities and brain structure of the patient KA. Second, our goal was to explore how prior knowledge affects new declarative learning or, put simply, how do we learn what we know? In a first series of behavioural and neuroimaging experiments, we have shown in this patient a severe and selective damage of the whole extended hippocampal system, but preserved subhippocampal structures (entorhinal, perirhinal and parahippocampal cortex). The patient suffers from severe episodic amnesia, but we bring striking evidence for supranormal semantic knowledge as well as normal explicit learning skills. These skills were, however, restricted to familiar stimuli, that is, stimuli carrying pre-experimental knowledge. In a second series of behavioural and neuroimaging experiments, we explored the hypothesis that prior knowledge can facilitate new learning in declarative memory, even in aging or in situations where structures of the medial temporal lobe are or injured, as in amnesia or Alzheimer's disease. Our results suggest the existence of processes allowing fast learning in declarative memory, independently of the hippocampal system, and that are sensitive to the presence of pre-existing representations in long-term memory. Such learning processes appear to be selectively affected by Alzheimer's disease at the pre-dementia stage, in relation to a lack of activation of subhippocampal regions. In contrast, healthy elderly were able to rely on these learning processes to compensate for the decline in associative memory associated with aging. This work lends support to the models postulating a functional dissociation with respect to learning in declarative memory. It indeed strengthens recent neurocognitive and computational accounts that suggest a rapid neocortical learning path under certain circumstances. It highlights the dynamics of learning in declarative memory and in particular the fundamental entanglement between "knowing" and "remembering". What I know profoundly impacts what I will remember. The present thesis points towards new cognitive tools for the diagnosis of Alzheimer's disease. It further brings evidence that medial temporal lesions differentially impact learning depending on the status of the memoranda in long-term memory, which sheds a new light on material-specific effects in amnesia. Our work speaks for a thorough consideration of whether the contents of events have prior representations within long-term memory, and to further better characterize their nature if we are to better understand learning mechanisms. It also brings additional clues for a deeper understanding of how learning and memory can be preserved in aging. More generally, it contributes to a better understanding of the factors determining successful learning, with a focus on how retrieval and acquisition processes overlap during learning. Such findings have potential applications in the educational field

From sequences to cognitive structures : neurocomputational mechanisms

Ph. D. Thesis.Understanding how the brain forms representations of structured information distributed in time is a challenging neuroscientific endeavour, necessitating computationally and neurobiologically informed study. Human neuroimaging evidence demonstrates engagement of a fronto-temporal network, including ventrolateral prefrontal cortex (vlPFC), during language comprehension. Corresponding regions are engaged when processing dependencies between word-like items in Artificial Grammar (AG) paradigms. However, the neurocomputations supporting dependency processing and sequential structure-building are poorly understood. This work aimed to clarify these processes in humans, integrating behavioural, electrophysiological and computational evidence. I devised a novel auditory AG task to assess simultaneous learning of dependencies between adjacent and non-adjacent items, incorporating learning aids including prosody, feedback, delineated sequence boundaries, staged pre-exposure, and variable intervening items. Behavioural data obtained in 50 healthy adults revealed strongly bimodal performance despite these cues. Notably, however, reaction times revealed sensitivity to the grammar even in low performers. Behavioural and intracranial electrode data was subsequently obtained in 12 neurosurgical patients performing this task. Despite chance behavioural performance, time- and time-frequency domain electrophysiological analysis revealed selective responsiveness to sequence grammaticality in regions including vlPFC. I developed a novel neurocomputational model (VS-BIND: “Vector-symbolic Sequencing of Binding INstantiating Dependencies”), triangulating evidence to clarify putative mechanisms in the fronto-temporal language network. I then undertook multivariate analyses on the AG task neural data, revealing responses compatible with the presence of ordinal codes in vlPFC, consistent with VS-BIND. I also developed a novel method of causal analysis on multivariate patterns, representational Granger causality, capable of detecting flow of distinct representations within the brain. This alluded to top-down transmission of syntactic predictions during the AG task, from vlPFC to auditory cortex, largely in the opposite direction to stimulus encodings, consistent with predictive coding accounts. It finally suggested roles for the temporoparietal junction and frontal operculum during grammaticality processing, congruent with prior literature. This work provides novel insights into the neurocomputational basis of cognitive structure-building, generating hypotheses for future study, and potentially contributing to AI and translational efforts.Wellcome Trust, European Research Counci

Enhancing cognitive and functional assessment for Alzheimer’s disease

One major challenge recently recognised by a EU consensus conference is the need of brief, reliable, simple methods to assess Alzheimer’s Disease (AD). There is an emphasis on the need for cognitive, behavioural and functional measures that are sensitive and specific for detecting the cognitive impairment earlier of the course of the disease. In my thesis I explore what are these measures. This thesis sought to explore two themes: the first, where I enhance the existing knowledge about Visual Short-Term Memory Binding as a sensitive, specific and early cognitive marker for AD. The second theme is to look at which everyday functional abilities decline first in the trajectory from the healthy ageing to dementia. I argue that everyday financial decline is the earliest functional impairment in the course of the disease. There are evidences indicating that the Temporal Memory Binding (TMB) reliably detects asymptomatic carriers of the Presenil-1 gene mutation E280A that leads to familial AD and amnestic Mild Cognitive Impairment (aMCI) patients who are at a high risk of conversion to dementia; the test is not affected by healthy ageing and chronic depression; it has been proved culturally unbiased. All these factors make the test a perfect marker for AD. The TMB test has been developed as a computer version. This poses several limitation of using the test globally: it has low mobility, difficulty testing on the older population and patients with AD who have preferences more to conventional paper-and-pencil tests. Therefore, one of the main aims of the thesis was to create a more clinically and user-friendly version of the test. I created a Flash-Card version of the test that contained several modifications from the standard computer version to make the test more clinically oriented: all participants were presented only with two items and the test was presented as a recognition task. The alternative version of the test was presented in the form of the Tablet PC. The first series of experiments (Experiments 1-7) reported in this thesis were dedicated to compare these three formats of testing. I showed that all three methods of testing are equivalent to each other. In these experiments I also have confirmed that the test is unaffected by age in order to serve as a baseline performance on the Flash-Cards and Tablet PC to measure AD performance. In the following Chapter 3, I focused on addressing the question, what is the neurological reason for older adults to perform as well as younger participants. For that I employed a mobile low-density EEG system that has advantages in its mobility and user-friendliness, which is important in clinical setting and in research with frail older participants. The results of the study showed increased activity over all electrodes sites suggesting that older participants recruited more neural resources to achieve levels of performance similar to those observed in younger adults. To show that my Flash-cards TMB task still holds specificity to only AD I recruited patients with aMCI, AD patients and Parkinson’s disease (PD) patients with and without cognitive deterioration (Chapter 4). The results of the study showed that only patients with AD presented impaired performance on the TMB task. On the contrary, compared to either cognitively healthy older individuals or PD patients with normal cognition, patients with PD dementia did not show impairment on the TMB. The other main aim of this thesis was to investigate what functional abilities decline first on the course of the disease and what are those “minimal functional problems”. As part of this thesis I conducted a literature review (Chapter 7) that showed that everyday financial abilities represent the most complex and multidimensional functional activities. I coined a term Acreemagnosia to highlight the specificity of the symptom. In order to assess this specific symptom I developed The Acreemagnosia Measurement (TAM) that is a multi-items measure that inquiries about a person’s awareness of financial abilities and examines actual performance on the broad range of everyday financial tasks. On the groups of healthy middle-aged and older people I validated TAM (Chapter 8). I used a two-parameter IRT model to analyse the psychometric properties of TAM and established the best items that would describe financial abilities of participants in different age and gender groups. The analysis suggests that TAM is measuring most reliably at low to average levels of financial ability, meaning that TAM is potentially a good financial measure for people with limited financial proficiency, which is in keeping with the design and intended use of the instrument with elderly retired people and people with cognitive impairment. The results did not reveal any differential item functioning across different gender and age groups in the scale that indicates that tendency to endorse the item reflects the ability level and are not affected by variables such as gender and age. As TAM is intended for patients with cognitive impairment, I recruited patients with amnestic-MCI and mild-AD (Chapter 9). In this feasibility study I showed that patients with amnestic-MCI are already present with some problems in everyday financial abilities and are unaware of these problems demonstrating that they present with Acreemagnosia, thus confirming the sensitivity of TAM to capture such impairments. The general findings of this thesis indicate that TMB task and TAM can enhance the assessment of dementia and potentially serve in the detection of cognitive impairment at the pre-clinical level

Working memory binding: insights from neuroimaging, behavioural and clinical studies

Working memory binding (WMB) entails the integration of multiple sources of information to form and temporarily store coherent object representations (or conjunctions). To date, cognitive research on binding has mostly focused on visual WM and change detection paradigms (i.e., the WMB task), and documented that WMB is a function sensitive and specific to Alzheimer’s Disease (AD). Following a review of the most relevant studies on the topic in Chapter I, this PhD project aimed at addressing two main pending questions: 1) Whether deficits in WMB tasks reveal abnormalities in individuals at risk of developing dementia, such as those suffering from Mild Cognitive Impairment (MCI); 2) Whether visual WMB deficits observed in AD may generalise for material processed across different modalities (i.e., crossmodal WMB). The first aim was addressed in Chapters II and III. Chapter II reports on the results from an fMRI study showing that MCI patients’ conjunctive WMB abilities are impaired compared to healthy controls, and that such WMB deficits are coupled with lack of activation in key brain areas of the temporo-parietal-occipital network subtending WMB mechanisms for feature conjunctions. Results detailed in Chapter III reveal that MCI patients’ performance on the WMB task is associated with reduced connectivity of structural networks formed by white matter tracts across the whole brain. Importantly, this held true especially for those MCI patients with more severe WMB deficits. The second aim was addressed in Chapter IV, which reports on crossmodal WMB mechanisms found to be impaired in AD, but not in healthy ageing. This was true regardless of the modality though which features were integrated. Chapter V brings together the relevant findings from Chapter II through IV to review current understanding of WMB as a diagnostic tool for AD. Relevant contributions from the above-mentioned studies are discussed and further research questions generated in the light of current findings. This PhD thesis suggests that WMB functions are disrupted in the course of AD, thus, acknowledging that WMB deficits are a hallmark of the disease since the initial stages of its continuum. As such, WMB tasks are recommended as a valid neuropsychological tool to assess patients cross-sectionally or to screen for patients to be included in intervention trials aimed at investigating long-term effects on the disease progression

Visual Search and Task-irrelevant Shape Information in Autism Spectrum Disorder

Visual processing in autism has become a popular topic of research in the past two decades. However, many findings in the reported visual processing abnormalities in autism remain contradictory (Greenaway & Plaisted-Grant, 2013; Simmons et al., 2009). Theories of autism divide into those that favor an explanation of symptoms based on differences in social cognition and orienting on the one hand, and on perception, including vision, on the other. This thesis explores potential differences in visual attentional processing in autism spectrum disorder independently of social cognition factors. The focus lies on visual search as a task by which shifts of attention can be measured and inferences drawn as to underlying perceptual and cognitive processes. It reviews previous work on visual search in autism. The only recently published review on this topic lacked scrutiny (Kaldy et al., 2013). Reviewing the studies with respect to task designs and demographic features of the participants reveals a pattern of results consistent with a previously reported pattern (O’Riordan et al., 2001) but which is distinct from the common notion of “superior search abilities in autism in general”. While children with autism are faster in conjunction and only inefficient feature searches, findings are mixed with regard to adults and feature search and are largely lacking for conjunction search in adults with autism. This thesis also presents an experiment which makes use of the combination of eye-tracking techniques and a visual orientation-feature search task under interference from higher level shape recognition (Zhaoping & Guyader, 2007). Moreover, relevant studies addressing what may be called “task-irrelevant higher-level visual processing” in autism are reviewed. Finally, a more complete characterization of the disorder in terms of its symptomatology with regard to visual attention and visual perception might lead to a better understanding of its cause

Microcircuit remodeling processes underlying learning in the adult

One of the most intriguing discoveries in neuroscience of the past decades has been showing that experience is able to induce structural modifications in cortical microcircuit that might underlie the formation of memories upon learning (for a review, see Caroni, Donato and Muller 2012). Hence, learning induces phases of synapse formation and elimination that are strictly regulated by a variety of mechanisms, which impact on cortical microcircuits affecting both excitatory and inhibitory neurons. Nevertheless, the extent to which specific configurations might be implemented to support specific phases of learning, as well as the impact of experience-induced structural modifications on further learning, is still largely unknown. Here, I explore how the remodeling of identified microcircuits in the mouse hippocampus and neocortex supports learning in the adult. In the first part, I identifiy a microcircuit module engaging VIP and Parvalbumin (PV) positive interneurons to regulate the state of the PV+ network upon experience. This defines states of enhanced or reduced structural plasticity and learning based on the distribution of PV intensity in the network. In the second part, I demonstrate how specific hippocampal subdivisions are exploited to learn subtasks of trial-and-errors forms of learning via the deployment of increasingly precise searching strategies, and sequential recruitment of ventral, intermediate, and dorsal hippocampus. In the third part, I highlight the existence of genetically matched subpopulations of principal cells in the hippocampus, which achieve selective connectivity across hippocampal subdivisions via matched windows of neurogenesis and synaptogenesis during development. In the fourth part, I investigate the maturation of microcircuits mediating feedforward inhibition in the hippocampus, and highlight windows during development for the establishment of the proper baseline configuration in the adult. Moreover, I identify a critical window for cognitive enhancement during hippocampal development. In the fifth part, I study how ageing affects the PV network in hippocampal CA3, providing evidence for which age related neuronal loss correlates to reduced incidental learning performances in old mice. Therefore, by manipulating the PV network early during life, I provide strategies to modulate cognitive decline

A Computational Architecture for Machine Consciousness and Artificial Superintelligence: Updating Working Memory Iteratively

This theoretical article examines how to construct human-like working memory and thought processes within a computer. There should be two working memory stores, one analogous to sustained firing in association cortex, and one analogous to synaptic potentiation in the cerebral cortex. These stores must be constantly updated with new representations that arise from either environmental stimulation or internal processing. They should be updated continuously, and in an iterative fashion, meaning that, in the next state, some items in the set of coactive items should always be retained. Thus, the set of concepts coactive in working memory will evolve gradually and incrementally over time. This makes each state is a revised iteration of the preceding state and causes successive states to overlap and blend with respect to the set of representations they contain. As new representations are added and old ones are subtracted, some remain active for several seconds over the course of these changes. This persistent activity, similar to that used in artificial recurrent neural networks, is used to spread activation energy throughout the global workspace to search for the next associative update. The result is a chain of associatively linked intermediate states that are capable of advancing toward a solution or goal. Iterative updating is conceptualized here as an information processing strategy, a computational and neurophysiological determinant of the stream of thought, and an algorithm for designing and programming artificial intelligence

Visual attention shifting in autism spectrum disorder

Much research to date has been devoted to understanding the neurocognitive abnormalities characteristic of autism spectrum disorders (ASD). Abnormalities in visual attention are particularly notable in ASD and have the potential to inform an understanding of the aberrant neural networks underlying this disorder. The current study utilized a model integrating components of both a two-stage model of perceptual binding and Posner\u27s model of attention in order to provide a coherent account of previous findings of both enhanced and impaired visual attention abilities in ASD. To investigate a potential deficit in attention shifting underlying a variety of observed attentional abnormalities in ASD, the present study employed experimental paradigms requiring attentional shifting at two levels of visual information processing. Aims of the current study were (1) to investigate a general deficit in shifting attention at the level of both preattention and focused attention in ASD as compared to age- and gender-matched NT controls, as measured by both a visual search task with a dimensional shift component and a Navon-type letter task requiring participants to shift attention between global and local levels of a visual stimulus; and (2) to investigate the degree to which deficits in attention shifting as measured by these tasks in ASD as compared to age- and gender-matched NT are related to social functioning. Results were not consistent with a general deficit in attention shifting, but rather showed a qualitatively similar shifting response in ASD and neurotypicals. Preliminary support was found for a relationship between measures of social functioning and attention shifting at the level of both preattention and focused attention. Hypothesized relationships with underlying neural networks and directions for future research are discussed