Final report for the endowment of simulator agents with human-like episodic memory LDRD.

This report documents work undertaken to endow the cognitive framework currently under development at Sandia National Laboratories with a human-like memory for specific life episodes. Capabilities have been demonstrated within the context of three separate problem areas. The first year of the project developed a capability whereby simulated robots were able to utilize a record of shared experience to perform surveillance of a building to detect a source of smoke. The second year focused on simulations of social interactions providing a queriable record of interactions such that a time series of events could be constructed and reconstructed. The third year addressed tools to promote desktop productivity, creating a capability to query episodic logs in real time allowing the model of a user to build on itself based on observations of the user's behavior

The Abstract Language: Symbolic Cogniton And Its Relationship To Embodiment

Embodied theories presume that concepts are modality specific while symbolic theories suggest that all modalities for a given concept are integrated. Symbolic and embodied theories do fairly well with explaining and describing concrete concepts. Specifically, embodied theories seem well suited to describing the actual content of a concept while symbolic theories provide insight into how concepts operate. Conversely, neither symbolic nor embodied theories have been fully sufficient when attempting to describe and explain abstract concepts. Several pluralistic accounts have been put forth to describe how the semantic/lexical system interacts with the conceptual system. In this respect, they attempt to “embody” abstract concepts to the same extent as concrete concepts. Nevertheless, a concise and comprehensive theory for explaining how we learn/understand abstract concepts to the extent that we learn/understand concrete concepts remains elusive. One goal of the present review paper is to consider if abstract concepts can be defined by a unified theory or if subsets of abstract concepts will be defined by separate theories. Of particular focus will be Symbolic Interdependency Theory (SIT). It will be argued that SIT is suitable for grounding abstract concepts, as this theory infers that symbols bootstrap meaning from other symbols, highlighting the importance of abstract-to-abstract mapping in the same way that concrete-to-abstract mappings are created. Research will be considered to help outline a cohesive strategy for describing and understanding abstract concepts. Finally, as research has demonstrated efficiencies with concrete concept processing, analogous efficiencies will be explored for developing an understanding of abstract concepts. Such efforts could have both theoretical and practical implications for bolstering our knowledge of concept learning

Mathematical models of cognitive processes

The research activity carried out during the PhD course was focused on the development of mathematical models of some cognitive processes and their validation by means of data present in literature, with a double aim: i) to achieve a better interpretation and explanation of the great amount of data obtained on these processes from different methodologies (electrophysiological recordings on animals, neuropsychological, psychophysical and neuroimaging studies in humans), ii) to exploit model predictions and results to guide future research and experiments. In particular, the research activity has been focused on two different projects: 1) the first one concerns the development of neural oscillators networks, in order to investigate the mechanisms of synchronization of the neural oscillatory activity during cognitive processes, such as object recognition, memory, language, attention; 2) the second one concerns the mathematical modelling of multisensory integration processes (e.g. visual-acoustic), which occur in several cortical and subcortical regions (in particular in a subcortical structure named Superior Colliculus (SC)), and which are fundamental for orienting motor and attentive responses to external world stimuli. This activity has been realized in collaboration with the Center for Studies and Researches in Cognitive Neuroscience of the University of Bologna (in Cesena) and the Department of Neurobiology and Anatomy of the Wake Forest University School of Medicine (NC, USA). PART 1. Objects representation in a number of cognitive functions, like perception and recognition, foresees distribute processes in different cortical areas. One of the main neurophysiological question concerns how the correlation between these disparate areas is realized, in order to succeed in grouping together the characteristics of the same object (binding problem) and in maintaining segregated the properties belonging to different objects simultaneously present (segmentation problem). Different theories have been proposed to address these questions (Barlow, 1972). One of the most influential theory is the so called “assembly coding”, postulated by Singer (2003), according to which 1) an object is well described by a few fundamental properties, processing in different and distributed cortical areas; 2) the recognition of the object would be realized by means of the simultaneously activation of the cortical areas representing its different features; 3) groups of properties belonging to different objects would be kept separated in the time domain. In Chapter 1.1 and in Chapter 1.2 we present two neural network models for object recognition, based on the “assembly coding” hypothesis. These models are networks of Wilson-Cowan oscillators which exploit: i) two high-level “Gestalt Rules” (the similarity and previous knowledge rules), to realize the functional link between elements of different cortical areas representing properties of the same object (binding problem); 2) the synchronization of the neural oscillatory activity in the γ-band (30-100Hz), to segregate in time the representations of different objects simultaneously present (segmentation problem). These models are able to recognize and reconstruct multiple simultaneous external objects, even in difficult case (some wrong or lacking features, shared features, superimposed noise). In Chapter 1.3 the previous models are extended to realize a semantic memory, in which sensory-motor representations of objects are linked with words. To this aim, the network, previously developed, devoted to the representation of objects as a collection of sensory-motor features, is reciprocally linked with a second network devoted to the representation of words (lexical network) Synapses linking the two networks are trained via a time-dependent Hebbian rule, during a training period in which individual objects are presented together with the corresponding words. Simulation results demonstrate that, during the retrieval phase, the network can deal with the simultaneous presence of objects (from sensory-motor inputs) and words (from linguistic inputs), can correctly associate objects with words and segment objects even in the presence of incomplete information. Moreover, the network can realize some semantic links among words representing objects with some shared features. These results support the idea that semantic memory can be described as an integrated process, whose content is retrieved by the co-activation of different multimodal regions. In perspective, extended versions of this model may be used to test conceptual theories, and to provide a quantitative assessment of existing data (for instance concerning patients with neural deficits). PART 2. The ability of the brain to integrate information from different sensory channels is fundamental to perception of the external world (Stein et al, 1993). It is well documented that a number of extraprimary areas have neurons capable of such a task; one of the best known of these is the superior colliculus (SC). This midbrain structure receives auditory, visual and somatosensory inputs from different subcortical and cortical areas, and is involved in the control of orientation to external events (Wallace et al, 1993). SC neurons respond to each of these sensory inputs separately, but is also capable of integrating them (Stein et al, 1993) so that the response to the combined multisensory stimuli is greater than that to the individual component stimuli (enhancement). This enhancement is proportionately greater if the modality-specific paired stimuli are weaker (the principle of inverse effectiveness). Several studies have shown that the capability of SC neurons to engage in multisensory integration requires inputs from cortex; primarily the anterior ectosylvian sulcus (AES), but also the rostral lateral suprasylvian sulcus (rLS). If these cortical inputs are deactivated the response of SC neurons to cross-modal stimulation is no different from that evoked by the most effective of its individual component stimuli (Jiang et al 2001). This phenomenon can be better understood through mathematical models. The use of mathematical models and neural networks can place the mass of data that has been accumulated about this phenomenon and its underlying circuitry into a coherent theoretical structure. In Chapter 2.1 a simple neural network model of this structure is presented; this model is able to reproduce a large number of SC behaviours like multisensory enhancement, multisensory and unisensory depression, inverse effectiveness. In Chapter 2.2 this model was improved by incorporating more neurophysiological knowledge about the neural circuitry underlying SC multisensory integration, in order to suggest possible physiological mechanisms through which it is effected. This endeavour was realized in collaboration with Professor B.E. Stein and Doctor B. Rowland during the 6 months-period spent at the Department of Neurobiology and Anatomy of the Wake Forest University School of Medicine (NC, USA), within the Marco Polo Project. The model includes four distinct unisensory areas that are devoted to a topological representation of external stimuli. Two of them represent subregions of the AES (i.e., FAES, an auditory area, and AEV, a visual area) and send descending inputs to the ipsilateral SC; the other two represent subcortical areas (one auditory and one visual) projecting ascending inputs to the same SC. Different competitive mechanisms, realized by means of population of interneurons, are used in the model to reproduce the different behaviour of SC neurons in conditions of cortical activation and deactivation. The model, with a single set of parameters, is able to mimic the behaviour of SC multisensory neurons in response to very different stimulus conditions (multisensory enhancement, inverse effectiveness, within- and cross-modal suppression of spatially disparate stimuli), with cortex functional and cortex deactivated, and with a particular type of membrane receptors (NMDA receptors) active or inhibited. All these results agree with the data reported in Jiang et al. (2001) and in Binns and Salt (1996). The model suggests that non-linearities in neural responses and synaptic (excitatory and inhibitory) connections can explain the fundamental aspects of multisensory integration, and provides a biologically plausible hypothesis about the underlying circuitry

Artificial general intelligence: Proceedings of the Second Conference on Artificial General Intelligence, AGI 2009, Arlington, Virginia, USA, March 6-9, 2009

Artificial General Intelligence (AGI) research focuses on the original and ultimate goal of AI – to create broad human-like and transhuman intelligence, by exploring all available paths, including theoretical and experimental computer science, cognitive science, neuroscience, and innovative interdisciplinary methodologies. Due to the difficulty of this task, for the last few decades the majority of AI researchers have focused on what has been called narrow AI – the production of AI systems displaying intelligence regarding specific, highly constrained tasks. In recent years, however, more and more researchers have recognized the necessity – and feasibility – of returning to the original goals of the field. Increasingly, there is a call for a transition back to confronting the more difficult issues of human level intelligence and more broadly artificial general intelligence

L'organisation du système lexico-sémantique dans le cerveau monolingue et bilingue en développement

The present doctoral research explored the developing lexical-semantic system in monolingual and bilingual toddlers. The question of how and when word meanings are first related to each other and become integrated into an interconnected semantic system was investigated. Three studies were conducted with monolingual French learning children which aimed at exploring how words are organized, that is, according to taxonomic relationships (e.g., pig - horse) and to semantic similarity distances between words (e.g., cow - sheep versus cow - deer), and whether cognitive mechanisms, such as automatic activation and controlled processes, underlie priming effects. An additional two studies conducted with children learning two languages simultaneously, aimed at determining, first, whether taxonomically related word meanings, in each of the two languages, are processed in a similar manner. The second goal was to explore whether words presented in one language activate words in another language, and vice versa. In an attempt to answer these questions, lexical-semantic processing was explored by two techniques: eye-tracking and event-related potentials (ERPs) techniques. Both techniques provide high temporal resolution measures of word processing but differ in terms of responses. Eye-movement measurements (Study III) reflect looking preferences in response to spoken words and their time-course, whereas ERPs reflect implicit brain responses and their activity patterns (Study I, II, IV, and V). Study I and II revealed that words are taxonomically organized at 18 and 24-month-olds. Both automatic and controlled processes were shown to be involved in word processing during language development (Study II). Study III revealed that at 24-month-olds, categorical and feature overlap between items underpin the developing lexical-semantic system. That is, lexical-items in each semantic category are organized according to graded similarity distances. Productive vocabulary skills influenced word recognition and were related to underlying cognitive mechanisms. Study IV revealed no differences in terms of semantic processing in the bilinguals¿ two languages, but the ERP distribution across the scalp varied according to the language being processed. Study V showed that words presented in one language activate their semantic representations in the second language and the other way around. The distribution of the ERPs depended, however, on the direction of translation. The results suggest that even early dual language experience yields distinct neural resources underlying lexical-semantic processing in the dominant and non-dominant languages during language acquisition.L'objectif de cette thèse est d'étudier le développement du système lexico-sémantique chez les enfants monolingues et bilingues. La question posée est la suivante : quand et comment les significations des mots commencent à être reliées entre elles et à s'intégrer dans un système sémantique interconnecté. Dans un premier temps, trois études ont été menées chez des enfants monolingues français. L'Etude 1, a pour but d'observer si les mots sont organisés selon des liens taxonomiques (e.g., cochon - cheval). L'Etude 2 explore si l'effet d'amorçage sémantique est sous-tendu par des mécanismes cognitifs, comme les processus d'activation automatique et contrôlé. Puis enfin, l'Etude 3 observe si les mots sont organisés en fonction de leur distance de similarité sémantique (e.g., vache - mouton versus vache - cerf). Dans un deuxième temps, deux études ont été conduites chez des enfants apprenant deux langues simultanément. L'Etude 4 vise à déterminer si les mots sont taxonomiquement liés dans chacune des langues. L'Etude 5 explore si les mots présentés dans une langue activent leurs représentations sémantiques dans l'autre langue et vice versa. Dans le but de répondre à ces questions, le traitement lexico-sémantique a été étudié en utilisant deux techniques : l'eye-tracking et les potentiels évoqués (PEs). Ces deux techniques enregistrent lors de la présentation des mots des réponses comportementales (Etude 3) et neuronales (Etude 1, 2, 4 et 5) de haute résolution temporelle. Les Etudes 1 et 2 montrent que chez les monolingues les mots sont liés taxonomiquement à l'âge de 18 et 24 mois. Durant le développement du langage, les deux processus d'activation automatique et contrôlé sont impliqués dans le traitement des mots (Etude 2). L'Etude 3 montre qu'à 24 mois, les mots sont organisés dans le système lexico-sémantique en développement selon la distance des similarités sémantiques. L'Etude 4 montre que chez les enfants bilingues, le traitement sémantique ne diffère pas selon les deux langues, mais la topographie des PEs varie selon la langue traitée. L'Etude 5 montre que les mots présentés dans une langue activent leurs représentations sémantiques dans la deuxième langue et vice versa. Toutefois, la topographie des PEs est modulée selon la direction de traduction. Ces résultats suggèrent que l'acquisition de deux langues, bien qu'elle soit très précoce, requière deux ressources neuronales bien distinctes, sous-tendant ainsi le traitement lexico-sémantique des langues dominante et non-dominante

Self-organization in Communicating Groups: the emergence of coordination, shared references and collective intelligence\ud

The present paper will sketch the basic ideas of the complexity paradigm, and then apply them to social systems, and in particular to groups of communicating individuals who together need to agree about how to tackle some problem or how to coordinate their actions. I will elaborate these concepts to provide an integrated foundation for a theory of self-organization, to be understood as a non-linear process of spontaneous coordination between actions. Such coordination will be shown to consist of the following components: alignment, division of labor, workflow and aggregation. I will then review some paradigmatic simulations and experiments that illustrate the alignment of references and communicative conventions between communicating agents. Finally, the paper will summarize the preliminary results of a series of experiments that I devised in order to observe the emergence of collective intelligence within a communicating group, and interpret these observations in terms of alignment, division of labor and workflow

The Impact of Mild Traumatic Brain injury on Neuronal Networks and Neurobehavior

Despite its enormous incidence, mild traumatic brain injury is not well understood. One aspect that needs more definition is how the mechanical energy during injury affects neural circuit function. Recent developments in cellular imaging probes provide an opportunity to assess the dynamic state of neural networks with single-cell resolution. In this dissertation, we developed imaging methods to assess the state of dissociated cortical networks exposed to mild injury. We probed the microarchitecture of an injured cortical circuit subject to two different injury levels, mild stretch (10% peak) and mild/moderate (35%). We found that mild injury produced a transient increase in calcium activity that dissipated within 1 h after injury. Alternatively, mild/moderate mechanical injury produced immediate disruption in network synchrony, loss in excitatory tone, and increased modular topology, suggesting a threshold for repair and degradation. The more significant changes in network behavior at moderate stretch are influenced by NMDA receptor activation and subsequent proteolytic changes in the neuronal populations. With the ability to analyze individual neurons in a circuit before and after injury, we identified several biomarkers that confer increased risk or protection from mechanical injury. We found that pre-injury connectivity and NMDA receptor subtype composition (NR2A and NR2B content) are important predictors of node loss and remodeling. Mechanistically, stretch injury caused a reduction in voltage-dependent Mg2+ block of the NR2B-cotaning NMDA receptors, resulting in increased uncorrelated activity both at the single channel and network level. The reduced coincidence detection of the NMDA receptor and overactivation of these receptors further impaired network function and plasticity. Given the demonstrated link between NR2B-NMDARs and mitochondrial dysfunction, we discovered that neuronal de-integration from the network is mediated through mitochondrial signaling. Finally, we bridged these network level studies with an investigation of changes in neurobehavior following blast-induced traumatic brain injury (bTBI), a form of mild TBI. We first developed and validated an open-source toolbox for automating the scoring of several common behavior tasks to study the deficits that occur following bTBI. We then specifically evaluated the role of neuronal transcription factor Elk-1 in mediating deficits following blast by exposing Elk-1 knockout mouse to equivalent blast pressure loading. Our systems-level behavior analysis showed that bTBI creates a complex change in behavior, with an increase in anxiety and loss of habituation in object recognition. Moreover, we found these behavioral deficits were eliminated in Elk-1 knockout animals exposed to blast loading. Together, we merged information from different perspectives (in silico, in vitro, and in vivo) and length scales (single channels, single-cells, networks, and animals) to study the impact of mild traumatic brain injury on neuronal networks and neurobehavior

Imagery, affect, and the embodied mind: implications for reading and responding to literature

Since Plato first banished poets from his Republic, the relationship between the aesthetic and moral value of literature has been subject to philosophical, critical, and pedagogical debate. In this philosophical investigation, I sought to explain how the evocation of the senses during literary transactions shapes the phenomenal experience of the reader. Recent developments in neuroscience (Damasio, 1999, 2003; Edelman, 1992) provide strong evidence in support of embodied theories of cognition in which imagery and affect play a central role. The purposes of this philosophical investigation were to describe the structure and function of imagery and affect in the cognitive act of reading, to provide a detailed account of how we exercise our capacity for imaginative thought in order to achieve literal, inferential, and critical comprehension, and to explore the implications of an embodied mind for reading and responding to literary texts. The investigation yielded a critical review of contemporary theories of reading (Kintsch, 1998; Rumelhart, 1977; Sadoski & Paivio, 2001) to examine their ability to explain the phenomena associated with the literary experience. Dual coding theory (Sadoski & Paivio, 2001) which maintains an empirical and embodied view of the mind was shown to have considerable theoretical advantages over rationalist computational theories of cognition in explaining phenomena associated with reading and responding to literary texts. A neurobiological account of consciousness provides support for the idea that literature can engage readers imaginatively in the process moral deliberation (Dewey, 1932/1985). In addition, I concluded that considerable evidence exists to suggest that somatic and visceral changes experienced as a result of undergoing the text can potentially incite individual and social change