The Development of Orthographic Knowledge: A Cognitive Neuroscience Investigation of Reading Skill

This investigation compared the effects of explicit letter-sound training to holistic word training on the development of word recognition in a novel orthography paradigm. In a between-subjects design, participants were trained to read spoken English words printed in the alphabet script of Korean Hangul. Training took place over four separate sessions with assessment measures conducted throughout. Compared to the holistic training, the component training condition resulted in significantly better transfer to novel word forms and retention of previously learned items. Furthermore, compared to component training, holistic training yielded greater sensitivity to frequency. Variability in the holistically trained condition revealed bimodal distribution of performance: a high and low performing subset. Functional MRI measured cortical responses to the training conditions. Imaging results revealed generally greater responses in the "reading network" overall for the explicit component-based training compared to holistic training, in particular, regions of the inferior and superior parietal gyri as well as the left precentral gyrus. In a comparison of readers within the holistic group, we found that readers who implicitly derived the sublexical patterns in the writing system activated more of the reading network than those who did not sufficiently acquire this knowledge. This latter group primarily activated ventral visual regions. We conclude that explicit training of sublexical components leads to optimal word recognition performance in alphabetic writing systems due to the redundant mechanisms of decoding and specific word form knowledge

Cognitive control and language network connectivity associated with language production in aphasia

Aphasia is the breakdown of language comprehension and production due to an acquired brain injury of the left hemisphere. Investigation of the neurological underpinnings of aphasia have advanced from post-mortem investigation of specific regions in the 1800s to the utilization of brain imaging technology to understand brain networks. These approaches have helped us to appreciate the reorganization of the brain and its networks post stroke, particularly as it relates or is modified for adequate versus impaired performance. Research into neuroplastic changes can elucidate differences between healthy and lesioned brains. Furthermore, identification of adaptive (or maladaptive) neuroplastic changes can also inform diagnostics or aid in monitoring the neuroplastic effects of evidence-based treatment. This study utilized resting state functional MRI to characterize graph theory metrics of language (LN) and cognitive control networks (frontoparietal, FPN) in 21 persons with aphasia (PWA) and 18 healthy controls (HC). This study further investigated the relationship between strength of connectivity and semantic access and errors in PWA during a picture description task. When comparing resting state network connectivity of the LN in PWA vs. HC, many edges (10/14) and node degree hubs (3/3) were common to both groups for the LN, suggesting that an inherent network that remains relatively intact even post-stroke. Analyses yielded similar results for resting state FPN network connectivity with common edges and node degree hubs. When investigating correlations between network edges and language measures, correlations between FPN edges and CIU’s and retracing suggested the importance of right hemisphere and ‘healthy’ edge integrity

The Neural Representation of Concepts in Bilinguals: An Evaluation of Factors Influencing Cross-language Overlap Using fMRI-based Multivariate Pattern Analysis

161 p.The neurocognitive mechanisms that support the generalization of semantic representations across different languages remain to be determined. Current psycholinguistic models propose that semantic representations are likely to overlap across languages, although there is evidence also to the contrary. Neuroimaging studies observed that brain activity patterns associated with the meaning of words may be similar across languages. However, the factors that mediate cross-language generalization of semantic representations are not known. In a series of functional MRI research studies, we investigate how factors including state of visual awareness, depth of word processing and lexico-semantic characteristics of words influence cross-language generalization of semantic representations. Using multivariate pattern analysis, we found that fully conscious and deep processing of high concrete and high frequency words leads to above-chance cross-language generalization in putative areas of the semantic network. These results have ramifications for existing psycholinguistic models and theories of meaning representation.bcbl:basque center on cognition, brain & languag

Deep Neural Networks and Brain Alignment: Brain Encoding and Decoding (Survey)

How does the brain represent different modes of information? Can we design a system that automatically understands what the user is thinking? Such questions can be answered by studying brain recordings like functional magnetic resonance imaging (fMRI). As a first step, the neuroscience community has contributed several large cognitive neuroscience datasets related to passive reading/listening/viewing of concept words, narratives, pictures and movies. Encoding and decoding models using these datasets have also been proposed in the past two decades. These models serve as additional tools for basic research in cognitive science and neuroscience. Encoding models aim at generating fMRI brain representations given a stimulus automatically. They have several practical applications in evaluating and diagnosing neurological conditions and thus also help design therapies for brain damage. Decoding models solve the inverse problem of reconstructing the stimuli given the fMRI. They are useful for designing brain-machine or brain-computer interfaces. Inspired by the effectiveness of deep learning models for natural language processing, computer vision, and speech, recently several neural encoding and decoding models have been proposed. In this survey, we will first discuss popular representations of language, vision and speech stimuli, and present a summary of neuroscience datasets. Further, we will review popular deep learning based encoding and decoding architectures and note their benefits and limitations. Finally, we will conclude with a brief summary and discussion about future trends. Given the large amount of recently published work in the `computational cognitive neuroscience' community, we believe that this survey nicely organizes the plethora of work and presents it as a coherent story.Comment: 16 pages, 10 figure

Extracting Spatiotemporal Word and Semantic Representations from Multiscale Neurophysiological Recordings in Humans

With the recent advent of neuroimaging techniques, the majority of the research studying the neural basis of language processing has focused on the localization of various lexical and semantic functions. Unfortunately, the limited time resolution of functional neuroimaging prevents a detailed analysis of the dynamics involved in word recognition, and the hemodynamic basis of these techniques prevents the study of the underlying neurophysiology. Compounding this problem, current techniques for the analysis of high-dimensional neural data are mainly sensitive to large effects in a small area, preventing a thorough study of the distributed processing involved for representing semantic knowledge. This thesis demonstrates the use of multivariate machine-learning techniques for the study of the neural representation of semantic and speech information in electro/magneto-physiological recordings with high temporal resolution. Support vector machines (SVMs) allow for the decoding of semantic category and word-specific information from non-invasive electroencephalography (EEG) and magnetoenecephalography (MEG) and demonstrate the consistent, but spatially and temporally distributed nature of such information. Moreover, the anteroventral temporal lobe (avTL) may be important for coordinating these distributed representations, as supported by the presence of supramodal category-specific information in intracranial recordings from the avTL as early as 150ms after auditory or visual word presentation. Finally, to study the inputs to this lexico-semantic system, recordings from a high density microelectrode array in anterior superior temporal gyrus (aSTG) are obtained, and the recorded spiking activity demonstrates the presence of single neurons that respond specifically to speech sounds. The successful decoding of word identity from this firing rate information suggests that the aSTG may be involved in the population coding of acousto-phonetic speech information that is likely on the pathway for mapping speech-sounds to meaning in the avTL. The feasibility of extracting semantic and phonological information from multichannel neural recordings using machine learning techniques provides a powerful method for studying language using large datasets and has potential implications for the development of fast and intuitive communication prostheses.Engineering and Applied Science