Single-Trial {MEG} Data Can Be Denoised Through Cross-Subject Predictive Modeling

A pervasive challenge in brain imaging is the presence of noise that hinders investigation of underlying neural processes, with Magnetoencephalography (MEG) in particular having very low Signal-to-Noise Ratio (SNR). The established strategy to increase MEG's SNR involves averaging multiple repetitions of data corresponding to the same stimulus. However, repetition of stimulus can be undesirable, because underlying neural activity has been shown to change across trials, and repeating stimuli limits the breadth of the stimulus space experienced by subjects. In particular, the rising popularity of naturalistic studies with a single viewing of a movie or story necessitates the discovery of new approaches to increase SNR. We introduce a simple framework to reduce noise in single-trial MEG data by leveraging correlations in neural responses across subjects as they experience the same stimulus. We demonstrate its use in a naturalistic reading comprehension task with 8 subjects, with MEG data collected while they read the same story a single time. We find that our procedure results in data with reduced noise and allows for better discovery of neural phenomena. As proof-of-concept, we show that the N400m's correlation with word surprisal, an established finding in literature, is far more clearly observed in the denoised data than the original data. The denoised data also shows higher decoding and encoding accuracy than the original data, indicating that the neural signals associated with reading are either preserved or enhanced after the denoising procedure

Semantic processing with and without awareness. Insights from computational linguistics and semantic priming.

During my PhD, I’ve explored how native speakers access semantic information from lexical stimuli, and weather consciousness plays a role in the process of meaning construction. In a first study, I exploited the metaphor linking time and space to assess the specific contribution of linguistically–coded information to the emergence of priming. In fact, time is metaphorically arranged on either the horizontal or the sagittal axis in space (Clark, 1973), but only the latter comes up in language (e.g., "a bright future in front of you"). In a semantic categorization task, temporal target words (e.g., earlier, later) were primed by spatial words that were processed either consciously (unmasked) or unconsciously (masked). With visible primes, priming was observed for both lateral and sagittal words; yet, only the latter ones led to a significant effect when the primes were masked. Thus, unconscious word processing may be limited to those aspects of meaning that emerge in language use. In a second series of experiments, I tried to better characterize these aspects by taking advantage of Distributional Semantic Models (DSMs; Marelli, 2017), which represent word meaning as vectors built upon word co–occurrences in large textual database. I compared state–of–the–art DSMs with Pointwise Mutual Information (PMI; Church & Hanks, 1990), a measure of local association between words that is merely based on their surface co–occurrence. In particular, I tested how the two indexes perform on a semantic priming dataset comprising visible and masked primes, and different stimulus onset asynchronies between the two stimuli. Subliminally, none of the predictor alone elicited significant priming, although participants who showed some residual prime visibility showed larger effect. Post-hoc analyses showed that for subliminal priming to emerge, the additive contribution of both PMI and DSM was required. Supraliminally, PMI outperforms DSM in the fit to the behavioral data. According to these results, what has been traditionally thought of as unconscious semantic priming may mostly rely on local associations based on shallow word cooccurrence. Of course, masked priming is only one possible way to model unconscious perception. In an attempt to provide converging evidence, I also tested overt and covert semantic facilitation by presenting prime words in the unattended vs. attended visual hemifield of brain–injured patients suffering from neglect. In seven sub–acute cases, data show more solid PMI–based than DSM–based priming in the unattended hemifield, confirming the results obtained from healthy participants. Finally, in a fourth work package, I explored the neural underpinnings of semantic processing as revealed by EEG (Kutas & Federmeier, 2011). As the behavioral results of the previous study were much clearer when the primes were visible, I focused on this condition only. Semantic congruency was dichotomized in order to compare the ERP evoked by related and unrelated pairs. Three different types of semantic similarity were taken into account: in a first category, primes and targets were often co–occurring but far in the DSM (e.g., cheese-mouse), while in a second category the two words were closed in the DSM, but not likely to co-occur (e.g., lamp-torch). As a control condition, we added a third category with pairs that were both high in PMI and close in DSMs (e.g., lemon-orange). Mirroring the behavioral results, we observed a significant PMI effect in the N400 time window; no such effect emerged for DSM. References Church, K. W., & Hanks, P. (1990). Word association norms, mutual information, and lexicography. Computational linguistics, 16(1), 22-29. Clark, H. H. (1973). Space, time, semantics, and the child. In Cognitive development and acquisition of language (pp. 27-63). Academic Press. Kutas, M., & Federmeier, K. D. (2011). Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). Annual review of psychology, 62, 621-647. Marelli, M. (2017). Word-Embeddings Italian Semantic Spaces: a semantic model for psycholinguistic research. Psihologija, 50(4), 503-520. Commentat

Using a high-dimensional model of semantic space to predict neural activity

This dissertation research developed the GOLD model (Graph Of Language Distribution), a graph-structured semantic space model constructed based on co-occurrence in a large corpus of natural language, with the intent that it may be used to explore what information may be present about relationships between words in such a model and the degree to which this information may be used to predict brain responses and behavior in language tasks. The present study employed GOLD to examine genera relatedness as well as two specific types of relationship between words: semantic similarity, which refers to the degree of overlap in meaning between words, and associative relatedness, which refers to the degree to which two words occur in the same schematic context. It was hypothesized that this graph-structured model of language constructed based on co-occurrence should easily capture associative relatedness, because this type of relationship is thought to be present directly in lexical co-occurrence. Additionally, it was hypothesized that semantic similarity may be extracted from the intersection of the set of first-order connections, because two words that are semantically similar may occupy similar thematic or syntactic roles across contexts and thus would co-occur lexically with the same set of nodes. Based on these hypotheses, a set of relationship metrics were extracted from the GOLD model, and machine learning techniques were used to explore predictive properties of these metrics. GOLD successfully predicted behavioral data as well as neural activity in response to words with varying relationships, and its predictions outperformed those of certain competing models. These results suggest that a single-mechanism account of learning word meaning from context may suffice to account for a variety of relationships between words. Further benefits of graph models of language are discussed, including their transparent record of language experience, easy interpretability, and increased psychologically plausibility over models that perform complex transformations of meaning representation

From sequences to cognitive structures : neurocomputational mechanisms

Ph. D. Thesis.Understanding how the brain forms representations of structured information distributed in time is a challenging neuroscientific endeavour, necessitating computationally and neurobiologically informed study. Human neuroimaging evidence demonstrates engagement of a fronto-temporal network, including ventrolateral prefrontal cortex (vlPFC), during language comprehension. Corresponding regions are engaged when processing dependencies between word-like items in Artificial Grammar (AG) paradigms. However, the neurocomputations supporting dependency processing and sequential structure-building are poorly understood. This work aimed to clarify these processes in humans, integrating behavioural, electrophysiological and computational evidence. I devised a novel auditory AG task to assess simultaneous learning of dependencies between adjacent and non-adjacent items, incorporating learning aids including prosody, feedback, delineated sequence boundaries, staged pre-exposure, and variable intervening items. Behavioural data obtained in 50 healthy adults revealed strongly bimodal performance despite these cues. Notably, however, reaction times revealed sensitivity to the grammar even in low performers. Behavioural and intracranial electrode data was subsequently obtained in 12 neurosurgical patients performing this task. Despite chance behavioural performance, time- and time-frequency domain electrophysiological analysis revealed selective responsiveness to sequence grammaticality in regions including vlPFC. I developed a novel neurocomputational model (VS-BIND: “Vector-symbolic Sequencing of Binding INstantiating Dependencies”), triangulating evidence to clarify putative mechanisms in the fronto-temporal language network. I then undertook multivariate analyses on the AG task neural data, revealing responses compatible with the presence of ordinal codes in vlPFC, consistent with VS-BIND. I also developed a novel method of causal analysis on multivariate patterns, representational Granger causality, capable of detecting flow of distinct representations within the brain. This alluded to top-down transmission of syntactic predictions during the AG task, from vlPFC to auditory cortex, largely in the opposite direction to stimulus encodings, consistent with predictive coding accounts. It finally suggested roles for the temporoparietal junction and frontal operculum during grammaticality processing, congruent with prior literature. This work provides novel insights into the neurocomputational basis of cognitive structure-building, generating hypotheses for future study, and potentially contributing to AI and translational efforts.Wellcome Trust, European Research Counci

Relating lexical and syntactic processes in language: Bridging research in humans and machines

Potential to bridge research on language in humans and machines is substantial - as linguists and cognitive scientists apply scientific theory and methods to understand how language is processed and represented by humans, computer scientists apply computational methods to determine how to process and represent language in machines. The present work integrates approaches from each of these domains in order to tackle an issue of relevance for both: the nature of the relationship between low-level lexical processes and syntactically-driven interpretation processes. In the first part of the dissertation, this distinction between lexical and syntactic processes focuses on understanding asyntactic lexical effects in online sentence comprehension in humans, and the relationship of those effects to syntactically-driven interpretation processes. I draw on computational methods for simulating these lexical effects and their relationship to interpretation processes. In the latter part of the dissertation, the lexical/syntactic distinction is focused on the application of semantic composition to complex lexical content, for derivation of sentence meaning. For this work I draw on methodology from cognitive neuroscience and linguistics to analyze the capacity of natural language processing systems to do vector-based sentence composition, in order to improve the capacities of models to compose and represent sentence meaning

Using Language Models and Latent Semantic Analysis to Characterise the N400m Neural Response

The N400 is a human neuroelectric response to semantic incongruity in on-line sentence processing, and implausibility in context has been identified as one of the factors that influence the size of the N400. In this paper we investigate whether predictors derived from Latent Semantic Analysis, language models, and Roark’s parser are significant in modeling of the N400m (the neuromagnetic version of the N400). We also investigate significance of a novel pairwise-priming language model based on the IBM Model 1 translation model. Our experiments show that all the predictors are significant. Moreover, we show that predictors based on the 4-gram language model and the pairwise-priming language model are highly correlated with the manual annotation of contextual plausibility, suggesting that these predictors are capable of playing the same role as the manual annotations in prediction of the N400m response. We also show that the proposed predictors can be grouped into two clusters of significant predictors, suggesting that each cluster is capturing a different characteristic of the N400m response.