Filaments of Meaning in Word Space

Word space models, in the sense of vector space models built on distributional data taken from texts, are used to model semantic relations between words. We argue that the high dimensionality of typical vector space models lead to unintuitive effects on modeling likeness of meaning and that the local structure of word spaces is where interesting semantic relations reside. We show that the local structure of word spaces has substantially different dimensionality and character than the global space and that this structure shows potential to be exploited for further semantic analysis using methods for local analysis of vector space structure rather than globally scoped methods typically in use today such as singular value decomposition or principal component analysis

Aberrant activity in conceptual networks underlies N400 deficits and unusual thoughts in schizophrenia.

BackgroundThe N400 event-related potential (ERP) is triggered by meaningful stimuli that are incongruous, or unmatched, with their semantic context. Functional magnetic resonance imaging (fMRI) studies have identified brain regions activated by semantic incongruity, but their precise links to the N400 ERP are unclear. In schizophrenia (SZ), N400 amplitude reduction is thought to reflect overly broad associations in semantic networks, but the abnormalities in brain networks underlying deficient N400 remain unknown. We utilized joint independent component analysis (JICA) to link temporal patterns in ERPs to neuroanatomical patterns from fMRI and investigate relationships between N400 amplitude and neuroanatomical activation in SZ patients and healthy controls (HC).MethodsSZ patients (n = 24) and HC participants (n = 25) performed a picture-word matching task, in which words were either matched (APPLE→apple) by preceding pictures, or were unmatched by semantically related (in-category; IC, APPLE→lemon) or unrelated (out of category; OC, APPLE→cow) pictures, in separate ERP and fMRI sessions. A JICA "data fusion" analysis was conducted to identify the fMRI brain regions specifically associated with the ERP N400 component. SZ and HC loading weights were compared and correlations with clinical symptoms were assessed.ResultsJICA identified an ERP-fMRI "fused" component that captured the N400, with loading weights that were reduced in SZ. The JICA map for the IC condition showed peaks of activation in the cingulate, precuneus, bilateral temporal poles and cerebellum, whereas the JICA map from the OC condition was linked primarily to visual cortical activation and the left temporal pole. Among SZ patients, fMRI activity from the IC condition was inversely correlated with unusual thought content.ConclusionsThe neural networks associated with the N400 ERP response to semantic violations depends on conceptual relatedness. These findings are consistent with a distributed network underlying neural responses to semantic incongruity including unimodal visual areas as well as integrative, transmodal areas. Unusual thoughts in SZ may reflect impaired processing in transmodal hub regions such as the precuneus, leading to overly broad semantic associations

N400-like potentials and reaction times index semantic relations between highly repeated individual words

The N400 ERP is an electrophysiological index of semantic processing. Its amplitude varies with the semantic category of words, their concreteness, or whether their meaning matches that of a preceding context. The results of a number of studies suggest that these effects could be markedly reduced or suppressed for stimuli that are repeated. Nevertheless, we have recently shown that significant effects of semantic matching and category could be obtained on N400-like potentials elicited by massively repeated target words in a prime–target semantic categorization task. If such effects could be obtained when primes also are repeated, it would then be possible to study the semantic associations between individual words. The present study thus aimed to test this hypothesis while (1) controlling for a potential contribution of physical matching to the processing of repeated targets and (2) testing if the N400-like effects obtained in these conditions are modulated by task instruction, as are classic N400 effects. Two category words were used as primes and two exemplars as targets. In one block of trials, subjects had to respond according to the semantic relation between prime and target (semantic instruction) and, in another block, they had to report changes in letter case (physical instruction). Results showed that the amplitude of the N400-like ERP obtained was modulated by semantic matching and category but not by letter case. The effect of semantic matching was observed only in the semantic instruction block. Interestingly, the effect of category was not modulated by task instruction. An independent component analysis showed that the component that made the greatest contribution to the effect of semantic matching in the time window of the N400-like potential had a scalp distribution similar to that reported for the N400 and was best fit as a bilateral generator in the superior temporal gyrus. The use of repetition could thus allow, at least in explicit semantic tasks, a drastic simplification of N400 protocols. Highly repeated individual words could be used to study semantic relations between individual concepts

Discovering Universal Geometry in Embeddings with ICA

This study utilizes Independent Component Analysis (ICA) to unveil a consistent semantic structure within embeddings of words or images. Our approach extracts independent semantic components from the embeddings of a pre-trained model by leveraging anisotropic information that remains after the whitening process in Principal Component Analysis (PCA). We demonstrate that each embedding can be expressed as a composition of a few intrinsic interpretable axes and that these semantic axes remain consistent across different languages, algorithms, and modalities. The discovery of a universal semantic structure in the geometric patterns of embeddings enhances our understanding of the representations in embeddings.Comment: 29 pages, EMNLP 202

Response-related potentials during semantic priming: the effect of a speeded button response task on ERPs

This study examines the influence of a button response task on the event-related potential (ERP) in a semantic priming experiment. Of particular interest is the N400 component. In many semantic priming studies, subjects are asked to respond to a stimulus as fast and accurately as possible by pressing a button. Response time (RT) is recorded in parallel with an electroencephalogram (EEG) for ERP analysis. In this case, the response occurs in the time window used for ERP analysis and response-related components may overlap with stimulus-locked ones such as the N400. This has led to a recommendation against such a design, although the issue has not been explored in depth. Since studies keep being published that disregard this issue, a more detailed examination of influence of response-related potentials on the ERP is needed. Two experiments were performed in which subjects pressed one of two buttons with their dominant hand in response to word-pairs with varying association strength (AS), indicating a personal judgement of association between the two words. In the first experiment, subjects were instructed to respond as fast and accurately as possible. In the second experiment, subjects delayed their button response to enforce a one second interval between the onset of the target word and the button response. Results show that in the first experiment a P3 component and motor-related potentials (MRPs) overlap with the N400 component, which can cause a misinterpretation of the latter. In order to study the N400 component, the button response should be delayed to avoid contamination of the ERP with response-related components

Image encoding by independent principal components

The encoding of images by semantic entities is still an unresolved task. This paper proposes the encoding of images by only a few important components or image primitives. Classically, this can be done by the Principal Component Analysis (PCA). Recently, the Independent Component Analysis (ICA) has found strong interest in the signal processing and neural network community. Using this as pattern primitives we aim for source patterns with the highest occurrence probability or highest information. For the example of a synthetic image composed by characters this idea selects the salient ones. For natural images it does not lead to an acceptable reproduction error since no a-priori probabilities can be computed. Combining the traditional principal component criteria of PCA with the independence property of ICA we obtain a better encoding. It turns out that the Independent Principal Components (IPC) in contrast to the Principal Independent Components (PIC) implement the classical demand of Shannon’s rate distortion theory