Attentional modulation of orthographic neighborhood effects during reading: Evidence from event-related brain potentials in a psychological refractory period paradigm

It is often assumed that word reading proceeds automatically. Here, we tested this assumption by recording event-related potentials during a psychological refractory period (PRP) paradigm, requiring lexical decisions about written words. Specifically, we selected words differing in their orthographic neighborhood size–the number of words that can be obtained from a target by exchanging a single letter–and investigated how influences of this variable depend on the availability of central attention. As expected, when attentional resources for lexical decisions were unconstrained, words with many orthographic neighbors elicited larger N400 amplitudes than those with few neighbors. However, under conditions of high temporal overlap with a high priority primary task, the N400 effect was not statistically different from zero. This finding indicates strong attentional influences on processes sensitive to orthographic neighbors during word reading, providing novel evidence against the full automaticity of processes involved in word reading. Furthermore, in conjunction with the observation of an underadditive interaction between stimulus onset asynchrony (SOA) and orthographic neighborhood size in lexical decision performance, commonly taken to indicate automaticity, our results raise issues concerning the standard logic of cognitive slack in the PRP paradigm

The time course of semantic richness effects in visual word recognition

The richness of semantic representations associated with individual words has emerged as an important variable in reading. In the present study we contrasted different measures of semantic richness and explored the time course of their influences during visual word processing as reflected in event-related brain potentials (ERPs). ERPs were recorded while participants performed a lexical decision task on visually presented words and pseudowords. For word stimuli, we orthogonally manipulated two frequently employed measures of semantic richness: the number of semantic features generated in feature-listing tasks and the number of associates based on free association norms. We did not find any influence of the number of associates. In contrast, the number of semantic features modulated ERP amplitudes at central sites starting at about 190 ms, as well as during the later N400 component over centro-parietal regions (300–500 ms). Thus, initial access to semantic representations of single words is fast and word meaning continues to modulate processing later on during reading

Semantic richness effects in visual word processing

Lesen zielt darauf ab, Bedeutung aus geschriebenem Text zu extrahieren. Interessanterweise unterscheiden sich Wörter beträchtlich hinsichtlich der Menge mit ihnen assoziierter Bedeutung, und es wurde kürzlich gezeigt, dass eine hohe Bedeutungshaltigkeit lexikalische und semantische Aufgaben erleichtert. Die vorliegende Dissertation kombiniert ereigniskorrelierte Potentiale (EKPs) und konnektionistische Modellierung, um einige offene Fragen zur Rolle der Bedeutungshaltigkeit bei der Wortverarbeitung anzugehen. Hierbei wurden EKPs verwendet, um den Zeitverlauf unabhängiger Einflüsse der Anzahl semantischer Merkmale und Assoziationen beim Wortlesen zu bestimmen sowie Einflüsse von Bedeutungshaltigkeit auf implizites Wortlernen zu untersuchen. Um die zugrundeliegenden Mechanismen besser zu verstehen, wurden die Ergebnisse anschließend mittels eines semantischen Netzwerk-Modells simuliert. Es zeigten sich keine Einflüsse der Anzahl der Assoziationen, aber eine schnelle Aktivierung semantischer Merkmale, die das EKP bereits ab 190 ms beeinflussten - nur 20 bis 30 ms nach und zeitlich überlappend mit der Aktivierung orthographischer Repräsentationen, die durch N1-Lexikalitätseffekte angezeigt wurden. Im weiteren Verlauf ging eine hohe Merkmalsanzahl mit größeren N400-Amplituden einher. Zudem verstärkten semantische Merkmale Wiederholungseinflüsse auf die Akkuratheit lexikalischer Entscheidungen und N400-Amplituden, was einen ersten Hinweis auf Einflüsse von Bedeutungshaltigkeit auf implizites Wortlernen darstellt. Diese Ergebnisse stehen im Einklang mit merkmalsbasierten semantischen Netzwerk-Modellen. Simulationen legen nahe, dass semantische Aktivierung lexikalische Entscheidungen erleichtert, während Netzwerk-Fehler in engem Zusammenhang mit N400-Amplituden stehen. Da Netzwerk-Fehler psychologisch als implizite Vorhersagefehler interpretiert werden, deuten diese Ergebnisse darauf hin, dass N400-Amplituden implizite Vorhersagefehler im semantischen System widerspiegeln.Language ultimately aims to convey meaning. Importantly, the amount of associated semantic information varies considerably between words. Recent evidence suggests that the richness of semantic representations facilitates performance in lexical and semantic tasks, but much remains to be learned about semantic richness effects. The present dissertation combined event-related brain potentials (ERPs) and connectionist modeling to address several unresolved issues concerning the role of semantic richness in word processing. Specifically, ERPs were employed to investigate the time course of independent influences of the number of semantic features and associates during word reading (study 1) and influences of semantic richness on implicit word learning (study 2). Aiming at advancing a mechanistic understanding of the obtained results, both studies were subsequently simulated using a network model of semantic cognition (study 3). Results showed no influences of the number of associates, but fast access to semantic features, with influences of feature-based semantic richness starting at about 190 ms - a mere 20 to 30 ms after and temporally overlapping with the activation of orthographic representations as reflected by N1 lexicality effects. Later on, a high number of semantic features induced larger N400 amplitudes. Furthermore, the number of semantic features enhanced repetition priming effects on lexical decision accuracy and N400 amplitudes, providing initial evidence for influences of semantic richness on implicit word learning. These results are in line with feature-based network models of semantic cognition. Simulations with such a model suggest that semantic activation can facilitate lexical decisions, while network error closely corresponds to N400 amplitudes. In psychological terms, network error has been conceptualized as implicit prediction error. Thus, these results are taken to suggest that N400 amplitudes reflect implicit prediction error in semantic memory

Predicting the N400 ERP component using the Sentence Gestalt model trained on a large scale corpus

The N400 component of the event related brain potential is widely used to investigate language and meaning processing. However, despite much research the component’s functional basis remains actively debated. Recent work showed that the update of the predictive representation of sentence meaning (semantic update, or SU) generated by the Sentence Gestalt model (McClelland, St. John, & Taraban, 1989) consistently displayed a similar pattern to the N400 amplitude in a series of conditions known to modulate this event-related potential. These results led (Rabovsky, Hansen, and McClelland,2018) to suggest that the N400 might reflect change in a probabilistic representation of meaning corresponding to an implicit semantic prediction error. However, a limitation of this work is that the model was trained on a small artificial training corpus and thus could not be presented with the same naturalistic stimuli presented in empirical experiments. In the present study, we overcome this limitation and directly model the amplitude of the N400 elicited during naturalistic sentence processing by using as predictor the SU generated by a Sentence Gestalt model trained on a large corpus of texts. The results reported in this paper corroborate the hypothesis that the N400 component reflects the change in a probabilistic representation of meaning after every word presentation. Further analyses demonstrate that the SU of the Sentence Gestalt model and the amplitude of the N400 are influenced similarly by the stochastic and positional properties of the linguistic input

Simulating the N400 ERP Component as Semantic Network Error: Insights from a Feature-Based Connectionist Attractor Model of Word Meaning

The N400 ERP component is widely used in research on language and semantic memory. Although the component’s relation to semantic processing is well-established, the computational mechanisms underlying N400 generation are currently unclear (Kutas & Federmeier, 2011). We explored the mechanisms underlying the N400 by examining how a connectionist model’s performance measures covary with N400 amplitudes. We simulated seven N400 effects obtained in human empirical research. Network error was consistently in the same direction as N400 amplitudes, namely larger for low frequency words, larger for words with many features, larger for words with many orthographic neighbors, and smaller for semantically related target words as well as repeated words. Furthermore, the repetition-induced decrease was stronger for low frequency words, and for words with many semantic features. In contrast, semantic activation corresponded less well with the N400. Our results suggest an interesting relation between N400 amplitudes and semantic network error. In psychological terms, error values in connectionist models have been conceptualized as implicit prediction error, and we interpret our results as support for the idea that N400 amplitudes reflect implicit prediction error in semantic memory (McClelland, 1994)

The role of syntactic and semantic cues in preventing illusions of plausibility

Unexpected words within a context elicit large N400 brain potentials. However, sometimes the N400 at an unexpected word is small when stereotypical agent and patient roles are reversed, such as at ‘arrested’ in ‘the cop that the thief arrested’. In a study of 74 native German speakers, we demonstrate evidence that readers can avoid this so-called “N400 semantic illusion” if the verb is delayed with neutral information such as ‘that evening’, but less so if the delay contains cues that could further strengthen the semantic association between the verb and its arguments, such as ‘with handcuffs’. In doing so, we provide a conceptual replication of a relatively new finding, and extend previous research by showing that the semantic content of the delay is important. Moreover, we demonstrate evidence that the effect of only the neutral delay increases as the experiment progresses. We interpret these findings as evidence supporting the Sentence Gestalt model, which proposes that the initial illusion results from uncertainty and an erroneous interpretation based on a strong semantic attractor. The SG may be able to accommodate the delay effect by assuming that the neutral verb delay strengthens syntactic constraints on interpretation by confirming the sentence parse, while the semantically associated delay contains information that further strengthens the semantic attractor. Strengthening the syntactic attractor by showing participants repeated examples of similar sentence structures could account for the additional finding that the neutral delay effect strengthened across the experiment

Meaning in brains and machines: Internal activation update in large-scale language model partially reflects the N400 brain potential

The N400 brain potential has been used as a neural correlate of meaning-related processing in the brain, but its underlying computational mechanism is still not well understood. Although efforts to model the N400 as an update of a probabilistic representation of meaning have been promising, the limited scope of earlier models has restricted experiments to highly simplified sentences. Here, we expand modelling of the N400 to naturalistic sentences using a large-scale, state-of-the-art deep learning language model. We investigate the correspondence between updates in the internal state of the model and the N400 in one quantitative experiment and four qualitative experiments. Our findings suggest that activation updates in the model correspond to several N400 effects, but cannot account for all of them

The N400 event-related potential component reflects a learning signal

A lot of studies on the N400 event-related brain potential (ERP) component have tried to understand its functional significance. Recently, the N400 was modeled in a neural network model as the update of a probabilistic representation of sentence meaning. The change in activation induced by a word was proposed to reflect a semantic prediction error that drives adaptation of the model’s connections (Rabovsky et al., 2018), which implies that more negative N400 amplitudes should lead to greater adaptation. By experimentally manipulating expectancy in a sentence reading task (n=33), we could show that this manipulation did not only influence N400 amplitudes, but also implicit memory: reaction times in a perceptual identification task were significantly faster for previously unexpected words. Additionally, difference in N400 amplitude correlated with the reaction time benefit for unexpected compared to expected items. These finding support the interpretation of the N400 as a prediction error and learning signal