Expectation-based Comprehension : Modeling the Interaction of World Knowledge and Linguistic Experience

The processing difficulty of each word we encounter in a sentence is affected by both our prior linguistic experience and our general knowledge about the world. Computational models of incremental language processing have, however, been limited in accounting for the influence of world knowledge. We develop an incremental model of language comprehension that constructs—on a word-by-word basis—rich, probabilistic situation model representations. To quantify linguistic processing effort, we adopt Surprisal Theory, which asserts that the processing difficulty incurred by a word is inversely proportional to its expectancy (Hale, 2001; Levy, 2008). In contrast with typical language model implementations of surprisal, the proposed model instantiates a novel comprehension-centric metric of surprisal that reflects the likelihood of the unfolding utterance meaning as established after processing each word. Simulations are presented that demonstrate that linguistic experience and world knowledge are integrated in the model at the level of interpretation and combine in determining online expectations

Semantic Entropy in Language Comprehension

Language is processed on a more or less word-by-word basis, and the processing difficulty induced by each word is affected by our prior linguistic experience as well as our general knowledge about the world. Surprisal and entropy reduction have been independently proposed as linking theories between word processing difficulty and probabilistic language models. Extant models, however, are typically limited to capturing linguistic experience and hence cannot account for the influence of world knowledge. A recent comprehension model by Venhuizen, Crocker, and Brouwer (2019, Discourse Processes) improves upon this situation by instantiating a comprehension-centric metric of surprisal that integrates linguistic experience and world knowledge at the level of interpretation and combines them in determining online expectations. Here, we extend this work by deriving a comprehension-centric metric of entropy reduction from this model. In contrast to previous work, which has found that surprisal and entropy reduction are not easily dissociated, we do find a clear dissociation in our model. While both surprisal and entropy reduction derive from the same cognitive process—the word-by-word updating of the unfolding interpretation—they reflect different aspects of this process: state-by-state expectation (surprisal) versus end-state confirmation (entropy reduction)

Neurobehavioral Correlates of Surprisal in Language Comprehension : A Neurocomputational Model

Expectation-based theories of language comprehension, in particular Surprisal Theory, go a long way in accounting for the behavioral correlates of word-by-word processing difficulty, such as reading times. An open question, however, is in which component(s) of the Event-Related brain Potential (ERP) signal Surprisal is reflected, and how these electrophysiological correlates relate to behavioral processing indices. Here, we address this question by instantiating an explicit neurocomputational model of incremental, word-by-word language comprehension that produces estimates of the N400 and the P600—the two most salient ERP components for language processing—as well as estimates of “comprehension-centric” Surprisal for each word in a sentence. We derive model predictions for a recent experimental design that directly investigates “world-knowledge”-induced Surprisal. By relating these predictions to both empirical electrophysiological and behavioral results, we establish a close link between Surprisal, as indexed by reading times, and the P600 component of the ERP signal. The resultant model thus offers an integrated neurobehavioral account of processing difficulty in language comprehension

Distributional Formal Semantics

Natural language semantics has recently sought to combine the complementary strengths of formal and distributional approaches to meaning. More specifically, proposals have been put forward to augment formal semantic machinery with distributional meaning representations, thereby introducing the notion of semantic similarity into formal semantics, or to define distributional systems that aim to incorporate formal notions such as entailment and compositionality. However, given the fundamentally different 'representational currency' underlying formal and distributional approaches - models of the world versus linguistic co-occurrence - their unification has proven extremely difficult. Here, we define a Distributional Formal Semantics that integrates distributionality into a formal semantic system on the level of formal models. This approach offers probabilistic, distributed meaning representations that are also inherently compositional, and that naturally capture fundamental semantic notions such as quantification and entailment. Furthermore, we show how the probabilistic nature of these representations allows for probabilistic inference, and how the information-theoretic notion of "information" (measured in terms of Entropy and Surprisal) naturally follows from it. Finally, we illustrate how meaning representations can be derived incrementally from linguistic input using a recurrent neural network model, and how the resultant incremental semantic construction procedure intuitively captures key semantic phenomena, including negation, presupposition, and anaphoricity.Comment: To appear in: Information and Computation (WoLLIC 2019 Special Issue

A Neurocomputational Model of the N400 and the P600 in Language Processing

Ten years ago, researchers using event-related brain potentials (ERPs) to study language comprehension were puzzled by what looked like a Semantic Illusion: Semantically anomalous, but structurally well-formed sentences did not affect the N400 component—traditionally taken to reflect semantic integration—but instead produced a P600 effect, which is generally linked to syntactic processing. This finding led to a considerable amount of debate, and a number of complex processing models have been proposed as an explanation. What these models have in common is that they postulate two or more separate processing streams, in order to reconcile the Semantic Illusion and other semantically induced P600 effects with the traditional interpretations of the N400 and the P600. Recently, however, these multi-stream models have been called into question, and a simpler single-stream model has been proposed. According to this alternative model, the N400 component reflects the retrieval of word meaning from semantic memory, and the P600 component indexes the integration of this meaning into the unfolding utterance interpretation. In the present paper, we provide support for this “Retrieval–Integration (RI)” account by instantiating it as a neurocomputational model. This neurocomputational model is the first to successfully simulate the N400 and P600 amplitude in language comprehension, and simulations with this model provide a proof of concept of the single-stream RI account of semantically induced patterns of N400 and P600 modulations

Discourse semantics with information structure

The property of projection poses a challenge to formal semantic theories, due to its apparent non-compositional nature. Projected content is therefore typically analyzed as being different from and independent of asserted content. Recent evidence, however, suggests that these types of content in fact closely interact, thereby calling for a more integrated analysis that captures their similarities, while respecting their differences. Here, we propose such a unified, compositional semantic analysis of asserted and projected content. Our analysis captures the similarities and differences between presuppositions, anaphora, conventional implicatures and assertions on the basis of their information structure, that is, on basis of how their content is contributed to the unfolding discourse context. We formalize our analysis in an extension of the dynamic semantic framework of Discourse Representation Theory (DRT)—called Projective DRT (PDRT)—that employs projection variables to capture the information-structural aspects of semantic content; different constellations of such variables capture the differences between the different types of projected and asserted content within a single dimension of meaning. We formally derive the structural and compositional properties of PDRT, as well as its semantic interpretation. By instantiating PDRT as a mature semantic formalism, we argue that it paves way for a more focused investigation of the information-structural aspects of meaning

Semantic Entropy in Language Comprehension

Language is processed on a more or less word-by-word basis, and the processing difficulty induced by each word is affected by our prior linguistic experience as well as our general knowledge about the world. Surprisal and entropy reduction have been independently proposed as linking theories between word processing difficulty and probabilistic language models. Extant models, however, are typically limited to capturing linguistic experience and hence cannot account for the influence of world knowledge. A recent comprehension model by Venhuizen, Crocker, and Brouwer (2019, Discourse Processes) improves upon this situation by instantiating a comprehension-centric metric of surprisal that integrates linguistic experience and world knowledge at the level of interpretation and combines them in determining online expectations. Here, we extend this work by deriving a comprehension-centric metric of entropy reduction from this model. In contrast to previous work, which has found that surprisal and entropy reduction are not easily dissociated, we do find a clear dissociation in our model. While both surprisal and entropy reduction derive from the same cognitive process—the word-by-word updating of the unfolding interpretation—they reflect different aspects of this process: state-by-state expectation (surprisal) versus end-state confirmation (entropy reduction)

2013. Gamification for word sense labeling

Obtaining gold standard data for word sense disambiguation is important but costly. We show how it can be done using a “Game with a Purpose ” (GWAP) called Wordrobe. This game consists of a large set of multiple-choice questions on word senses generated from the Groningen Meaning Bank. The players need to answer these questions, scoring points depending on the agreement with fellow players. The working assumption is that the right sense for a word can be determined by the answers given by the players. To evaluate our method, we gold-standard tagged a portion of the data that was also used in the GWAP. A comparison yielded promising results, ranging from a precision of 0.88 and recall of 0.83 for relative majority agreement, to a precision of 0.98 and recall of 0.35 for questions that were answered unanimously.

How and why conventional implicatures project

Conventional Implicatures (CIs; in the sense of Potts 2005) are part of a larger class of projection phenomena. These phenomena also include presuppositions and anaphora, and can described as content that is not at-issue (cf. Simons, Tonhauser, Beaver & Roberts 2010). Despite the shared property of projection, CIs differ from other projection phenomena with respect to the information status of their contribution. Presuppositions, for instance, refer to established, or old information, whereas CIs contribute novel information to the discourse, like at-issue content. Here, we propose a unidimensional analysis of CIs and at-issue content, which highlights the similarity in projection behaviour of CIs, presuppositions, and anaphora. This analysis treats CIs as ‘piggybacking’ on their anchor; they introduce an anaphoric dependency on the interpretation site of their anchor, while at the same time requiring their anchor to refer to a specific referent in the discourse context. CIs are thus elaborations on the description of the referent referred to by their anchor. This analysis of CIs is formalized in Projective Discourse Representation Theory (PDRT; Venhuizen, Bos & Brouwer 2013), a representational framework in which the property of projection is accounted for by explicitly distinguishing between the introduction and interpretation site of semantic content. Our formal analysis explains the interpretational differences between CIs, presuppositions, anaphora, and at-issue content, without stipulating a fundamental distinction between them