37 research outputs found
Semantic Entropy in Language Comprehension
Language is processed on a more or less word-by-word basis, and the processing difficulty
induced by each word is affected by our prior linguistic experience as well as our general knowledge
about the world. Surprisal and entropy reduction have been independently proposed as linking
theories between word processing difficulty and probabilistic language models. Extant models, however,
are typically limited to capturing linguistic experience and hence cannot account for the influence of
world knowledge. A recent comprehension model by Venhuizen, Crocker, and Brouwer (2019, Discourse
Processes) improves upon this situation by instantiating a comprehension-centric metric of surprisal that
integrates linguistic experience and world knowledge at the level of interpretation and combines them in
determining online expectations. Here, we extend this work by deriving a comprehension-centric metric
of entropy reduction from this model. In contrast to previous work, which has found that surprisal and
entropy reduction are not easily dissociated, we do find a clear dissociation in our model. While both
surprisal and entropy reduction derive from the same cognitive process—the word-by-word updating
of the unfolding interpretation—they reflect different aspects of this process: state-by-state expectation
(surprisal) versus end-state confirmation (entropy reduction)
Expectation-based Comprehension : Modeling the Interaction of World Knowledge and Linguistic Experience
The processing difficulty of each word we encounter in a sentence is affected by both our prior linguistic experience and our general knowledge about the world. Computational models of incremental language processing have, however, been limited in accounting for the influence of world knowledge. We develop an incremental model of language comprehension that constructs—on a word-by-word basis—rich, probabilistic situation model representations. To quantify linguistic processing effort, we adopt Surprisal Theory, which asserts that the processing difficulty incurred by a word is inversely proportional to its expectancy (Hale, 2001; Levy, 2008). In contrast with typical language model implementations of surprisal, the proposed model instantiates a novel comprehension-centric metric of surprisal that reflects the likelihood of the unfolding utterance meaning as established after processing each word. Simulations are presented that demonstrate that linguistic experience and world knowledge are integrated in the model at the level of interpretation and combine in determining online expectations
Neurobehavioral Correlates of Surprisal in Language Comprehension : A Neurocomputational Model
Expectation-based theories of language comprehension, in particular Surprisal Theory,
go a long way in accounting for the behavioral correlates of word-by-word processing
difficulty, such as reading times. An open question, however, is in which component(s)
of the Event-Related brain Potential (ERP) signal Surprisal is reflected, and how these
electrophysiological correlates relate to behavioral processing indices. Here, we address
this question by instantiating an explicit neurocomputational model of incremental,
word-by-word language comprehension that produces estimates of the N400 and
the P600—the two most salient ERP components for language processing—as well
as estimates of “comprehension-centric” Surprisal for each word in a sentence. We
derive model predictions for a recent experimental design that directly investigates
“world-knowledge”-induced Surprisal. By relating these predictions to both empirical
electrophysiological and behavioral results, we establish a close link between Surprisal,
as indexed by reading times, and the P600 component of the ERP signal. The resultant
model thus offers an integrated neurobehavioral account of processing difficulty in
language comprehension
Distributional Formal Semantics
Natural language semantics has recently sought to combine the complementary
strengths of formal and distributional approaches to meaning. More
specifically, proposals have been put forward to augment formal semantic
machinery with distributional meaning representations, thereby introducing the
notion of semantic similarity into formal semantics, or to define
distributional systems that aim to incorporate formal notions such as
entailment and compositionality. However, given the fundamentally different
'representational currency' underlying formal and distributional approaches -
models of the world versus linguistic co-occurrence - their unification has
proven extremely difficult. Here, we define a Distributional Formal Semantics
that integrates distributionality into a formal semantic system on the level of
formal models. This approach offers probabilistic, distributed meaning
representations that are also inherently compositional, and that naturally
capture fundamental semantic notions such as quantification and entailment.
Furthermore, we show how the probabilistic nature of these representations
allows for probabilistic inference, and how the information-theoretic notion of
"information" (measured in terms of Entropy and Surprisal) naturally follows
from it. Finally, we illustrate how meaning representations can be derived
incrementally from linguistic input using a recurrent neural network model, and
how the resultant incremental semantic construction procedure intuitively
captures key semantic phenomena, including negation, presupposition, and
anaphoricity.Comment: To appear in: Information and Computation (WoLLIC 2019 Special Issue
A Neurocomputational Model of the N400 and the P600 in Language Processing
Ten years ago, researchers using event-related brain potentials (ERPs) to study language comprehension were puzzled by what looked like a Semantic Illusion: Semantically anomalous, but structurally well-formed sentences did not affect the N400 component—traditionally taken to reflect semantic integration—but instead produced a P600 effect, which is generally linked to syntactic processing. This finding led to a considerable amount of debate, and a number of complex processing models have been proposed as an explanation. What these models have in common is that they postulate two or more separate processing streams, in order to reconcile the Semantic Illusion and other semantically induced P600 effects with the traditional interpretations of the N400 and the P600. Recently, however, these multi-stream models have been called into question, and a simpler single-stream model has been proposed. According to this alternative model, the N400 component reflects the retrieval of word meaning from semantic memory, and the P600 component indexes the integration of this meaning into the unfolding utterance interpretation. In the present paper, we provide support for this “Retrieval–Integration (RI)” account by instantiating it as a neurocomputational model. This neurocomputational model is the first to successfully simulate the N400 and P600 amplitude in language comprehension, and simulations with this model provide a proof of concept of the single-stream RI account of semantically induced patterns of N400 and P600 modulations
Recommended from our members
Initial Performance Characterization for a Thermalized Neutron Beam for Neutron Capture Therapy Research at Washington State University
The Idaho National Engineering and Environmental Laboratory (INEEL) and Washington State University (WSU) have constructed a new epithermal-neutron beam for collaborative Boron Neutron Capture Therapy (BNCT) preclinical research at the WSU TRIGATM research reactor facility1. More recently, additional beamline components were developed to permit the optional thermalization of the beam for certain types of studies where it is advantageous to use a thermal neutron source rather than an epithermal source. This article summarizes the results of some initial neutronic performance measurements for the thermalized system, with a comparison to the expected performance from the design computations
Recommended from our members
Experimental Transport Benchmarks for Physical Dosimetry to Support Development of Fast-Neutron Therapy with Neutron Capture Augmentation
The Idaho National Laboratory (INL), the University of Washington (UW) Neutron Therapy Center, the University of Essen (Germany) Neutron Therapy Clinic, and the Northern Illinois University(NIU) Institute for Neutron Therapy at Fermilab have been collaborating in the development of fast-neutron therapy (FNT) with concurrent neutron capture (NCT) augmentation [1,2]. As part of this effort, we have conducted measurements to produce suitable benchmark data as an aid in validation of advanced three-dimensional treatment planning methodologies required for successful administration of FNT/NCT. Free-beam spectral measurements as well as phantom measurements with Lucite{trademark} cylinders using thermal, resonance, and threshold activation foil techniques have now been completed at all three clinical accelerator facilities. The same protocol was used for all measurements to facilitate intercomparison of data. The results will be useful for further detailed characterization of the neutron beams of interest as well as for validation of various charged particle and neutron transport codes and methodologies for FNT/NCT computational dosimetry, such as MCNP [3], LAHET [4], and MINERVA [5]
A high quality Arabidopsis transcriptome for accurate transcript-level analysis of alternative splicing
Alternative splicing generates multiple transcript and protein isoforms from the same gene and thus is important in gene expression regulation. To date, RNA-sequencing (RNA-seq) is the standard method for quantifying changes in alternative splicing on a genome-wide scale. Understanding the current limitations of RNA-seq is crucial for reliable analysis and the lack of high quality, comprehensive transcriptomes for most species, including model organisms such as Arabidopsis, is a major constraint in accurate quantification of transcript isoforms. To address this, we designed a novel pipeline with stringent filters and assembled a comprehensive Reference Transcript Dataset for Arabidopsis (AtRTD2) containing 82,190 non-redundant transcripts from 34 212 genes. Extensive experimental validation showed that AtRTD2 and its modified version, AtRTD2-QUASI, for use in Quantification of Alternatively Spliced Isoforms, outperform other available transcriptomes in RNA-seq analysis. This strategy can be implemented in other species to build a pipeline for transcript-level expression and alternative splicing analyses
Recommended from our members
SERA: Simulation Environment for Radiotherapy Applications - Users Manual Version 1CO
This document is the user manual for the Simulation Environment for Radiotherapy Applications (SERA) software program developed for boron-neutron capture therapy (BNCT) patient treatment planning by researchers at the Idaho National Engineering and Environmental Laboratory (INEEL) and students and faculty at Montana State University (MSU) Computer Science Department. This manual corresponds to the final release of the program, Version 1C0, developed to run under the RedHat Linux Operating System (version 7.2 or newer) or the Solaris™ Operating System (version 2.6 or newer). SERA is a suite of command line or interactively launched software modules, including graphical, geometric reconstruction, and execution interface modules for developing BNCT treatment plans. The program allows the user to develop geometric models of the patient as derived from Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) images, perform dose computation for these geometric models, and display the computed doses on overlays of the original images as three dimensional representations. This manual provides a guide to the practical use of SERA, but is not an exhaustive treatment of each feature of the code