On the Effect of Anticipation on Reading Times

Over the past two decades, numerous studies have demonstrated how less predictable (i.e., higher surprisal) words take more time to read. In general, these studies have implicitly assumed the reading process is purely responsive: Readers observe a new word and allocate time to process it as required. We argue that prior results are also compatible with a reading process that is at least partially anticipatory: Readers could make predictions about a future word and allocate time to process it based on their expectation. In this work, we operationalize this anticipation as a word's contextual entropy. We assess the effect of anticipation on reading by comparing how well surprisal and contextual entropy predict reading times on four naturalistic reading datasets: two self-paced and two eye-tracking. Experimentally, across datasets and analyses, we find substantial evidence for effects of contextual entropy over surprisal on a word's reading time (RT): in fact, entropy is sometimes better than surprisal in predicting a word's RT. Spillover effects, however, are generally not captured by entropy, but only by surprisal. Further, we hypothesize four cognitive mechanisms through which contextual entropy could impact RTs -- three of which we are able to design experiments to analyze. Overall, our results support a view of reading that is not just responsive, but also anticipatory.Comment: This is a pre-MIT Press publication version of the paper. Code is available in https://github.com/rycolab/anticipation-on-reading-time

Testing the Predictions of Surprisal Theory in 11 Languages

A fundamental result in psycholinguistics is that less predictable words take a longer time to process. One theoretical explanation for this finding is Surprisal Theory (Hale, 2001; Levy, 2008), which quantifies a word's predictability as its surprisal, i.e. its negative log-probability given a context. While evidence supporting the predictions of Surprisal Theory have been replicated widely, most have focused on a very narrow slice of data: native English speakers reading English texts. Indeed, no comprehensive multilingual analysis exists. We address this gap in the current literature by investigating the relationship between surprisal and reading times in eleven different languages, distributed across five language families. Deriving estimates from language models trained on monolingual and multilingual corpora, we test three predictions associated with surprisal theory: (i) whether surprisal is predictive of reading times; (ii) whether expected surprisal, i.e. contextual entropy, is predictive of reading times; (iii) and whether the linking function between surprisal and reading times is linear. We find that all three predictions are borne out crosslinguistically. By focusing on a more diverse set of languages, we argue that these results offer the most robust link to-date between information theory and incremental language processing across languages.Comment: This is a pre-MIT Press publication version of the pape

Revisiting the Uniform Information Density Hypothesis

The uniform information density (UID) hypothesis posits a preference among language users for utterances structured such that information is distributed uniformly across a signal. While its implications on language production have been well explored, the hypothesis potentially makes predictions about language comprehension and linguistic acceptability as well. Further, it is unclear how uniformity in a linguistic signal -- or lack thereof -- should be measured, and over which linguistic unit, e.g., the sentence or language level, this uniformity should hold. Here we investigate these facets of the UID hypothesis using reading time and acceptability data. While our reading time results are generally consistent with previous work, they are also consistent with a weakly super-linear effect of surprisal, which would be compatible with UID's predictions. For acceptability judgments, we find clearer evidence that non-uniformity in information density is predictive of lower acceptability. We then explore multiple operationalizations of UID, motivated by different interpretations of the original hypothesis, and analyze the scope over which the pressure towards uniformity is exerted. The explanatory power of a subset of the proposed operationalizations suggests that the strongest trend may be a regression towards a mean surprisal across the language, rather than the phrase, sentence, or document -- a finding that supports a typical interpretation of UID, namely that it is the byproduct of language users maximizing the use of a (hypothetical) communication channel

A Cross-Linguistic Pressure for Uniform Information Density in Word Order

While natural languages differ widely in both canonical word order and word order flexibility, their word orders still follow shared cross-linguistic statistical patterns, often attributed to functional pressures. In the effort to identify these pressures, prior work has compared real and counterfactual word orders. Yet one functional pressure has been overlooked in such investigations: the uniform information density (UID) hypothesis, which holds that information should be spread evenly throughout an utterance. Here, we ask whether a pressure for UID may have influenced word order patterns cross-linguistically. To this end, we use computational models to test whether real orders lead to greater information uniformity than counterfactual orders. In our empirical study of 10 typologically diverse languages, we find that: (i) among SVO languages, real word orders consistently have greater uniformity than reverse word orders, and (ii) only linguistically implausible counterfactual orders consistently exceed the uniformity of real orders. These findings are compatible with a pressure for information uniformity in the development and usage of natural languages

Is Hamlet Scandinavian Crime Fiction?

This paper combines two of the suggested topics: “law as an instrument of ideology” and “ideological interpellation through law” as it explores whether or not it is possible to use literary fiction as part of an argument in legal argumentation. The use of such an argument is strongly connected with an attitude of state to art, culture and values included both in art and law. Art as a part of a socio-cultural system is one of the material sources of law so it would be natural to admit it in just judicial decision. Unfortunately it is not typical for law to use such relationships between art and law to make judicial decisions better or at least more persuasive. It may be caused by the fact that the choice of what literary fiction is suitable for legal argumentation and which is not can be seen as a kind of ideology. Therefore the state determines which art is “good enough” to be a part of legal reasoning. Usually it differs between “high art” and “mass culture.” It results in a form of “labelling” of art. Unfortunately, by evaluating art in such a manner each state manifests itself as almost a totalitarian one. Socialist realism was a very expressive example. So it is not a matter for the democratic state to decide which piece of art is capable to influence law, is it? In this contribution I will emphasize Žižek’s critique of ideology in order to deal with ideology in argumentation by literary fiction in law. Besides that I will draw inspiration from Law and Literature movements. The aim of my paper is to explain how literary fiction can be used as a legal argument in a proper way as a necessary social appeal through law.This paper combines two of the suggested topics: “law as an instrument of ideology” and “ideological interpellation through law” as it explores whether or not it is possible to use literary fiction as part of an argument in legal argumentation. The use of such an argument is strongly connected with an attitude of state to art, culture and values included both in art and law. Art as a part of a socio-cultural system is one of the material sources of law so it would be natural to admit it in just judicial decision. Unfortunately it is not typical for law to use such relationships between art and law to make judicial decisions better or at least more persuasive. It may be caused by the fact that the choice of what literary fiction is suitable for legal argumentation and which is not can be seen as a kind of ideology. Therefore the state determines which art is “good enough” to be a part of legal reasoning. Usually it differs between “high art” and “mass culture.” It results in a form of “labelling” of art. Unfortunately, by evaluating art in such a manner each state manifests itself as almost a totalitarian one. Socialist realism was a very expressive example. So it is not a matter for the democratic state to decide which piece of art is capable to influence law, is it? In this contribution I will emphasize Žižek’s critique of ideology in order to deal with ideology in argumentation by literary fiction in law. Besides that I will draw inspiration from Law and Literature movements. The aim of my paper is to explain how literary fiction can be used as a legal argument in a proper way as a necessary social appeal through law

Navigating New Landscapes: The Contribution of Socio-Legal Scholarship in Mapping the Plurality of International Economic Law and Locating Power in International Economic Relations

The evolution of international economic law in the past two decades has been characterised by the growth and diversification of international economic actors, the expansion in the substantive areas governed by international law, and, crucially, the proliferation of multiple sites of international economic governance. This web of multi-layered international economic governance is, in turn, underpinned by complex dynamics of power which structure the legal and economic relations between the subjects of international economic law and other actors impacted by international legal rules and regulation. The challenge for international legal scholarship lay not only in mapping the multiple sites of international economic governance but also in unmasking the power dynamics inherent in international economic relations. Locating and analysing power relations underlying international economic law is to crucial to understanding the cause and effect of international economic rules and institutions for rulemaking. Conventional legal scholarship with its doctrinal focus, while useful in providing the foundational basis for analysis, cannot adequately capture the complexity of contemporary international economic law. Socio-legal approaches may be able to overcome these epistemological limitations by supplying: a) the methodologies to study international economic law beyond a focus on rules and institutions; and b) the critical theoretical lens to understand the power dynamics inherent in international legal relations. The objective of this paper is twofold: firstly, it will seek to identify the contributions of socio-legal approaches to the study of international economic law; and secondly, it will explore how socio-legal scholarship can provide a methodological and theoretical framework to construct an understanding of the pluralistic nature of international economic regulatory regimes and their underlying dynamics of power. In doing so, the paper will also consider the value of juxtaposing an empirical methodology for mapping legal regimes with a critical normative approach for analysing power relations in international economic law

A Plug-and-Play Method for Controlled Text Generation

Large pre-trained language models have repeatedly shown their ability to produce fluent text. Yet even when starting from a prompt, generation can continue in many plausible directions. Current decoding methods with the goal of controlling generation, e.g., to ensure specific words are included, either require additional models or fine-tuning, or work poorly when the task at hand is semantically unconstrained, e.g., story generation. In this work, we present a plug-and-play decoding method for controlled language generation that is so simple and intuitive, it can be described in a single sentence: given a topic or keyword, we add a shift to the probability distribution over our vocabulary towards semantically similar words. We show how annealing this distribution can be used to impose hard constraints on language generation, something no other plug-and-play method is currently able to do with SOTA language generators. Despite the simplicity of this approach, we see it works incredibly well in practice: decoding from GPT-2 leads to diverse and fluent sentences while guaranteeing the appearance of given guide words. We perform two user studies, revealing that (1) our method outperforms competing methods in human evaluations; and (2) forcing the guide words to appear in the generated text has no impact on the fluency of the generated text