Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESupposition

Natural language inference (NLI) is an increasingly important task for natural language understanding, which requires one to infer whether a sentence entails another. However, the ability of NLI models to make pragmatic inferences remains understudied. We create an IMPlicature and PRESupposition diagnostic dataset (IMPPRES), consisting of >25k semiautomatically generated sentence pairs illustrating well-studied pragmatic inference types. We use IMPPRES to evaluate whether BERT, InferSent, and BOW NLI models trained on MultiNLI (Williams et al., 2018) learn to make pragmatic inferences. Although MultiNLI appears to contain very few pairs illustrating these inference types, we find that BERT learns to draw pragmatic inferences. It reliably treats scalar implicatures triggered by "some" as entailments. For some presupposition triggers like "only", BERT reliably recognizes the presupposition as an entailment, even when the trigger is embedded under an entailment canceling operator like negation. BOW and InferSent show weaker evidence of pragmatic reasoning. We conclude that NLI training encourages models to learn some, but not all, pragmatic inferences.Comment: to appear in Proceedings of ACL 202

Distinct neural correlates for pragmatic and semantic meaning processing: An event-related potential investigation of scalar implicature processing using picture-sentence verification

The present study examines the brain-level representation and composition of meaning in scalar quantifiers (e.g., some), which have both a semantic meaning (at least one) and a pragmatic meaning (not all). We adopted a picture-sentence verification design to examine event-related potential (ERP) effects of reading infelicitous quantifiers for which the semantic meaning was correct with respect to the context but the pragmatic meaning was not, compared to quantifiers for which the semantic meaning was inconsistent with the context and no additional pragmatic meaning is available. In the first experiment, only pragmatically inconsistent quantifiers, not semantically inconsistent quantifiers, elicited a sustained posterior negative component. This late negativity contrasts with the N400 effect typically elicited by nouns that are incongruent with their context, suggesting that the recognition of scalar implicature errors elicits a qualitatively different ERP signature than the recognition of lexico-semantic errors. We hypothesize that the sustained negativity reflects cancellation of the pragmatic inference and retrieval of the semantic meaning. In our second experiment, we found that the process of re-interpreting the quantifier was independent from lexico-semantic processing: the N400 elicited by lexico-semantic violations was not modulated by the presence of a pragmatic inconsistency. These findings suggest that inferential pragmatic aspects of meaning are processed using different mechanisms than lexical or combinatorial semantic aspects of meaning, that inferential pragmatic meaning can be realized rapidly, and that the computation of meaning involves continuous negotiation between different aspects of meaning

Expectations over Unspoken Alternatives Predict Pragmatic Inferences

Scalar inferences (SI) are a signature example of how humans interpret language based on unspoken alternatives. While empirical studies have demonstrated that human SI rates are highly variable -- both within instances of a single scale, and across different scales -- there have been few proposals that quantitatively explain both cross- and within-scale variation. Furthermore, while it is generally assumed that SIs arise through reasoning about unspoken alternatives, it remains debated whether humans reason about alternatives as linguistic forms, or at the level of concepts. Here, we test a shared mechanism explaining SI rates within and across scales: context-driven expectations about the unspoken alternatives. Using neural language models to approximate human predictive distributions, we find that SI rates are captured by the expectedness of the strong scalemate as an alternative. Crucially, however, expectedness robustly predicts cross-scale variation only under a meaning-based view of alternatives. Our results suggest that pragmatic inferences arise from context-driven expectations over alternatives, and these expectations operate at the level of concepts.Comment: To appear in TACL (pre-MIT Press publication version

Cognitive Processing of Verbal Quantifiers in the Context of Affirmative and Negative Sentences: a Croatian Study

Studies from English and German have found differences in the processing of affirmative and negative sentences. However, little attention has been given to quantifiers that form negations. A picture-sentence verification task was used to investigate the processing of different types of quantifiers in Croatian: universal quantifiers in affirmative sentences (e.g. all), non-universal quantifiers in compositional negations (e.g. not all), null quantifiers in negative concord (e.g. none) and relative disproportionate quantifiers in both affirmative and negative sentences (e.g. some). The results showed that non-universal and null quantifiers, as well as negations were processed significantly slower compared to affirmative sentences, which is in line with previous findings supporting the two-step model. The results also confirmed that more complex tasks require a longer reaction time. A significant difference in the processing of same-polarity sentences with first-order quantifiers was observed: sentences with null quantifiers were processed faster and more accurately than sentences with disproportional and non-universal quantifiers. A difference in reaction time was also found in affirmatives with different quantifiers: sentences with universal quantifiers were processed significantly faster and more accurately compared to sentences with relative disproportionate quantifiers. These findings indicate that the processing of quantifiers follows after the processing of affirmative information. In the context of the two-step model, the processing of quantifiers occurs in the second step, along with negations

The role of alternatives in language

In this review we provide a discussion of the concept of alternatives and its role in linguistic and psycholinguistic theorizing in the context of the contributions that have appeared in the Frontiers Research Topic The Role of Alternatives in Language. We are discussing the linguistic phenomena for which alternatives have been argued to play a paramount role: negation, counterfactual sentences, scalar implicatures and exhaustivity, focus, contrastive topics, and sentences with bare plurals and with definite plurals. We review in how far alternatives are relevant for these phenomena and how this relevance has been captured by theoretical linguistic accounts. Regarding processing, we discuss the mental activation of alternatives: its mandatory vs. optional nature, its time course. We also address the methodological issue of how experimental studies operationalize alternatives. Finally, we explore the phenomenon of individual variation, which increasingly attracts attention in linguistics. In sum, this review gives an inclusive and broad discussion of alternatives by bringing together different research strands whose findings and theoretical proposals can advance our knowledge of alternatives in inspiring cross-fertilization.Peer Reviewe

Flexible Expectations of Speaker Informativeness Shape Pragmatic Inference

Human communication relies on shared expectations between speakers and hearers. For example, upon hearing a sentence like “Some of my dogs bark,” the listener typically assumes that the speaker did not intend the literal semantic meaning (“At least one (and possibly all) of my dogs bark”). Instead, s/he is likely to derive a scalar implicature (SI), inferring that the speaker intended to convey “Not all of my dogs bark.” Properties of the speaker are known to affect whether listeners compute SIs, with comprehenders being less likely to make a pragmatic inference when the speaker is not knowledgeable of the situation at hand. What is unclear is whether listeners also use previously-held expectations about speaker groups (e.g., children, non-native speakers) to override Gricean principles, in such a way that is stable across situational contexts and does not require an adaptation period. Across two experiments, we investigated how listeners interpret under-informative utterances produced by native and non-native speakers. We found that a subset of individuals is more tolerant to pragmatic infelicities produced by non-native speakers (Exp. 2), but that this tolerance is subject to individual differences in language processing ability and does not emerge in a speeded task where the utterances are supported by visual context (Exp. 1)