BERT Knows Punta Cana is not just Beautiful, it's Gorgeous : Ranking Scalar Adjectives with Contextualised Representations

Adjectives like pretty, beautiful and gorgeous describe positive properties of the nouns they modify but with different intensity. These differences are important for natural language understanding and reasoning. We propose a novel BERT-based approach to intensity detection for scalar adjectives. We model intensity by vectors directly derived from contextualised representations and show they can successfully rank scalar adjectives. We evaluate our models both intrinsically, on gold standard datasets, and on an Indirect Question Answering task. Our results demonstrate that BERT encodes rich knowledge about the semantics of scalar adjectives, and is able to provide better quality intensity rankings than static embeddings and previous models with access to dedicated resources.Peer reviewe

Representation Of Lexical Stylistic Features In Language Models' Embedding Space

The representation space built by pretrained Language Models (LMs) encodes rich information about words and their relationships (e.g., similarity, hypernymy/hyponymy, polysemy) as well as abstract semantic notions (e.g., intensity). In this paper, we demonstrate that lexical stylistic notions such as complexity, formality, and figurativeness, can also be identified in this space. We show that it is possible to derive a vector representation for each of these stylistic notions, from only a small number of seed text pairs. Using these vectors, we can characterize new texts in terms of these dimensions using simple calculations in the corresponding embedding space. We perform experiments on five datasets and find that static embeddings encode these features more accurately at the level of words and phrases, whereas contextualized LMs perform better on longer texts. The lower performance of contextualized representations at the word level is partially attributable to the anisotropy of their vector space, which can be corrected through techniques like standardization to further improve performance.Comment: Accepted at *SEM 202

Linguistic-based computational treatment of textual entailment recognition

In this thesis, I investigate how lexical resources based on the organisation of lexical knowledge in classes which share common (syntactic, semantic, etc.) features support natural language processing and in particular symbolic recognition of textual entailment. First, I present a robust and wide coverage approach to lexico-structural verb paraphrase recognition based on Levin\u27s (1993) classification of English verbs. Then, I show that by extending Levin\u27s framework to general inference patterns, a classification of English adjectives can be obtained that compared with previous approaches, provides a more fine grained semantic characterisation of their inferential properties. Further, I develop a compositional semantic framework to assign a semantic representation to adjectives based on an ontologically promiscuous approach (Hobbs, 1985) and thereby supporting first order inference for all types of adjectives including extensional ones. Finally, I present a test suite for adjectival inference I developed as a resource for the evaluation of computational systems handling natural language inference.In der vorliegenden Dissertation habe ich untersucht, wie lexikalische Ressourcen, die auf der Gliederung lexikalischen Wissens in Klassen mit gemeinsamen Eigenschaften (lexikalische, semantische etc,) basieren, die computergestützte Verarbeitung natürlicher Sprache und insbesondere die symbolische Erkennung von Entailment unterstützen. Basierend auf Levins (1993) Klassifikation englischer Verben, wurde zuerst ein robuster, für die Verarbeitung beliebiger Texte geeigneter Ansatz zur Paraphrasenerkennung vorgestellt. Dann habe ich aufgezeigt, dass man durch eine Erweiterung von Levins Systematik zur Behandlung allgemeiner Inferenzmuster, eine Klassifikation von englischen Adjektiven erhält, die verglichen mit früheren Ansätzen, eine feinkörnige semantische Charakterisierung ihrer inferentiellen Eigenschaften gestattet und so die Basis für die computergestützte Behandlung von Inferenz bei Adjektiven bildet. Ein anderes beachtliches Ergebnis der vorliegenden Arbeit ist die Test Suite, die ich entwickelt habe und die als Ressource für NPL Anwendungen, die Inferenzen (insbesondere Inferenzen bei Adjektiven) behandeln, genutzt werden kann. Durch die Konstruktion dieser Test Suite beabsichtige ich, den Weg für die Schaffung von Ressourcen zu ebnen, die einen tieferen Einblick in die für Inferenz verantwortlichen Phänomene ermöglichen

Sentiment polarity shifters : creating lexical resources through manual annotation and bootstrapped machine learning

Alleviating pain is good and abandoning hope is bad. We instinctively understand how words like "alleviate" and "abandon" affect the polarity of a phrase, inverting or weakening it. When these words are content words, such as verbs, nouns and adjectives, we refer to them as polarity shifters. Shifters are a frequent occurrence in human language and an important part of successfully modeling negation in sentiment analysis; yet research on negation modeling has focussed almost exclusively on a small handful of closed class negation words, such as "not", "no" and "without. A major reason for this is that shifters are far more lexically diverse than negation words, but no resources exist to help identify them. We seek to remedy this lack of shifter resources. Our most central step towards this is the creation of a large lexicon of polarity shifters that covers verbs, nouns and adjectives. To reduce the prohibitive cost of such a large annotation task, we develop a bootstrapping approach that combines automatic classification with human verification. This ensures the high quality of our lexicon while reducing annotation cost by over 70%. In designing the bootstrap classifier we develop a variety of features which use both existing semantic resources and linguistically informed text patterns. In addition we investigate how knowledge about polarity shifters might be shared across different parts of speech, highlighting both the potential and limitations of such an approach. The applicability of our bootstrapping approach extends beyond the creation of a single resource. We show how it can further be used to introduce polarity shifter resources for other languages. Through the example case of German we show that all our features are transferable to other languages. Keeping in mind the requirements of under-resourced languages, we also explore how well a classifier would do when relying only on data- but not resource-driven features. We also introduce ways to use cross-lingual information, leveraging the shifter resources we previously created for other languages. Apart from the general question of which words can be polarity shifters, we also explore a number of other factors. One of these is the matter of shifting directions, which indicates whether a shifter affects positive polarities, negative polarities or whether it can shift in either direction. Using a supervised classifier we add shifting direction information to our bootstrapped lexicon. For other aspects of polarity shifting, manual annotation is preferable to automatic classification. Not every word that can cause polarity shifting does so for every of its word senses. As word sense disambiguation technology is not robust enough to allow the automatic handling of such nuances, we manually create a complete sense-level annotation of verbal polarity shifters. To verify the usefulness of the lexica which we create, we provide an extrinsic evaluation in which we apply them to a sentiment analysis task. In this task the different lexica are not only compared amongst each other, but also against a state-of-the-art compositional polarity neural network classifier that has been shown to be able to implicitly learn the negating effect of negation words from a training corpus. However, we find that the same is not true for the far more lexically diverse polarity shifters. Instead, the use of the explicit knowledge provided by our shifter lexica brings clear gains in performance.Deutsche Forschungsgesellschaf

On past participle agreement in transitive clauses in French

This paper provides a Minimalist analysis of past participle agreement in French in transitive clauses. Our account posits that the head v of vP in such structures carries an (accusativeassigning) structural case feature which may apply (with or without concomitant agreement) to case-mark a clause-mate object, the subject of a defective complement clause, or an intermediate copy of a preposed subject in spec-CP. In structures where a goal is extracted from vP (e.g. via wh-movement) v also carries an edge feature, and may also carry a specificity feature and a set of (number and gender) agreement features. We show how these assumptions account for agreement of a participle with a preposed specific clausemate object or defective-clause subject, and for the absence of agreement with an embedded object, with the complement of an impersonal verb, and with the subject of an embedded (finite or nonfinite) CP complement. We also argue that the absence of agreement marking (in expected contexts) on the participles faitmade and laissélet in infinitive structures is essentially viral in nature. Finally, we claim that obligatory participle agreement with reflexive and reciprocal objects arises because the derivation of reflexives involves A-movement and concomitant agreement

Approximation in Morphology

This Special Issue "Approximation in Morphology" has been collated from peer-reviewed papers presented at the ApproxiMo 'discontinuous' workshop (2022), which was held online between December 2021 and May 2022, and organized by Francesca Masini (Bologna), Muriel Norde (Berlin) and Kristel Van Goethem (Louvain)

Let us have articles betwixt us: Papers in Historical and Comparative Linguistics in Honour of Johanna L. Wood

Papers in Historical and Comparative Linguistics in Honour of Johanna L. Woo