Towards Semantic Validation of a Derivational Lexicon

Abstract Derivationally related lemmas like friend N -friendly A -friendship N are derived from a common stem. Frequently, their meanings are also systematically related. However, there are also many examples of derivationally related lemma pairs whose meanings differ substantially, e.g., object N -objective N . Most broad-coverage derivational lexicons do not reflect this distinction, mixing up semantically related and unrelated word pairs. In this paper, we investigate strategies to recover the above distinction by recognizing semantically related lemma pairs, a process we call semantic validation. We make two main contributions: First, we perform a detailed data analysis on the basis of a large German derivational lexicon. It reveals two promising sources of information (distributional semantics and structural information about derivational rules), but also systematic problems with these sources. Second, we develop a classification model for the task that reflects the noisy nature of the data. It achieves an improvement of 13.6% in precision and 5.8% in F1-score over a strong majority class baseline. Our experiments confirm that both information sources contribute to semantic validation, and that they are complementary enough that the best results are obtained from a combined model

A Proof-Theoretic Approach to Scope Ambiguity in Compositional Vector Space Models

We investigate the extent to which compositional vector space models can be used to account for scope ambiguity in quantified sentences (of the form "Every man loves some woman"). Such sentences containing two quantifiers introduce two readings, a direct scope reading and an inverse scope reading. This ambiguity has been treated in a vector space model using bialgebras by (Hedges and Sadrzadeh, 2016) and (Sadrzadeh, 2016), though without an explanation of the mechanism by which the ambiguity arises. We combine a polarised focussed sequent calculus for the non-associative Lambek calculus NL, as described in (Moortgat and Moot, 2011), with the vector based approach to quantifier scope ambiguity. In particular, we establish a procedure for obtaining a vector space model for quantifier scope ambiguity in a derivational way.Comment: This is a preprint of a paper to appear in: Journal of Language Modelling, 201

A Graph Auto-encoder Model of Derivational Morphology

There has been little work on modeling the morphological well-formedness (MWF) of derivatives, a problem judged to be complex and difficult in linguistics (Bauer, 2019). We present a graph auto-encoder that learns em- beddings capturing information about the com- patibility of affixes and stems in derivation. The auto-encoder models MWF in English sur- prisingly well by combining syntactic and se- mantic information with associative informa- tion from the mental lexicon

Superbizarre Is Not Superb: Derivational Morphology Improves BERT’s Interpretation of Complex Words

How does the input segmentation of pretrained language models (PLMs) affect their interpretations of complex words? We present the first study investigating this question, taking BERT as the example PLM and focusing on its semantic representations of English derivatives. We show that PLMs can be interpreted as serial dual-route models, i.e., the meanings of complex words are either stored or else need to be computed from the subwords, which implies that maximally meaningful input tokens should allow for the best generalization on new words. This hypothesis is confirmed by a series of semantic probing tasks on which DelBERT (Derivation leveraging BERT), a model with derivational input segmentation, substantially outperforms BERT with WordPiece segmentation. Our results suggest that the generalization capabilities of PLMs could be further improved if a morphologically-informed vocabulary of input tokens were used

Vengriški kokybės užtikrinimo srities terminai naujadarai

By employing a functional-cognitive frame, this paper, in which neologisms derived from English are analysed, focuses on the semantics of new Hungarian terms of quality assurance (quality management in general). Although the importance of unambiguous terms in scientific communication is often emphasised (Temmerman 2002: 211), it has been observed that the presence of conceptual metonymies and metaphors (Lakoff and Johnson 1980, Panther and Thornburg 2003, Kövecses 2015) also fosters understanding of technical languages. The author’s previous research in the field of the semantics of Hungarian neologisms (e.g. Sólyom 2014a, b, 2016) has also revealed that the presence of metonymies and metaphors has a significant impact upon the process of meaning construal. The present research assumes that various metonymic and metaphorical meanings occur in the semantics of novel Hungarian terms of quality assurance. To attest this, examples from a questionnaire filled by Hungarian quality engineers in 2018 will be analysed. Another question addressed in this paper is whether there is a mental reason for the fact that although there are colloquial Hungarian words and expressions for describing the processes of manufacturing, experts in the field do not use them, but rather employ neologisms with English roots. Indeed, this is how specialists distinguish technical terms from everyday expressions.Straipsnyje funkciniu ir kognityviniu aspektu nagrinėjami nauji vengriški kokybės užtikrinimo srities terminai, naujadarai analizuojami atsižvelgiant į jų gramatines ir semantines savybes. Medžiaga rinkta 2018 metais, šios srities ekspertai (kokybės inžinieriai) užpildė klausimyną. Naujus terminus vienija tai, kad jie kurti naudojant angliškų žodžių šaknis, o iš kelių žodžių sudarytuose terminuose esama ir angliškų žodžių.Nors mokslinėje literatūroje dažnai pabrėžiama vienareikšmių terminų svarba mokslinėje komunikacijoje (Temmerman 2002: 211), šiandien jau aišku, kad konceptualiosios metaforos ir metonimai (Lakoff-Johnson 1980, Panther-Thornburg 2003, Kövecses 2015) gali turėti svarbų vaidmenį kuriant žodžių reikšmes, jie taip pat palengvina terminijos supratimo procesą. Straipsnio autorės tyrimai atskleidė, kad metaforų ir metonimų vartojimas gali turėti reikšmingą poveikį reikšmių kūrimo procesui sudarant kasdien vartojamus naujadarus, kai kalboje pradedama vartoti naują žodį arba naują konstrukciją.Šiame straipsnyje teigiama, kad tai būdinga ir naujiems vengriškiems kokybės užtikrinimo srities terminams, todėl šiuose terminuose taip pat galima rasti įvairių metoniminių ir metaforinių semantinių struktūrų. Ypatingas dėmesys skiriamas gramatinių procesų semantikai, t. y. žodžių darybai, nes didelę dalį šių terminų sudaro tokie vengriški veiksmažodžiai, kurie yra sudaryti prie angliškos šaknies prijungus produktyvias vengrų kalbos darybos morfemas -(V)z arba -(V)l. Kitas svarbus dalykas kuriant šiuos terminus yra vengriškų priešdėlių semantika, nes jie suteikia naujiems veiksmažodžiams perfektinę reikšmę, pabrėžia veiksmo užbaigtumą.Antroje straipsnio dalyje iškeliama hipotezė, kad esama ir mentalinių priežasčių, kodėl, nepaisant to, kad gamybos procesus galima aprašyti kasdieniais (šnekamosios) vengrų kalbos žodžiais ir žodžių junginiais, nagrinėjamosios srities ekspertai, užuot vartoję juos, pirmenybę teikia naujadarams su angliškomis šaknimis. Šiai hipotezei pagrįsti straipsnyje aptariami informantų atsakymai apie nagrinėjamų terminų vartojimą. Analizė atskleidė, kad vartodami naujadarus kokybės užtikrinimo srities specialistai gali atskirti specialiuosius terminus nuo bendrų, kasdienių leksemų

Induction, Semantic Validation and Evaluation of a Derivational Morphology Lexicon for German

This thesis is about computational morphology for German derivation. Derivation is a word formation process that creates new words from existing ones, where the base and the derived word share the same stem. Mostly, derivation is conducted by means of relatively regular affixation rules, as in to bake - bakery. In German, derivation is highly productive, thus leading to a high language variability which can be employed to express similar facts in different ways, as derivationally related words are often also semantically related (or transparent). However, linguistic variance is a challenge for computational applications, particularly in semantic processing: It makes it more difficult to automatically grasp the meaning of texts and to match similar information onto each other. Thus, computational systems require linguistic knowledge. We develop methods to induce and represent derivational knowledge, and to apply it in language processing. The main outcome of our study is DErivBase, a German derivational lexicon. It groups derivationally related words (words that are derived from the same stem) into derivational families. To achieve high quality and high coverage, we induce DErivBase by combining rule-based and data-driven methods: We implement linguistic derivation rules to define derivational processes, and feed lemmas extracted from a German corpus into the rules to derive new lemmas. All words that are connected - directly or indirectly - by such rules are considered a derivational family. As mentioned above, a derivational relationship often implies semantic relationship, but this is not always the case. Semantic drifts can cause semantically unrelated (opaque) derivational relations, such as to depart - department. Capturing the difference between transparent and opaque relations is important from a linguistic as well as a practical point of view. Thus, we conduct a semantic refinement of DErivBase, i.e., we determine which lemma pairs are derivationally and semantically related, and which are not. We establish a second, semantically validated version of our lexicon, where families are sub-clustered according to semantic coherence, using supervised machine learning methods: We learn a binary classifier based on features that arise from structural information about the derivation rules, and from distributional information about the semantic relatedness of lemmas. Accordingly, the derivational families are subdivided into semantically coherent clusters. To demonstrate the utility of the two lexicon versions, we evaluate them on three extrinsic - and in the broadest sense, semantic - tasks. The underlying assumption for applying DErivBase to semantic tasks is that derivational relatedness is a reasonable approximation to semantic relatedness, since derivation is often semantically transparent. Our three experiments are the following: 1., we incorporate DErivBase into distributional semantic models to overcome sparsity problems and to improve the prediction quality of the underlying model. We test this method, which we call derivational smoothing, for semantic similarity prediction, and for synonym choice. 2., we employ DErivBase to model a psycholinguistic experiment that examines priming effects of transparent and opaque derivations to draw conclusions about the mental lexical representation in German. Derivational information is again incorporated into a distributional model, but this time, it introduces a kind of morphological generalisation. 3., in order to solve the task of Recognising Textual Entailment, we integrate DErivBase into a matching-based entailment system by means of a query expansion. Assuming that derivational relationships between two texts suggest them to be entailing rather than non-entailing, this expansion increases the chance of a lexical overlap, which should improve the system's entailment predictions. The incorporation of DErivBase indeed improves the performance of the underlying systems in each task, however, it is differently suitable in different settings. In experiment 1., the semantically validated lexicon yields improvements over the purely morphological lexicon, and the more coarse-grained similarity prediction profits more from DErivBase than the synonym choice. In experiment 2., purely morphological information clearly outperforms the other lexicon version, as the latter cannot model opaque derivations. On the entailment task in experiment 3., DErivBase has only minor impact, because textual entailment is hard to solve by addressing only one linguistic phenomenon. In sum, our findings show that the induction of a high-quality, high-coverage derivational lexicon is beneficial for very different applications in computational linguistics. It might be worthwhile to further investigate the semantic aspects of derivation to better understand its impact on language and thus, on language processing