Steedman, Mark

Weber, Sabine

English

This paper presents ongoing work on the construction and alignment of predicate entailment graphs in English and German. We extract predicate-argument pairs from large corpora of monolingual English and German news text and construct monolingual paraphrase clusters and entailment graphs. We use an aligned subset of entities to derive the bilingual alignment of entities and relations, and achieve better than baseline results on a translated subset of a predicate entailment data set (Levy and Dagan, 2016) and the German portion of XNLI (Conneau et al., 2018)

Edinburgh Research Explorer

     Edinburgh Research Explorer                                      Construction and Alignment of Multilingual Entailment Graphsfor Semantic InferenceCitation for published version:Weber, S & Steedman, M 2019, Construction and Alignment of Multilingual Entailment Graphs for SemanticInference. in Proceedings of the 2019 Workshop on Widening NLP. Association for ComputationalLinguistics, Florence, Italy, pp. 77-79, Widening Natural Language Processing Workshop 2019, Florence,Italy, 28/07/19.Link:Link to publication record in Edinburgh Research ExplorerDocument Version:Peer reviewed versionPublished In:Proceedings of the 2019 Workshop on Widening NLPGeneral rightsCopyright for the publications made accessible via the Edinburgh Research Explorer is retained by the author(s)and / or other copyright owners and it is a condition of accessing these publications that users recognise andabide by the legal requirements associated with these rights.Take down policyThe University of Edinburgh has made every reasonable effort to ensure that Edinburgh Research Explorercontent complies with UK legislation. If you believe that the public display of this file breaches copyright pleasecontact openaccess@ed.ac.uk providing details, and we will remove access to the work immediately andinvestigate your claim.Download date: 11. May. 2020Construction and Alignment of Multilingual Entailment Graphs forSemantic InferenceSabine WeberUniversity of Edinburghs.weber@sms.ed.ac.ukMark SteedmanUniversity of Edinburghsteedman@inf.ed.ac.ukAbstractThis paper presents ongoing work on the construction and alignment of predicate entailmentgraphs in English and German. We extract predicate-argument pairs from large corpora of mono-lingual English and German news text and construct monolingual paraphrase clusters and entail-ment graphs. We use an aligned subset of entities to derive the bilingual alignment of entities andrelations, and achieve better than baseline results on a translated subset of a predicate entailmentdata set (Levy and Dagan, 2016) and the German portion of XNLI (Conneau et al., 2018).1 IntroductionSemantic inference is necessary whenever a target meaning is to be inferred from a text. Entailment andparaphrasing are of key importance in question answering and semantic parsing. The question “Whoowns Yahoo?” might be answered by a sentence like “Verizon acquired Yahoo in 2016.”,which does notcorrespond with the question but rather entails or paraphrases the answer. A form-independent semanticrepresentation for natural language is needed to bridge this gap.A new layer of complexity is added to this problem when considering a multilingual case, where aquestion in one language can only be answered from text in a different language. An alternative totranslating either questions or answers is parsing both questions and answer candidates into a form- andlanguage-independent semantic representation that allows for semantic inference.Previous approaches (Berant et al., 2015) (Hosseini et al., 2018) have concentrated on learning predi-cate entailment relations (e.g. buying entails owning) from large amounts of monolingual text, creatingpredicate entailment graphs to enhance form dependent semantics. In this meaning representation eachpredicate is represented by a cluster containing all its paraphrases and entailments.In this paper we create paraphrase clusters and entailment graphs in both English and German froma large corpus of non-parallel news text. We align them using link prediction methods for knowledgegraphs and present further ideas how alignment and entailment can inform each other. We then addressmethods for testing the aligned graphs.2 Related Work(Lewis and Steedman, 2013a) approach the problem of paraphrase in a multilingual context by creatingaligned paraphrase clusters. They use Wikipedia articles describing the same topic, which can be consid-ered a form of parallel text. Even though there is no perfect alignment between sentences, the fact thatthe same topics are covered is helpful for the construction of paraphrase clusters. Lewis and Steedmanuse the inter-language links between named entities provided in the Wikipedia articles to align predicatesin different languages. Similarly they use the types that were provided with the Wikipedia links.Lewis and Steedman could work this way with perfect alignment of named entities to derive thealignment of predicates. Working with no parallel data poses new challenges. Because the linkingof German named entities to an external data base like Freebase (Bollacker et al., 2008) or DBPedia(Lehmann et al., 2015) is slow and incomplete, only a partial overlap between named entities can beachieved (only about 20% of all named entities in the corpus can be linked to DBPedia). Where typescan not be derived from DBPedia links other methods have to be employed.Paraphrase clusters can be seen as a precursor to entailment graphs. Due to the use of cosine similarityas a metric to construct the clusters, they contain both entailments and paraphrases. Typed predicateentailment graphs were first introduced by (Berant et al., 2015), but were not scalable to a large amountof data. (Hosseini et al., 2018) apply a scalable method that learns globally consistent similarity scores.3 MethodThe construction of entailment graphs can be divided into three steps: (1) extraction, (2) linking andtyping, and (3) the calculation of similarities and construction of paraphrase clusters or entailment graphs.Those steps are completed monolingually for German and English. For English we use the pipelineintroduced by (Hosseini et al., 2018).For German we first use dependency parsing and named entity recognition to extract triples of bi-nary relations (e.g. the triple “besuchen::Angela Merkel::Staatsoper” (visit::Angela Merkel::State OperaHouse)). The linking and typing step poses difficulties for German because the use of state-of-the-artnamed entity linkers is not feasible for our data size. Instead, we use string matches with DBPedia URIs.Typing is performed using the the Stanford Named Entity recognition component (Finkel et al., 2005)for named entities and GermaNet (Hamp and Feldweg, 1997) for general entities. We create paraphraseclusters using the method introduced by (Lewis and Steedman, 2013b) and construct entailment graphsusing the method introduced by (Hosseini et al., 2018).While Lewis and Steedman used the linked named entities of different languages to align predicates,we only have a partial linking. One way to approach this problem is to model the triples of two entitiesand their respective relation as triple in a knowledge graph. We use a link prediction method introducedby (Trouillon et al., 2016). We employ the partial linking of entities to get an alignment of both enti-ties and relations. This link prediction method creates an embedding of entities and relations. Insteadof embedding a monolingual knowledge graph we pass all German and English relation triples to themethod. Entities that are linked are only represented once. This way the learning algorithm sees linkedentities with both German and English relations and therefore will embed them close to each other invector space.4 Results and DiscussionEvaluating the alignment of predicate entailment graphs poses a challenge. Existing work has not yetexplored multilinguality. One approach is to evaluate entailment and alignment separately.There is still no sufficiently large predicate entailment data set for German. We run preliminary ex-periments using paraphrase clusters on the German portion of the XNLI corpus (Conneau et al., 2018),which is only of limited use because most of its entailments are not predicate entailments. Neverthelessour method performs 2.7% over random. Hosseini et al. use the predicate entailment data set introducedin (Levy and Dagan, 2016). We translate 1946 sentence pairs from it and perform 5.6% over random.One way to assess the alignment is to use a data set for bilingual dictionary alignment like the MUSEdata set (Conneau et al., 2017) although verbs only make up a small portion of it. Also the relationsextracted by us contain verb modifier constructions such as “try to reach” or light verb constructions like“have a border with”, which are not represented in such data sets.Preliminary qualitative results appear promising. In the intermediate step of paraphrase clustering, wesee paraphrase clusters like “formulieren, schreiben” (formulate, write) and “darstellen, zeigen” (repre-sent, show). For the full entailment results, we get e.g. “warten zu kommen” and “hoffen zu gelangen”entails “versuchen zu gelangen” (waiting to come, hoping to reach, trying to reach).This paper presented ongoing work on construction and alignment of predicate entailment graphs fromdifferent languages without parallel text. Aligned entailment graphs are a preliminary step to a form- andlanguage-independent semantic that can be used for various natural language understanding task like forexample question answering from multilingual text. As a next step we plan to incorporate the entailmentlinks between predicates to support the alignment. Preliminary tests show that the incorporation of en-tailment links for link prediction in knowledge graphs leads to significant improvement in performance.Translating the full data sets or creating a similar sets for German is another avenue for future work.ReferencesJonathan Berant, Noga Alon, Ido Dagan, and Jacob Goldberger. 2015. Efficient global learning of entailmentgraphs. Computational Linguistics, 41(2):249–291.Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: a collaborativelycreated graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD interna-tional conference on Management of data, pages 1247–1250. AcM.Alexis Conneau, Guillaume Lample, Marc’Aurelio Ranzato, Ludovic Denoyer, and Herve´ Je´gou. 2017. Wordtranslation without parallel data. arXiv preprint arXiv:1710.04087.Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and VeselinStoyanov. 2018. Xnli: Evaluating cross-lingual sentence representations. In Proceedings of the 2018 Confer-ence on Empirical Methods in Natural Language Processing, pages 2475–2485.Jenny Rose Finkel, Trond Grenager, and Christopher Manning. 2005. Incorporating non-local information intoinformation extraction systems by gibbs sampling. In Proceedings of the 43rd annual meeting on associationfor computational linguistics, pages 363–370. Association for Computational Linguistics.Birgit Hamp and Helmut Feldweg. 1997. Germanet-a lexical-semantic net for german. Automatic informationextraction and building of lexical semantic resources for NLP applications.Mohammad Javad Hosseini, Nathanael Chambers, Siva Reddy, Xavier R Holt, Shay B Cohen, Mark Johnson, andMark Steedman. 2018. Learning typed entailment graphs with global soft constraints. Transactions of theAssociation for Computational Linguistics, 6:703–717.Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch, Dimitris Kontokostas, Pablo N Mendes, Sebastian Hell-mann, Mohamed Morsey, Patrick Van Kleef, So¨ren Auer, et al. 2015. Dbpedia–a large-scale, multilingualknowledge base extracted from wikipedia. Semantic Web, 6(2):167–195.Omer Levy and Ido Dagan. 2016. Annotating relation inference in context via question answering. In Proceedingsof the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages249–255, Berlin, Germany, August. Association for Computational Linguistics.Mike Lewis and Mark Steedman. 2013a. Combined distributional and logical semantics. Transactions of theAssociation for Computational Linguistics, 1:179–192.Mike Lewis and Mark Steedman. 2013b. Unsupervised induction of cross-lingual semantic relations. In Proceed-ings of the 2013 conference on empirical methods in natural language processing, pages 681–692.The´o Trouillon, Johannes Welbl, Sebastian Riedel, E´ric Gaussier, and Guillaume Bouchard. 2016. Complexembeddings for simple link prediction. In International Conference on Machine Learning, pages 2071–2080.

Construction and Alignment of Multilingual Entailment Graphs for Semantic Inference

https://www.pure.ed.ac.uk/ws/files/138841474/Construction_and_Alignmen_WEBER_DOA28052019_AFV.pdf

Construction and Alignment of Multilingual Entailment Graphs for Semantic Inference

Abstract

Similar works

Full text

Available Versions

Edinburgh Research Explorer