15 research outputs found
Temporality and modality in entailment graph induction
The ability to draw inferences is core to semantics and the field of Natural Language
Processing. Answering a seemingly simple question like âDid Arsenal play Manchester
yesterdayâ from textual evidence that says âArsenal won against Manchester yesterdayâ
requires modeling the inference that âwinningâ entails âplayingâ. One way of
modeling this type of lexical semantics is with Entailment Graphs, collections of meaning
postulates that can be learned in an unsupervised way from large text corpora.
In this work, we explore the role that temporality and linguistic modality can play
in inducing Entailment Graphs. We identify inferences that were previously not supported
by Entailment Graphs (such as that âvisitingâ entails an âarrivalâ before the visit)
and inferences that were likely to be learned incorrectly (such as that âwinningâ entails
âlosingâ). Temporality is shown to be useful in alleviating these challenges, in the
Entailment Graph representation as well as the learning algorithm. An exploration of
linguistic modality in the training data shows, counterintuitively, that there is valuable
signal in modalized predications. We develop three datasets for evaluating a systemâs
capability of modeling these inferences, which were previously underrepresented in
entailment rule evaluations. Finally, in support of the work on modality, we release
a relation extraction system that is capable of annotating linguistic modality, together
with a comprehensive modality lexicon
Unsupervised Learning of Relational Entailment Graphs from Text
Recognizing textual entailment and paraphrasing is critical to many core natural language processing applications including question answering and semantic parsing. The surface form of a sentence that answers a question such as âDoes Facebook own Instagram?â frequently does not directly correspond to the form of the question, but is rather a paraphrase or an expression such as âFacebook bought Instagramâ, that entails the answer. Relational entailments (e.g., buys entails owns) are crucial for bridging the gap between queries and text resources. In this thesis, we describe different unsupervised approaches to construct relational entailment graphs, with typed relations (e.g., company buys company) as nodes and entailment as directed edges. The entailment graphs provide an explainable resource for downstream tasks such as question answering; however, the existing methods suffer from noise and sparsity inherent to the data.
We extract predicate-argument structures from large multiple-source news corpora using a fast Combinatory Categorial Grammar parser. We compute entailment scores between relations based on the Distributional Inclusion Hypothesis which states that a word (relation) p entails another word (relation) q if and only if in any context that p can be used, q can be used in its place. The entailment scores are used to build local entailment graphs. We then build global entailment graphs by exploiting the dependencies between the entailment rules. Previous work has used transitivity constraints, but these constraints are intractable on large graphs. We instead propose a scalable method that learns globally consistent similarity scores based on new soft constraints that consider both the structures across typed entailment graphs and inside each graph. We show that our method significantly improves the entailment graphs.
Additionally, we show the duality of entailment graph induction with the task of link prediction. The link prediction task infers missing relations between entities in an incomplete knowledge graph and discovers new facts. We present a new method in which link prediction on the knowledge graph of assertions extracted from raw text is used to improve entailment graphs which are learned from the same text. The entailment graphs are in turn used to improve the link prediction task.
Finally, we define the contextual link prediction task that uses both the structure of the knowledge graph of assertions and their textual contexts. We fine-tune pre-trained language models with an unsupervised contextual link prediction objective. We augment the existing assertions with novel predictions of our model and use them to build higher quality entailment graphs. Similarly, we show that the entailment graphs improve the contextual link prediction task
Smoothing Entailment Graphs with Language Models
The diversity and Zipfian frequency distribution of natural language
predicates in corpora leads to sparsity in Entailment Graphs (EGs) built by
Open Relation Extraction (ORE). EGs are computationally efficient and
explainable models of natural language inference, but as symbolic models, they
fail if a novel premise or hypothesis vertex is missing at test-time. We
present theory and methodology for overcoming such sparsity in symbolic models.
First, we introduce a theory of optimal smoothing of EGs by constructing
transitive chains. We then demonstrate an efficient, open-domain, and
unsupervised smoothing method using an off-the-shelf Language Model to find
approximations of missing premise predicates. This improves recall by 25.1 and
16.3 percentage points on two difficult directional entailment datasets, while
raising average precision and maintaining model explainability. Further, in a
QA task we show that EG smoothing is most useful for answering questions with
lesser supporting text, where missing premise predicates are more costly.
Finally, controlled experiments with WordNet confirm our theory and show that
hypothesis smoothing is difficult, but possible in principle.Comment: Published at AACL 202