917 research outputs found
LangPro: Natural Language Theorem Prover
LangPro is an automated theorem prover for natural language
(https://github.com/kovvalsky/LangPro). Given a set of premises and a
hypothesis, it is able to prove semantic relations between them. The prover is
based on a version of analytic tableau method specially designed for natural
logic. The proof procedure operates on logical forms that preserve linguistic
expressions to a large extent. %This property makes the logical forms easily
obtainable from syntactic trees. %, in particular, Combinatory Categorial
Grammar derivation trees. The nature of proofs is deductive and transparent. On
the FraCaS and SICK textual entailment datasets, the prover achieves high
results comparable to state-of-the-art.Comment: 6 pages, 8 figures, Conference on Empirical Methods in Natural
Language Processing (EMNLP) 201
Breaking NLI Systems with Sentences that Require Simple Lexical Inferences
We create a new NLI test set that shows the deficiency of state-of-the-art
models in inferences that require lexical and world knowledge. The new examples
are simpler than the SNLI test set, containing sentences that differ by at most
one word from sentences in the training set. Yet, the performance on the new
test set is substantially worse across systems trained on SNLI, demonstrating
that these systems are limited in their generalization ability, failing to
capture many simple inferences.Comment: 6 pages, short paper at ACL 201
- …