4 research outputs found
Answering questions by learning to rank -- Learning to rank by answering questions
Answering multiple-choice questions in a setting in which no supporting
documents are explicitly provided continues to stand as a core problem in
natural language processing. The contribution of this article is two-fold.
First, it describes a method which can be used to semantically rank documents
extracted from Wikipedia or similar natural language corpora. Second, we
propose a model employing the semantic ranking that holds the first place in
two of the most popular leaderboards for answering multiple-choice questions:
ARC Easy and Challenge. To achieve this, we introduce a self-attention based
neural network that latently learns to rank documents by their importance
related to a given question, whilst optimizing the objective of predicting the
correct answer. These documents are considered relevant contexts for the
underlying question. We have published the ranked documents so that they can be
used off-the-shelf to improve downstream decision models.Comment: Presented at EMNLP 2019; 10 pages, 5 figure
Knowledge Fusion and Semantic Knowledge Ranking for Open Domain Question Answering
Open Domain Question Answering requires systems to retrieve external
knowledge and perform multi-hop reasoning by composing knowledge spread over
multiple sentences. In the recently introduced open domain question answering
challenge datasets, QASC and OpenBookQA, we need to perform retrieval of facts
and compose facts to correctly answer questions. In our work, we learn a
semantic knowledge ranking model to re-rank knowledge retrieved through Lucene
based information retrieval systems. We further propose a "knowledge fusion
model" which leverages knowledge in BERT-based language models with externally
retrieved knowledge and improves the knowledge understanding of the BERT-based
language models. On both OpenBookQA and QASC datasets, the knowledge fusion
model with semantically re-ranked knowledge outperforms previous attempts.Comment: 9 pages. 4 figures, 4 table
Natural Language QA Approaches using Reasoning with External Knowledge
Question answering (QA) in natural language (NL) has been an important aspect
of AI from its early days. Winograd's ``councilmen'' example in his 1972 paper
and McCarthy's Mr. Hug example of 1976 highlights the role of external
knowledge in NL understanding. While Machine Learning has been the go-to
approach in NL processing as well as NL question answering (NLQA) for the last
30 years, recently there has been an increasingly emphasized thread on NLQA
where external knowledge plays an important role. The challenges inspired by
Winograd's councilmen example, and recent developments such as the Rebooting AI
book, various NLQA datasets, research on knowledge acquisition in the NLQA
context, and their use in various NLQA models have brought the issue of NLQA
using ``reasoning'' with external knowledge to the forefront. In this paper, we
present a survey of the recent work on them. We believe our survey will help
establish a bridge between multiple fields of AI, especially between (a) the
traditional fields of knowledge representation and reasoning and (b) the field
of NL understanding and NLQA.Comment: 6 pages, 3 figures, Work in Progres
Using the Hammer Only on Nails: A Hybrid Method for Evidence Retrieval for Question Answering
Evidence retrieval is a key component of explainable question answering (QA).
We argue that, despite recent progress, transformer network-based approaches
such as universal sentence encoder (USE-QA) do not always outperform
traditional information retrieval (IR) methods such as BM25 for evidence
retrieval for QA. We introduce a lexical probing task that validates this
observation: we demonstrate that neural IR methods have the capacity to capture
lexical differences between questions and answers, but miss obvious lexical
overlap signal. Learning from this probing analysis, we introduce a hybrid
approach for evidence retrieval that combines the advantages of both IR
directions. Our approach uses a routing classifier that learns when to direct
incoming questions to BM25 vs. USE-QA for evidence retrieval using very simple
statistics, which can be efficiently extracted from the top candidate evidence
sentences produced by a BM25 model. We demonstrate that this hybrid evidence
retrieval generally performs better than either individual retrieval strategy
on three QA datasets: OpenBookQA, ReQA SQuAD, and ReQA NQ. Furthermore, we show
that the proposed routing strategy is considerably faster than neural methods,
with a runtime that is up to 5 times faster than USE-QA