24,590 research outputs found
Dual Co-Matching Network for Multi-choice Reading Comprehension
Multi-choice reading comprehension is a challenging task that requires
complex reasoning procedure. Given passage and question, a correct answer need
to be selected from a set of candidate answers. In this paper, we propose
\textbf{D}ual \textbf{C}o-\textbf{M}atching \textbf{N}etwork (\textbf{DCMN})
which model the relationship among passage, question and answer
bidirectionally. Different from existing approaches which only calculate
question-aware or option-aware passage representation, we calculate
passage-aware question representation and passage-aware answer representation
at the same time. To demonstrate the effectiveness of our model, we evaluate
our model on a large-scale multiple choice machine reading comprehension
dataset (i.e. RACE). Experimental result show that our proposed model achieves
new state-of-the-art results.Comment: arXiv admin note: text overlap with arXiv:1806.04068 by other author
Retrospective Reader for Machine Reading Comprehension
Machine reading comprehension (MRC) is an AI challenge that requires machine
to determine the correct answers to questions based on a given passage. MRC
systems must not only answer question when necessary but also distinguish when
no answer is available according to the given passage and then tactfully
abstain from answering. When unanswerable questions are involved in the MRC
task, an essential verification module called verifier is especially required
in addition to the encoder, though the latest practice on MRC modeling still
most benefits from adopting well pre-trained language models as the encoder
block by only focusing on the "reading". This paper devotes itself to exploring
better verifier design for the MRC task with unanswerable questions. Inspired
by how humans solve reading comprehension questions, we proposed a
retrospective reader (Retro-Reader) that integrates two stages of reading and
verification strategies: 1) sketchy reading that briefly investigates the
overall interactions of passage and question, and yield an initial judgment; 2)
intensive reading that verifies the answer and gives the final prediction. The
proposed reader is evaluated on two benchmark MRC challenge datasets SQuAD2.0
and NewsQA, achieving new state-of-the-art results. Significance tests show
that our model is significantly better than the strong ELECTRA and ALBERT
baselines. A series of analysis is also conducted to interpret the
effectiveness of the proposed reader.Comment: Accepted by AAAI 202
Retrieve-and-Read: Multi-task Learning of Information Retrieval and Reading Comprehension
This study considers the task of machine reading at scale (MRS) wherein,
given a question, a system first performs the information retrieval (IR) task
of finding relevant passages in a knowledge source and then carries out the
reading comprehension (RC) task of extracting an answer span from the passages.
Previous MRS studies, in which the IR component was trained without considering
answer spans, struggled to accurately find a small number of relevant passages
from a large set of passages. In this paper, we propose a simple and effective
approach that incorporates the IR and RC tasks by using supervised multi-task
learning in order that the IR component can be trained by considering answer
spans. Experimental results on the standard benchmark, answering SQuAD
questions using the full Wikipedia as the knowledge source, showed that our
model achieved state-of-the-art performance. Moreover, we thoroughly evaluated
the individual contributions of our model components with our new Japanese
dataset and SQuAD. The results showed significant improvements in the IR task
and provided a new perspective on IR for RC: it is effective to teach which
part of the passage answers the question rather than to give only a relevance
score to the whole passage.Comment: 10 pages, 6 figure. Accepted as a full paper at CIKM 201
SG-Net: Syntax-Guided Machine Reading Comprehension
For machine reading comprehension, the capacity of effectively modeling the
linguistic knowledge from the detail-riddled and lengthy passages and getting
ride of the noises is essential to improve its performance. Traditional
attentive models attend to all words without explicit constraint, which results
in inaccurate concentration on some dispensable words. In this work, we propose
using syntax to guide the text modeling by incorporating explicit syntactic
constraints into attention mechanism for better linguistically motivated word
representations. In detail, for self-attention network (SAN) sponsored
Transformer-based encoder, we introduce syntactic dependency of interest (SDOI)
design into the SAN to form an SDOI-SAN with syntax-guided self-attention.
Syntax-guided network (SG-Net) is then composed of this extra SDOI-SAN and the
SAN from the original Transformer encoder through a dual contextual
architecture for better linguistics inspired representation. To verify its
effectiveness, the proposed SG-Net is applied to typical pre-trained language
model BERT which is right based on a Transformer encoder. Extensive experiments
on popular benchmarks including SQuAD 2.0 and RACE show that the proposed
SG-Net design helps achieve substantial performance improvement over strong
baselines.Comment: Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-2020
- …