5 research outputs found
Stochastic Answer Networks for SQuAD 2.0
This paper presents an extension of the Stochastic Answer Network (SAN), one
of the state-of-the-art machine reading comprehension models, to be able to
judge whether a question is unanswerable or not. The extended SAN contains two
components: a span detector and a binary classifier for judging whether the
question is unanswerable, and both components are jointly optimized.
Experiments show that SAN achieves the results competitive to the
state-of-the-art on Stanford Question Answering Dataset (SQuAD) 2.0. To
facilitate the research on this field, we release our code:
https://github.com/kevinduh/san_mrc.Comment: 6 pages, 2 figures and 2 table
Fast Prototyping a Dialogue Comprehension System for Nurse-Patient Conversations on Symptom Monitoring
Data for human-human spoken dialogues for research and development are
currently very limited in quantity, variety, and sources; such data are even
scarcer in healthcare. In this work, we investigate fast prototyping of a
dialogue comprehension system by leveraging on minimal nurse-to-patient
conversations. We propose a framework inspired by nurse-initiated clinical
symptom monitoring conversations to construct a simulated human-human dialogue
dataset, embodying linguistic characteristics of spoken interactions like
thinking aloud, self-contradiction, and topic drift. We then adopt an
established bidirectional attention pointer network on this simulated dataset,
achieving more than 80% F1 score on a held-out test set from real-world
nurse-to-patient conversations. The ability to automatically comprehend
conversations in the healthcare domain by exploiting only limited data has
implications for improving clinical workflows through red flag symptom
detection and triaging capabilities. We demonstrate the feasibility for
efficient and effective extraction, retrieval and comprehension of symptom
checking information discussed in multi-turn human-human spoken conversations.Comment: 8 pages. To appear in NAACL 201
EQuANt (Enhanced Question Answer Network)
Machine Reading Comprehension (MRC) is an important topic in the domain of
automated question answering and in natural language processing more generally.
Since the release of the SQuAD 1.1 and SQuAD 2 datasets, progress in the field
has been particularly significant, with current state-of-the-art models now
exhibiting near-human performance at both answering well-posed questions and
detecting questions which are unanswerable given a corresponding context. In
this work, we present Enhanced Question Answer Network (EQuANt), an MRC model
which extends the successful QANet architecture of Yu et al. to cope with
unanswerable questions. By training and evaluating EQuANt on SQuAD 2, we show
that it is indeed possible to extend QANet to the unanswerable domain. We
achieve results which are close to 2 times better than our chosen baseline
obtained by evaluating a lightweight version of the original QANet architecture
on SQuAD 2. In addition, we report that the performance of EQuANt on SQuAD 1.1
after being trained on SQuAD2 exceeds that of our lightweight QANet
architecture trained and evaluated on SQuAD 1.1, demonstrating the utility of
multi-task learning in the MRC context
Relation Module for Non-answerable Prediction on Question Answering
Machine reading comprehension(MRC) has attracted significant amounts of
research attention recently, due to an increase of challenging reading
comprehension datasets. In this paper, we aim to improve a MRC model's ability
to determine whether a question has an answer in a given context (e.g. the
recently proposed SQuAD 2.0 task). Our solution is a relation module that is
adaptable to any MRC model. The relation module consists of both semantic
extraction and relational information. We first extract high level semantics as
objects from both question and context with multi-head self-attentive pooling.
These semantic objects are then passed to a relation network, which generates
relationship scores for each object pair in a sentence. These scores are used
to determine whether a question is non-answerable. We test the relation module
on the SQuAD 2.0 dataset using both BiDAF and BERT models as baseline readers.
We obtain 1.8% gain of F1 on top of the BiDAF reader, and 1.0% on top of the
BERT base model. These results show the effectiveness of our relation module on
MR
Improving Machine Reading Comprehension via Adversarial Training
Adversarial training (AT) as a regularization method has proved its
effectiveness in various tasks, such as image classification and text
classification. Though there are successful applications of AT in many tasks of
natural language processing (NLP), the mechanism behind it is still unclear. In
this paper, we aim to apply AT on machine reading comprehension (MRC) and study
its effects from multiple perspectives. We experiment with three different
kinds of RC tasks: span-based RC, span-based RC with unanswerable questions and
multi-choice RC. The experimental results show that the proposed method can
improve the performance significantly and universally on SQuAD1.1, SQuAD2.0 and
RACE. With virtual adversarial training (VAT), we explore the possibility of
improving the RC models with semi-supervised learning and prove that examples
from a different task are also beneficial. We also find that AT helps little in
defending against artificial adversarial examples, but AT helps the model to
learn better on examples that contain more low-frequency words