38 research outputs found
Stochastic Answer Networks for Machine Reading Comprehension
We propose a simple yet robust stochastic answer network (SAN) that simulates
multi-step reasoning in machine reading comprehension. Compared to previous
work such as ReasoNet which used reinforcement learning to determine the number
of steps, the unique feature is the use of a kind of stochastic prediction
dropout on the answer module (final layer) of the neural network during the
training. We show that this simple trick improves robustness and achieves
results competitive to the state-of-the-art on the Stanford Question Answering
Dataset (SQuAD), the Adversarial SQuAD, and the Microsoft MAchine Reading
COmprehension Dataset (MS MARCO).Comment: 11 pages, 5 figures, Accepted to ACL 201
A Fully Attention-Based Information Retriever
Recurrent neural networks are now the state-of-the-art in natural language
processing because they can build rich contextual representations and process
texts of arbitrary length. However, recent developments on attention mechanisms
have equipped feedforward networks with similar capabilities, hence enabling
faster computations due to the increase in the number of operations that can be
parallelized. We explore this new type of architecture in the domain of
question-answering and propose a novel approach that we call Fully Attention
Based Information Retriever (FABIR). We show that FABIR achieves competitive
results in the Stanford Question Answering Dataset (SQuAD) while having fewer
parameters and being faster at both learning and inference than rival methods.Comment: Accepted for presentation at the International Joint Conference on
Neural Networks (IJCNN) 201
Do Multi-hop Readers Dream of Reasoning Chains?
General Question Answering (QA) systems over texts require the multi-hop
reasoning capability, i.e. the ability to reason with information collected
from multiple passages to derive the answer. In this paper we conduct a
systematic analysis to assess such an ability of various existing models
proposed for multi-hop QA tasks. Specifically, our analysis investigates that
whether providing the full reasoning chain of multiple passages, instead of
just one final passage where the answer appears, could improve the performance
of the existing QA models. Surprisingly, when using the additional evidence
passages, the improvements of all the existing multi-hop reading approaches are
rather limited, with the highest error reduction of 5.8% on F1 (corresponding
to 1.3% absolute improvement) from the BERT model.
To better understand whether the reasoning chains could indeed help find
correct answers, we further develop a co-matching-based method that leads to
13.1% error reduction with passage chains when applied to two of our base
readers (including BERT). Our results demonstrate the existence of the
potential improvement using explicit multi-hop reasoning and the necessity to
develop models with better reasoning abilities.Comment: Accepted by MRQA Workshop 201