4 research outputs found
Fast Reading Comprehension with ConvNets
State-of-the-art deep reading comprehension models are dominated by recurrent
neural nets. Their sequential nature is a natural fit for language, but it also
precludes parallelization within an instances and often becomes the bottleneck
for deploying such models to latency critical scenarios. This is particularly
problematic for longer texts. Here we present a convolutional architecture as
an alternative to these recurrent architectures. Using simple dilated
convolutional units in place of recurrent ones, we achieve results comparable
to the state of the art on two question answering tasks, while at the same time
achieving up to two orders of magnitude speedups for question answering.Comment: 15 pages, 10 figures, submitted to ICLR 201
Robust Text Classifier on Test-Time Budgets
We propose a generic and interpretable learning framework for building robust
text classification model that achieves accuracy comparable to full models
under test-time budget constraints. Our approach learns a selector to identify
words that are relevant to the prediction tasks and passes them to the
classifier for processing. The selector is trained jointly with the classifier
and directly learns to incorporate with the classifier. We further propose a
data aggregation scheme to improve the robustness of the classifier. Our
learning framework is general and can be incorporated with any type of text
classification model. On real-world data, we show that the proposed approach
improves the performance of a given classifier and speeds up the model with a
mere loss in accuracy performance.Comment: To appear at EMNLP-IJCAI 2019, 6 pages + 2 pages appendi
FastFusionNet: New State-of-the-Art for DAWNBench SQuAD
In this technical report, we introduce FastFusionNet, an efficient variant of
FusionNet [12]. FusionNet is a high performing reading comprehension
architecture, which was designed primarily for maximum retrieval accuracy with
less regard towards computational requirements. For FastFusionNets we remove
the expensive CoVe layers [21] and substitute the BiLSTMs with far more
efficient SRU layers [19]. The resulting architecture obtains state-of-the-art
results on DAWNBench [5] while achieving the lowest training and inference time
on SQuAD [25] to-date. The code is available at
https://github.com/felixgwu/FastFusionNet.Comment: A Technical Repor
Integrated Triaging for Fast Reading Comprehension
Although according to several benchmarks automatic machine reading
comprehension (MRC) systems have recently reached super-human performance, less
attention has been paid to their computational efficiency. However, efficiency
is of crucial importance for training and deployment in real world
applications. This paper introduces Integrated Triaging, a framework that
prunes almost all context in early layers of a network, leaving the remaining
(deep) layers to scan only a tiny fraction of the full corpus. This pruning
drastically increases the efficiency of MRC models and further prevents the
later layers from overfitting to prevalent short paragraphs in the training
set. Our framework is extremely flexible and naturally applicable to a wide
variety of models. Our experiment on doc-SQuAD and TriviaQA tasks demonstrates
its effectiveness in consistently improving both speed and quality of several
diverse MRC models.Comment: Technical repor