523 research outputs found
Densely Connected Attention Propagation for Reading Comprehension
We propose DecaProp (Densely Connected Attention Propagation), a new densely
connected neural architecture for reading comprehension (RC). There are two
distinct characteristics of our model. Firstly, our model densely connects all
pairwise layers of the network, modeling relationships between passage and
query across all hierarchical levels. Secondly, the dense connectors in our
network are learned via attention instead of standard residual skip-connectors.
To this end, we propose novel Bidirectional Attention Connectors (BAC) for
efficiently forging connections throughout the network. We conduct extensive
experiments on four challenging RC benchmarks. Our proposed approach achieves
state-of-the-art results on all four, outperforming existing baselines by up to
in absolute F1 score.Comment: NIPS 201
A Character-Level Approach to the Text Normalization Problem Based on a New Causal Encoder
Text normalization is a ubiquitous process that appears as the first step of
many Natural Language Processing problems. However, previous Deep Learning
approaches have suffered from so-called silly errors, which are undetectable on
unsupervised frameworks, making those models unsuitable for deployment. In this
work, we make use of an attention-based encoder-decoder architecture that
overcomes these undetectable errors by using a fine-grained character-level
approach rather than a word-level one. Furthermore, our new general-purpose
encoder based on causal convolutions, called Causal Feature Extractor (CFE), is
introduced and compared to other common encoders. The experimental results show
the feasibility of this encoder, which leverages the attention mechanisms the
most and obtains better results in terms of accuracy, number of parameters and
convergence time. While our method results in a slightly worse initial accuracy
(92.74%), errors can be automatically detected and, thus, more readily solved,
obtaining a more robust model for deployment. Furthermore, there is still
plenty of room for future improvements that will push even further these
advantages.Comment: 19 pages, 14 figures, journa
Dual Ask-Answer Network for Machine Reading Comprehension
There are three modalities in the reading comprehension setting: question,
answer and context. The task of question answering or question generation aims
to infer an answer or a question when given the counterpart based on context.
We present a novel two-way neural sequence transduction model that connects
three modalities, allowing it to learn two tasks simultaneously and mutually
benefit one another. During training, the model receives
question-context-answer triplets as input and captures the cross-modal
interaction via a hierarchical attention process. Unlike previous joint
learning paradigms that leverage the duality of question generation and
question answering at data level, we solve such dual tasks at the architecture
level by mirroring the network structure and partially sharing components at
different layers. This enables the knowledge to be transferred from one task to
another, helping the model to find a general representation for each modality.
The evaluation on four public datasets shows that our dual-learning model
outperforms the mono-learning counterpart as well as the state-of-the-art joint
models on both question answering and question generation tasks.Comment: 8 pages, 5 figures, 4 tables. Code is available at
https://github.com/hanxiao/daane
Can Neural Networks Understand Logical Entailment?
We introduce a new dataset of logical entailments for the purpose of
measuring models' ability to capture and exploit the structure of logical
expressions against an entailment prediction task. We use this task to compare
a series of architectures which are ubiquitous in the sequence-processing
literature, in addition to a new model class---PossibleWorldNets---which
computes entailment as a "convolution over possible worlds". Results show that
convolutional networks present the wrong inductive bias for this class of
problems relative to LSTM RNNs, tree-structured neural networks outperform LSTM
RNNs due to their enhanced ability to exploit the syntax of logic, and
PossibleWorldNets outperform all benchmarks.Comment: Published at ICLR 2018 (main conference
QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
Current end-to-end machine reading and question answering (Q\&A) models are
primarily based on recurrent neural networks (RNNs) with attention. Despite
their success, these models are often slow for both training and inference due
to the sequential nature of RNNs. We propose a new Q\&A architecture called
QANet, which does not require recurrent networks: Its encoder consists
exclusively of convolution and self-attention, where convolution models local
interactions and self-attention models global interactions. On the SQuAD
dataset, our model is 3x to 13x faster in training and 4x to 9x faster in
inference, while achieving equivalent accuracy to recurrent models. The
speed-up gain allows us to train the model with much more data. We hence
combine our model with data generated by backtranslation from a neural machine
translation model. On the SQuAD dataset, our single model, trained with
augmented data, achieves 84.6 F1 score on the test set, which is significantly
better than the best published F1 score of 81.8.Comment: Published as full paper in ICLR 201
Unsupervised Paraphrasing without Translation
Paraphrasing exemplifies the ability to abstract semantic content from
surface forms. Recent work on automatic paraphrasing is dominated by methods
leveraging Machine Translation (MT) as an intermediate step. This contrasts
with humans, who can paraphrase without being bilingual. This work proposes to
learn paraphrasing models from an unlabeled monolingual corpus only. To that
end, we propose a residual variant of vector-quantized variational
auto-encoder.
We compare with MT-based approaches on paraphrase identification, generation,
and training augmentation. Monolingual paraphrasing outperforms unsupervised
translation in all settings. Comparisons with supervised translation are more
mixed: monolingual paraphrasing is interesting for identification and
augmentation; supervised translation is superior for generation.Comment: ACL 201
The Natural Language Decathlon: Multitask Learning as Question Answering
Deep learning has improved performance on many natural language processing
(NLP) tasks individually. However, general NLP models cannot emerge within a
paradigm that focuses on the particularities of a single metric, dataset, and
task. We introduce the Natural Language Decathlon (decaNLP), a challenge that
spans ten tasks: question answering, machine translation, summarization,
natural language inference, sentiment analysis, semantic role labeling,
zero-shot relation extraction, goal-oriented dialogue, semantic parsing, and
commonsense pronoun resolution. We cast all tasks as question answering over a
context. Furthermore, we present a new Multitask Question Answering Network
(MQAN) jointly learns all tasks in decaNLP without any task-specific modules or
parameters in the multitask setting. MQAN shows improvements in transfer
learning for machine translation and named entity recognition, domain
adaptation for sentiment analysis and natural language inference, and zero-shot
capabilities for text classification. We demonstrate that the MQAN's
multi-pointer-generator decoder is key to this success and performance further
improves with an anti-curriculum training strategy. Though designed for
decaNLP, MQAN also achieves state of the art results on the WikiSQL semantic
parsing task in the single-task setting. We also release code for procuring and
processing data, training and evaluating models, and reproducing all
experiments for decaNLP
FigureQA: An Annotated Figure Dataset for Visual Reasoning
We introduce FigureQA, a visual reasoning corpus of over one million
question-answer pairs grounded in over 100,000 images. The images are
synthetic, scientific-style figures from five classes: line plots, dot-line
plots, vertical and horizontal bar graphs, and pie charts. We formulate our
reasoning task by generating questions from 15 templates; questions concern
various relationships between plot elements and examine characteristics like
the maximum, the minimum, area-under-the-curve, smoothness, and intersection.
To resolve, such questions often require reference to multiple plot elements
and synthesis of information distributed spatially throughout a figure. To
facilitate the training of machine learning systems, the corpus also includes
side data that can be used to formulate auxiliary objectives. In particular, we
provide the numerical data used to generate each figure as well as bounding-box
annotations for all plot elements. We study the proposed visual reasoning task
by training several models, including the recently proposed Relation Network as
a strong baseline. Preliminary results indicate that the task poses a
significant machine learning challenge. We envision FigureQA as a first step
towards developing models that can intuitively recognize patterns from visual
representations of data.Comment: workshop paper at ICLR 201
Multimodal Dialogue State Tracking By QA Approach with Data Augmentation
Recently, a more challenging state tracking task, Audio-Video Scene-Aware
Dialogue (AVSD), is catching an increasing amount of attention among
researchers. Different from purely text-based dialogue state tracking, the
dialogue in AVSD contains a sequence of question-answer pairs about a video and
the final answer to the given question requires additional understanding of the
video. This paper interprets the AVSD task from an open-domain Question
Answering (QA) point of view and proposes a multimodal open-domain QA system to
deal with the problem. The proposed QA system uses common encoder-decoder
framework with multimodal fusion and attention. Teacher forcing is applied to
train a natural language generator. We also propose a new data augmentation
approach specifically under QA assumption. Our experiments show that our model
and techniques bring significant improvements over the baseline model on the
DSTC7-AVSD dataset and demonstrate the potentials of our data augmentation
techniques.Comment: AAAI DSTC8 Worksho
NSURL-2019 Shared Task 8: Semantic Question Similarity in Arabic
Question semantic similarity (Q2Q) is a challenging task that is very useful
in many NLP applications, such as detecting duplicate questions and question
answering systems. In this paper, we present the results and findings of the
shared task (Semantic Question Similarity in Arabic). The task was organized as
part of the first workshop on NLP Solutions for Under Resourced Languages
(NSURL 2019) The goal of the task is to predict whether two questions are
semantically similar or not, even if they are phrased differently. A total of 9
teams participated in the task. The datasets created for this task are made
publicly available to support further research on Arabic Q2Q.Comment: 8 pages, 2 figure, 3 tables, conference pape
- …