10,145 research outputs found
Adversarial Training for Community Question Answer Selection Based on Multi-scale Matching
Community-based question answering (CQA) websites represent an important
source of information. As a result, the problem of matching the most valuable
answers to their corresponding questions has become an increasingly popular
research topic. We frame this task as a binary (relevant/irrelevant)
classification problem, and present an adversarial training framework to
alleviate label imbalance issue. We employ a generative model to iteratively
sample a subset of challenging negative samples to fool our classification
model. Both models are alternatively optimized using REINFORCE algorithm. The
proposed method is completely different from previous ones, where negative
samples in training set are directly used or uniformly down-sampled. Further,
we propose using Multi-scale Matching which explicitly inspects the correlation
between words and ngrams of different levels of granularity. We evaluate the
proposed method on SemEval 2016 and SemEval 2017 datasets and achieves
state-of-the-art or similar performance
Quizbowl: The Case for Incremental Question Answering
Scholastic trivia competitions test knowledge and intelligence through
mastery of question answering. Modern question answering benchmarks are one
variant of the Turing test. Specifically, answering a set of questions as well
as a human is a minimum bar towards demonstrating human-like intelligence. This
paper makes the case that the format of one competition -- where participants
can answer in the middle of hearing a question (incremental) -- better
differentiates the skill between (human or machine) players. Additionally,
merging a sequential decision-making sub-task with question answering (QA)
provides a good setting for research in model calibration and opponent
modeling. Thus, embedded in this task are three machine learning challenges:
(1) factoid QA over thousands of Wikipedia-like answers, (2) calibration of the
QA model's confidence scores, and (3) sequential decision-making that
incorporates knowledge of the QA model, its calibration, and what the opponent
may do. We make two contributions: (1) collecting and curating a large factoid
QA dataset and an accompanying gameplay dataset, and (2) developing a model
that addresses these three machine learning challenges. In addition to offline
evaluation, we pitted our model against some of the most accomplished trivia
players in the world in a series of exhibition matches spanning several years.
Throughout this paper, we show that collaborations with the vibrant trivia
community have contributed to the quality of our dataset, spawned new research
directions, and doubled as an exciting way to engage the public with research
in machine learning and natural language processing
Building Chatbots from Forum Data: Model Selection Using Question Answering Metrics
We propose to use question answering (QA) data from Web forums to train
chatbots from scratch, i.e., without dialog training data. First, we extract
pairs of question and answer sentences from the typically much longer texts of
questions and answers in a forum. We then use these shorter texts to train
seq2seq models in a more efficient way. We further improve the parameter
optimization using a new model selection strategy based on QA measures.
Finally, we propose to use extrinsic evaluation with respect to a QA task as an
automatic evaluation method for chatbots. The evaluation shows that the model
achieves a MAP of 63.5% on the extrinsic task. Moreover, it can answer
correctly 49.5% of the questions when they are similar to questions asked in
the forum, and 47.3% of the questions when they are more conversational in
style.Comment: RANLP-201
Adversarial TableQA: Attention Supervision for Question Answering on Tables
The task of answering a question given a text passage has shown great
developments on model performance thanks to community efforts in building
useful datasets. Recently, there have been doubts whether such rapid progress
has been based on truly understanding language. The same question has not been
asked in the table question answering (TableQA) task, where we are tasked to
answer a query given a table. We show that existing efforts, of using "answers"
for both evaluation and supervision for TableQA, show deteriorating
performances in adversarial settings of perturbations that do not affect the
answer. This insight naturally motivates to develop new models that understand
question and table more precisely. For this goal, we propose Neural Operator
(NeOp), a multi-layer sequential network with attention supervision to answer
the query given a table. NeOp uses multiple Selective Recurrent Units (SelRUs)
to further help the interpretability of the answers of the model. Experiments
show that the use of operand information to train the model significantly
improves the performance and interpretability of TableQA models. NeOp
outperforms all the previous models by a big margin.Comment: ACML 201
A Survey of Document Grounded Dialogue Systems (DGDS)
Dialogue system (DS) attracts great attention from industry and academia
because of its wide application prospects. Researchers usually divide the DS
according to the function. However, many conversations require the DS to switch
between different functions. For example, movie discussion can change from
chit-chat to QA, the conversational recommendation can transform from chit-chat
to recommendation, etc. Therefore, classification according to functions may
not be enough to help us appreciate the current development trend. We classify
the DS based on background knowledge. Specifically, study the latest DS based
on the unstructured document(s). We define Document Grounded Dialogue System
(DGDS) as the DS that the dialogues are centering on the given document(s). The
DGDS can be used in scenarios such as talking over merchandise against product
Manual, commenting on news reports, etc. We believe that extracting
unstructured document(s) information is the future trend of the DS because a
great amount of human knowledge lies in these document(s). The research of the
DGDS not only possesses a broad application prospect but also facilitates AI to
better understand human knowledge and natural language. We analyze the
classification, architecture, datasets, models, and future development trends
of the DGDS, hoping to help researchers in this field.Comment: 30 pages, 4 figures, 13 table
Data Augmentation for Neural Online Chat Response Selection
Data augmentation seeks to manipulate the available data for training to
improve the generalization ability of models. We investigate two data
augmentation proxies, permutation and flipping, for neural dialog response
selection task on various models over multiple datasets, including both Chinese
and English languages. Different from standard data augmentation techniques,
our method combines the original and synthesized data for prediction. Empirical
results show that our approach can gain 1 to 3 recall-at-1 points over baseline
models in both full-scale and small-scale settings.Comment: EMNLP 2018 Worksho
Adversarial Classifier for Imbalanced Problems
Adversarial approach has been widely used for data generation in the last few
years. However, this approach has not been extensively utilized for classifier
training. In this paper, we propose an adversarial framework for classifier
training that can also handle imbalanced data. Indeed, a network is trained via
an adversarial approach to give weights to samples of the majority class such
that the obtained classification problem becomes more challenging for the
discriminator and thus boosts its classification capability. In addition to the
general imbalanced classification problems, the proposed method can also be
used for problems such as graph representation learning in which it is desired
to discriminate similar nodes from dissimilar nodes. Experimental results on
imbalanced data classification and on the tasks like graph link prediction show
the superiority of the proposed method compared to the state-of-the-art
methods
Beyond Leaderboards: A survey of methods for revealing weaknesses in Natural Language Inference data and models
Recent years have seen a growing number of publications that analyse Natural
Language Inference (NLI) datasets for superficial cues, whether they undermine
the complexity of the tasks underlying those datasets and how they impact those
models that are optimised and evaluated on this data. This structured survey
provides an overview of the evolving research area by categorising reported
weaknesses in models and datasets and the methods proposed to reveal and
alleviate those weaknesses for the English language. We summarise and discuss
the findings and conclude with a set of recommendations for possible future
research directions. We hope it will be a useful resource for researchers who
propose new datasets, to have a set of tools to assess the suitability and
quality of their data to evaluate various phenomena of interest, as well as
those who develop novel architectures, to further understand the implications
of their improvements with respect to their model's acquired capabilities.Comment: 10 Page
A Survey on Dialogue Systems: Recent Advances and New Frontiers
Dialogue systems have attracted more and more attention. Recent advances on
dialogue systems are overwhelmingly contributed by deep learning techniques,
which have been employed to enhance a wide range of big data applications such
as computer vision, natural language processing, and recommender systems. For
dialogue systems, deep learning can leverage a massive amount of data to learn
meaningful feature representations and response generation strategies, while
requiring a minimum amount of hand-crafting. In this article, we give an
overview to these recent advances on dialogue systems from various perspectives
and discuss some possible research directions. In particular, we generally
divide existing dialogue systems into task-oriented and non-task-oriented
models, then detail how deep learning techniques help them with representative
algorithms and finally discuss some appealing research directions that can
bring the dialogue system research into a new frontier.Comment: 13 pages. arXiv admin note: text overlap with arXiv:1703.01008 by
other author
Neural Approaches to Conversational AI
The present paper surveys neural approaches to conversational AI that have
been developed in the last few years. We group conversational systems into
three categories: (1) question answering agents, (2) task-oriented dialogue
agents, and (3) chatbots. For each category, we present a review of
state-of-the-art neural approaches, draw the connection between them and
traditional approaches, and discuss the progress that has been made and
challenges still being faced, using specific systems and models as case
studies.Comment: Foundations and Trends in Information Retrieval (95 pages
- …