1,203 research outputs found
A Sequential Matching Framework for Multi-turn Response Selection in Retrieval-based Chatbots
We study the problem of response selection for multi-turn conversation in
retrieval-based chatbots. The task requires matching a response candidate with
a conversation context, whose challenges include how to recognize important
parts of the context, and how to model the relationships among utterances in
the context. Existing matching methods may lose important information in
contexts as we can interpret them with a unified framework in which contexts
are transformed to fixed-length vectors without any interaction with responses
before matching. The analysis motivates us to propose a new matching framework
that can sufficiently carry the important information in contexts to matching
and model the relationships among utterances at the same time. The new
framework, which we call a sequential matching framework (SMF), lets each
utterance in a context interacts with a response candidate at the first step
and transforms the pair to a matching vector. The matching vectors are then
accumulated following the order of the utterances in the context with a
recurrent neural network (RNN) which models the relationships among the
utterances. The context-response matching is finally calculated with the hidden
states of the RNN. Under SMF, we propose a sequential convolutional network and
sequential attention network and conduct experiments on two public data sets to
test their performance. Experimental results show that both models can
significantly outperform the state-of-the-art matching methods. We also show
that the models are interpretable with visualizations that provide us insights
on how they capture and leverage the important information in contexts for
matching.Comment: Submitted to Computational Linguistic
Filtering before Iteratively Referring for Knowledge-Grounded Response Selection in Retrieval-Based Chatbots
The challenges of building knowledge-grounded retrieval-based chatbots lie in
how to ground a conversation on its background knowledge and how to match
response candidates with both context and knowledge simultaneously. This paper
proposes a method named Filtering before Iteratively REferring (FIRE) for this
task. In this method, a context filter and a knowledge filter are first built,
which derive knowledge-aware context representations and context-aware
knowledge representations respectively by global and bidirectional attention.
Besides, the entries irrelevant to the conversation are discarded by the
knowledge filter. After that, iteratively referring is performed between
context and response representations as well as between knowledge and response
representations, in order to collect deep matching features for scoring
response candidates. Experimental results show that FIRE outperforms previous
methods by margins larger than 2.8% and 4.1% on the PERSONA-CHAT dataset with
original and revised personas respectively, and margins larger than 3.1% on the
CMU_DoG dataset in terms of top-1 accuracy. We also show that FIRE is more
interpretable by visualizing the knowledge grounding process.Comment: Accepted by EMNLP 2020 Finding
Response Selection with Topic Clues for Retrieval-based Chatbots
We consider incorporating topic information into message-response matching to
boost responses with rich content in retrieval-based chatbots. To this end, we
propose a topic-aware convolutional neural tensor network (TACNTN). In TACNTN,
matching between a message and a response is not only conducted between a
message vector and a response vector generated by convolutional neural
networks, but also leverages extra topic information encoded in two topic
vectors. The two topic vectors are linear combinations of topic words of the
message and the response respectively, where the topic words are obtained from
a pre-trained LDA model and their weights are determined by themselves as well
as the message vector and the response vector. The message vector, the response
vector, and the two topic vectors are fed to neural tensors to calculate a
matching score. Empirical study on a public data set and a human annotated data
set shows that TACNTN can significantly outperform state-of-the-art methods for
message-response matching.Comment: under reviewed of AAAI 201
Lingke: A Fine-grained Multi-turn Chatbot for Customer Service
Traditional chatbots usually need a mass of human dialogue data, especially
when using supervised machine learning method. Though they can easily deal with
single-turn question answering, for multi-turn the performance is usually
unsatisfactory. In this paper, we present Lingke, an information retrieval
augmented chatbot which is able to answer questions based on given product
introduction document and deal with multi-turn conversations. We will introduce
a fine-grained pipeline processing to distill responses based on unstructured
documents, and attentive sequential context-response matching for multi-turn
conversations.Comment: Accepted by COLING 2018 demonstration pape
AI-Powered Text Generation for Harmonious Human-Machine Interaction: Current State and Future Directions
In the last two decades, the landscape of text generation has undergone
tremendous changes and is being reshaped by the success of deep learning. New
technologies for text generation ranging from template-based methods to neural
network-based methods emerged. Meanwhile, the research objectives have also
changed from generating smooth and coherent sentences to infusing personalized
traits to enrich the diversification of newly generated content. With the rapid
development of text generation solutions, one comprehensive survey is urgent to
summarize the achievements and track the state of the arts. In this survey
paper, we present the general systematical framework, illustrate the widely
utilized models and summarize the classic applications of text generation.Comment: Accepted by IEEE UIC 201
Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbots
Existing multi-turn context-response matching methods mainly concentrate on
obtaining multi-level and multi-dimension representations and better
interactions between context utterances and response. However, in real-place
conversation scenarios, whether a response candidate is suitable not only
counts on the given dialogue context but also other backgrounds, e.g., wording
habits, user-specific dialogue history content. To fill the gap between these
up-to-date methods and the real-world applications, we incorporate
user-specific dialogue history into the response selection and propose a
personalized hybrid matching network (PHMN). Our contributions are two-fold: 1)
our model extracts personalized wording behaviors from user-specific dialogue
history as extra matching information; 2) we perform hybrid representation
learning on context-response utterances and explicitly incorporate a customized
attention mechanism to extract vital information from context-response
interactions so as to improve the accuracy of matching. We evaluate our model
on two large datasets with user identification, i.e., personalized Ubuntu
dialogue Corpus (P-Ubuntu) and personalized Weibo dataset (P-Weibo).
Experimental results confirm that our method significantly outperforms several
strong models by combining personalized attention, wording behaviors, and
hybrid representation learning.Comment: Accepted by ACM Transactions on Information Systems, 25 pages, 2
figures, 9 table
A Repository of Conversational Datasets
Progress in Machine Learning is often driven by the availability of large
datasets, and consistent evaluation metrics for comparing modeling approaches.
To this end, we present a repository of conversational datasets consisting of
hundreds of millions of examples, and a standardised evaluation procedure for
conversational response selection models using '1-of-100 accuracy'. The
repository contains scripts that allow researchers to reproduce the standard
datasets, or to adapt the pre-processing and data filtering steps to their
needs. We introduce and evaluate several competitive baselines for
conversational response selection, whose implementations are shared in the
repository, as well as a neural encoder model that is trained on the entire
training set
Speaker-Aware BERT for Multi-Turn Response Selection in Retrieval-Based Chatbots
In this paper, we study the problem of employing pre-trained language models
for multi-turn response selection in retrieval-based chatbots. A new model,
named Speaker-Aware BERT (SA-BERT), is proposed in order to make the model
aware of the speaker change information, which is an important and intrinsic
property of multi-turn dialogues. Furthermore, a speaker-aware disentanglement
strategy is proposed to tackle the entangled dialogues. This strategy selects a
small number of most important utterances as the filtered context according to
the speakers' information in them. Finally, domain adaptation is performed to
incorporate the in-domain knowledge into pre-trained language models.
Experiments on five public datasets show that our proposed model outperforms
the present models on all metrics by large margins and achieves new
state-of-the-art performances for multi-turn response selection.Comment: Accepted by CIKM 202
Sequential Sentence Matching Network for Multi-turn Response Selection in Retrieval-based Chatbots
Recently, open domain multi-turn chatbots have attracted much interest from
lots of researchers in both academia and industry. The dominant retrieval-based
methods use context-response matching mechanisms for multi-turn response
selection. Specifically, the state-of-the-art methods perform the
context-response matching by word or segment similarity. However, these models
lack a full exploitation of the sentence-level semantic information, and make
simple mistakes that humans can easily avoid. In this work, we propose a
matching network, called sequential sentence matching network (S2M), to use the
sentence-level semantic information to address the problem. Firstly and most
importantly, we find that by using the sentence-level semantic information, the
network successfully addresses the problem and gets a significant improvement
on matching, resulting in a state-of-the-art performance. Furthermore, we
integrate the sentence matching we introduced here and the usual word
similarity matching reported in the current literature, to match at different
semantic levels. Experiments on three public data sets show that such
integration further improves the model performance.Comment: 10 pages, 4 figure
Interactive Matching Network for Multi-Turn Response Selection in Retrieval-Based Chatbots
In this paper, we propose an interactive matching network (IMN) for the
multi-turn response selection task. First, IMN constructs word representations
from three aspects to address the challenge of out-of-vocabulary (OOV) words.
Second, an attentive hierarchical recurrent encoder (AHRE), which is capable of
encoding sentences hierarchically and generating more descriptive
representations by aggregating with an attention mechanism, is designed.
Finally, the bidirectional interactions between whole multi-turn contexts and
response candidates are calculated to derive the matching information between
them. Experiments on four public datasets show that IMN outperforms the
baseline models on all metrics, achieving a new state-of-the-art performance
and demonstrating compatibility across domains for multi-turn response
selection.Comment: Accepted by CIKM 201
- …