14,136 research outputs found
An Attentional Neural Conversation Model with Improved Specificity
In this paper we propose a neural conversation model for conducting
dialogues. We demonstrate the use of this model to generate help desk
responses, where users are asking questions about PC applications. Our model is
distinguished by two characteristics. First, it models intention across turns
with a recurrent network, and incorporates an attention model that is
conditioned on the representation of intention. Secondly, it avoids generating
non-specific responses by incorporating an IDF term in the objective function.
The model is evaluated both as a pure generation model in which a help-desk
response is generated from scratch, and as a retrieval model with performance
measured using recall rates of the correct response. Experimental results
indicate that the model outperforms previously proposed neural conversation
architectures, and that using specificity in the objective function
significantly improves performances for both generation and retrieval
DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset
We develop a high-quality multi-turn dialog dataset, DailyDialog, which is
intriguing in several aspects. The language is human-written and less noisy.
The dialogues in the dataset reflect our daily communication way and cover
various topics about our daily life. We also manually label the developed
dataset with communication intention and emotion information. Then, we evaluate
existing approaches on DailyDialog dataset and hope it benefit the research
field of dialog systems.Comment: accepted by IJCNLP 201
Latent Intention Dialogue Models
Developing a dialogue agent that is capable of making autonomous decisions
and communicating by natural language is one of the long-term goals of machine
learning research. Traditional approaches either rely on hand-crafting a small
state-action set for applying reinforcement learning that is not scalable or
constructing deterministic models for learning dialogue sentences that fail to
capture natural conversational variability. In this paper, we propose a Latent
Intention Dialogue Model (LIDM) that employs a discrete latent variable to
learn underlying dialogue intentions in the framework of neural variational
inference. In a goal-oriented dialogue scenario, these latent intentions can be
interpreted as actions guiding the generation of machine responses, which can
be further refined autonomously by reinforcement learning. The experimental
evaluation of LIDM shows that the model out-performs published benchmarks for
both corpus-based and human evaluation, demonstrating the effectiveness of
discrete latent variable models for learning goal-oriented dialogues.Comment: Accepted at ICML 201
Neural Contextual Conversation Learning with Labeled Question-Answering Pairs
Neural conversational models tend to produce generic or safe responses in
different contexts, e.g., reply \textit{"Of course"} to narrative statements or
\textit{"I don't know"} to questions. In this paper, we propose an end-to-end
approach to avoid such problem in neural generative models. Additional memory
mechanisms have been introduced to standard sequence-to-sequence (seq2seq)
models, so that context can be considered while generating sentences. Three
seq2seq models, which memorize a fix-sized contextual vector from hidden input,
hidden input/output and a gated contextual attention structure respectively,
have been trained and tested on a dataset of labeled question-answering pairs
in Chinese. The model with contextual attention outperforms others including
the state-of-the-art seq2seq models on perplexity test. The novel contextual
model generates diverse and robust responses, and is able to carry out
conversations on a wide range of topics appropriately
Modeling Multi-turn Conversation with Deep Utterance Aggregation
Multi-turn conversation understanding is a major challenge for building
intelligent dialogue systems. This work focuses on retrieval-based response
matching for multi-turn conversation whose related work simply concatenates the
conversation utterances, ignoring the interactions among previous utterances
for context modeling. In this paper, we formulate previous utterances into
context using a proposed deep utterance aggregation model to form a
fine-grained context representation. In detail, a self-matching attention is
first introduced to route the vital information in each utterance. Then the
model matches a response with each refined utterance and the final matching
score is obtained after attentive turns aggregation. Experimental results show
our model outperforms the state-of-the-art methods on three multi-turn
conversation benchmarks, including a newly introduced e-commerce dialogue
corpus.Comment: Proceedings of the 27th International Conference on Computational
Linguistics (COLING 2018
Production Ready Chatbots: Generate if not Retrieve
In this paper, we present a hybrid model that combines a neural
conversational model and a rule-based graph dialogue system that assists users
in scheduling reminders through a chat conversation. The graph based system has
high precision and provides a grammatically accurate response but has a low
recall. The neural conversation model can cater to a variety of requests, as it
generates the responses word by word as opposed to using canned responses. The
hybrid system shows significant improvements over the existing baseline system
of rule based approach and caters to complex queries with a domain-restricted
neural model. Restricting the conversation topic and combination of graph based
retrieval system with a neural generative model makes the final system robust
enough for a real world application.Comment: DEEPDIAL-18, AAAI-201
Towards Neural Speaker Modeling in Multi-Party Conversation: The Task, Dataset, and Models
Neural network-based dialog systems are attracting increasing attention in
both academia and industry. Recently, researchers have begun to realize the
importance of speaker modeling in neural dialog systems, but there lacks
established tasks and datasets. In this paper, we propose speaker
classification as a surrogate task for general speaker modeling, and collect
massive data to facilitate research in this direction. We further investigate
temporal-based and content-based models of speakers, and propose several
hybrids of them. Experiments show that speaker classification is feasible, and
that hybrid models outperform each single component.Comment: In Proceedings of the Language Resources and Evaluation Conference
(LREC), 201
Affective Neural Response Generation
Existing neural conversational models process natural language primarily on a
lexico-syntactic level, thereby ignoring one of the most crucial components of
human-to-human dialogue: its affective content. We take a step in this
direction by proposing three novel ways to incorporate affective/emotional
aspects into long short term memory (LSTM) encoder-decoder neural conversation
models: (1) affective word embeddings, which are cognitively engineered, (2)
affect-based objective functions that augment the standard cross-entropy loss,
and (3) affectively diverse beam search for decoding. Experiments show that
these techniques improve the open-domain conversational prowess of
encoder-decoder networks by enabling them to produce emotionally rich responses
that are more interesting and natural.Comment: 8 page
Paying Attention to Attention: Highlighting Influential Samples in Sequential Analysis
In (Yang et al. 2016), a hierarchical attention network (HAN) is created for
document classification. The attention layer can be used to visualize text
influential in classifying the document, thereby explaining the model's
prediction. We successfully applied HAN to a sequential analysis task in the
form of real-time monitoring of turn taking in conversations. However, we
discovered instances where the attention weights were uniform at the stopping
point (indicating all turns were equivalently influential to the classifier),
preventing meaningful visualization for real-time human review or classifier
improvement. We observed that attention weights for turns fluctuated as the
conversations progressed, indicating turns had varying influence based on
conversation state. Leveraging this observation, we develop a method to create
more informative real-time visuals (as confirmed by human reviewers) in cases
of uniform attention weights using the changes in turn importance as a
conversation progresses over time
Answerer in Questioner's Mind: Information Theoretic Approach to Goal-Oriented Visual Dialog
Goal-oriented dialog has been given attention due to its numerous
applications in artificial intelligence. Goal-oriented dialogue tasks occur
when a questioner asks an action-oriented question and an answerer responds
with the intent of letting the questioner know a correct action to take. To ask
the adequate question, deep learning and reinforcement learning have been
recently applied. However, these approaches struggle to find a competent
recurrent neural questioner, owing to the complexity of learning a series of
sentences. Motivated by theory of mind, we propose "Answerer in Questioner's
Mind" (AQM), a novel information theoretic algorithm for goal-oriented dialog.
With AQM, a questioner asks and infers based on an approximated probabilistic
model of the answerer. The questioner figures out the answerer's intention via
selecting a plausible question by explicitly calculating the information gain
of the candidate intentions and possible answers to each question. We test our
framework on two goal-oriented visual dialog tasks: "MNIST Counting Dialog" and
"GuessWhat?!". In our experiments, AQM outperforms comparative algorithms by a
large margin.Comment: Selected for a spotlight presentation at NIPS, 2018. Camera ready
versio
- …