4,837 research outputs found
LSTM based Conversation Models
In this paper, we present a conversational model that incorporates both
context and participant role for two-party conversations. Different
architectures are explored for integrating participant role and context
information into a Long Short-term Memory (LSTM) language model. The
conversational model can function as a language model or a language generation
model. Experiments on the Ubuntu Dialog Corpus show that our model can capture
multiple turn interaction between participants. The proposed method outperforms
a traditional LSTM model as measured by language model perplexity and response
ranking. Generated responses show characteristic differences between the two
participant roles
Affective Neural Response Generation
Existing neural conversational models process natural language primarily on a
lexico-syntactic level, thereby ignoring one of the most crucial components of
human-to-human dialogue: its affective content. We take a step in this
direction by proposing three novel ways to incorporate affective/emotional
aspects into long short term memory (LSTM) encoder-decoder neural conversation
models: (1) affective word embeddings, which are cognitively engineered, (2)
affect-based objective functions that augment the standard cross-entropy loss,
and (3) affectively diverse beam search for decoding. Experiments show that
these techniques improve the open-domain conversational prowess of
encoder-decoder networks by enabling them to produce emotionally rich responses
that are more interesting and natural.Comment: 8 page
Incorporating Loose-Structured Knowledge into Conversation Modeling via Recall-Gate LSTM
Modeling human conversations is the essence for building satisfying chat-bots
with multi-turn dialog ability. Conversation modeling will notably benefit from
domain knowledge since the relationships between sentences can be clarified due
to semantic hints introduced by knowledge. In this paper, a deep neural network
is proposed to incorporate background knowledge for conversation modeling.
Through a specially designed Recall gate, domain knowledge can be transformed
into the extra global memory of Long Short-Term Memory (LSTM), so as to enhance
LSTM by cooperating with its local memory to capture the implicit semantic
relevance between sentences within conversations. In addition, this paper
introduces the loose structured domain knowledge base, which can be built with
slight amount of manual work and easily adopted by the Recall gate. Our model
is evaluated on the context-oriented response selecting task, and experimental
results on both two datasets have shown that our approach is promising for
modeling human conversations and building key components of automatic chatting
systems.Comment: under review of IJCNN 2017; 10 pages, 5 figure
Improving Retrieval Modeling Using Cross Convolution Networks And Multi Frequency Word Embedding
To build a satisfying chatbot that has the ability of managing a
goal-oriented multi-turn dialogue, accurate modeling of human conversation is
crucial. In this paper we concentrate on the task of response selection for
multi-turn human-computer conversation with a given context. Previous
approaches show weakness in capturing information of rare keywords that appear
in either or both context and correct response, and struggle with long input
sequences. We propose Cross Convolution Network (CCN) and Multi Frequency word
embedding to address both problems. We train several models using the Ubuntu
Dialogue dataset which is the largest freely available multi-turn based
dialogue corpus. We further build an ensemble model by averaging predictions of
multiple models. We achieve a new state-of-the-art on this dataset with
considerable improvements compared to previous best results
The Role of Conversation Context for Sarcasm Detection in Online Interactions
Computational models for sarcasm detection have often relied on the content
of utterances in isolation. However, speaker's sarcastic intent is not always
obvious without additional context. Focusing on social media discussions, we
investigate two issues: (1) does modeling of conversation context help in
sarcasm detection and (2) can we understand what part of conversation context
triggered the sarcastic reply. To address the first issue, we investigate
several types of Long Short-Term Memory (LSTM) networks that can model both the
conversation context and the sarcastic response. We show that the conditional
LSTM network (Rocktaschel et al., 2015) and LSTM networks with sentence level
attention on context and response outperform the LSTM model that reads only the
response. To address the second issue, we present a qualitative analysis of
attention weights produced by the LSTM models with attention and discuss the
results compared with human performance on the task.Comment: SIGDial 201
Neural Matching Models for Question Retrieval and Next Question Prediction in Conversation
The recent boom of AI has seen the emergence of many human-computer
conversation systems such as Google Assistant, Microsoft Cortana, Amazon Echo
and Apple Siri. We introduce and formalize the task of predicting questions in
conversations, where the goal is to predict the new question that the user will
ask, given the past conversational context. This task can be modeled as a
"sequence matching" problem, where two sequences are given and the aim is to
learn a model that maps any pair of sequences to a matching probability. Neural
matching models, which adopt deep neural networks to learn sequence
representations and matching scores, have attracted immense research interests
of information retrieval and natural language processing communities. In this
paper, we first study neural matching models for the question retrieval task
that has been widely explored in the literature, whereas the effectiveness of
neural models for this task is relatively unstudied. We further evaluate the
neural matching models in the next question prediction task in conversations.
We have used the publicly available Quora data and Ubuntu chat logs in our
experiments. Our evaluations investigate the potential of neural matching
models with representation learning for question retrieval and next question
prediction in conversations. Experimental results show that neural matching
models perform well for both tasks.Comment: Neu-IR 2017: The SIGIR 2017 Workshop on Neural Information Retrieval
(SIGIR Neu-IR 2017), Tokyo, Japan, August 7-11, 201
Incorporating Relevant Knowledge in Context Modeling and Response Generation
To sustain engaging conversation, it is critical for chatbots to make good
use of relevant knowledge. Equipped with a knowledge base, chatbots are able to
extract conversation-related attributes and entities to facilitate context
modeling and response generation. In this work, we distinguish the uses of
attribute and entity and incorporate them into the encoder-decoder architecture
in different manners. Based on the augmented architecture, our chatbot, namely
Mike, is able to generate responses by referring to proper entities from the
collected knowledge. To validate the proposed approach, we build a movie
conversation corpus on which the proposed approach significantly outperforms
other four knowledge-grounded models
Dialogue Act Sequence Labeling using Hierarchical encoder with CRF
Dialogue Act recognition associate dialogue acts (i.e., semantic labels) to
utterances in a conversation. The problem of associating semantic labels to
utterances can be treated as a sequence labeling problem. In this work, we
build a hierarchical recurrent neural network using bidirectional LSTM as a
base unit and the conditional random field (CRF) as the top layer to classify
each utterance into its corresponding dialogue act. The hierarchical network
learns representations at multiple levels, i.e., word level, utterance level,
and conversation level. The conversation level representations are input to the
CRF layer, which takes into account not only all previous utterances but also
their dialogue acts, thus modeling the dependency among both, labels and
utterances, an important consideration of natural dialogue. We validate our
approach on two different benchmark data sets, Switchboard and Meeting Recorder
Dialogue Act, and show performance improvement over the state-of-the-art
methods by and absolute points, respectively. It is worth
noting that the inter-annotator agreement on Switchboard data set is ,
and our method is able to achieve the accuracy of about despite being
trained on the noisy data
Augmenting End-to-End Dialog Systems with Commonsense Knowledge
Building dialog agents that can converse naturally with humans is a
challenging yet intriguing problem of artificial intelligence. In open-domain
human-computer conversation, where the conversational agent is expected to
respond to human responses in an interesting and engaging way, commonsense
knowledge has to be integrated into the model effectively. In this paper, we
investigate the impact of providing commonsense knowledge about the concepts
covered in the dialog. Our model represents the first attempt to integrating a
large commonsense knowledge base into end-to-end conversational models. In the
retrieval-based scenario, we propose the Tri-LSTM model to jointly take into
account message and commonsense for selecting an appropriate response. Our
experiments suggest that the knowledge-augmented models are superior to their
knowledge-free counterparts in automatic evaluation
A Sentiment-and-Semantics-Based Approach for Emotion Detection in Textual Conversations
Emotions are physiological states generated in humans in reaction to internal
or external events. They are complex and studied across numerous fields
including computer science. As humans, on reading "Why don't you ever text me!"
we can either interpret it as a sad or angry emotion and the same ambiguity
exists for machines. Lack of facial expressions and voice modulations make
detecting emotions from text a challenging problem. However, as humans
increasingly communicate using text messaging applications, and digital agents
gain popularity in our society, it is essential that these digital agents are
emotion aware, and respond accordingly.
In this paper, we propose a novel approach to detect emotions like happy, sad
or angry in textual conversations using an LSTM based Deep Learning model. Our
approach consists of semi-automated techniques to gather training data for our
model. We exploit advantages of semantic and sentiment based embeddings and
propose a solution combining both. Our work is evaluated on real-world
conversations and significantly outperforms traditional Machine Learning
baselines as well as other off-the-shelf Deep Learning models
- …