Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models

We investigate the task of building open domain, conversational dialogue systems based on large dialogue corpora using generative models. Generative models produce system responses that are autonomously generated word-by-word, opening up the possibility for realistic, flexible interactions. In support of this goal, we extend the recently proposed hierarchical recurrent encoder-decoder neural network to the dialogue domain, and demonstrate that this model is competitive with state-of-the-art neural language models and back-off n-gram models. We investigate the limitations of this and similar approaches, and show how its performance can be improved by bootstrapping the learning from a larger question-answer pair corpus and from pretrained word embeddings.Comment: 8 pages with references; Published in AAAI 2016 (Special Track on Cognitive Systems

Generative Conversational Agents- The State-of-the-Art and the Future of Intelligent Conversational Systems

Intelligent conversational agents that generate responses from scratch are rapidly gaining in popularity. Sequence-to-sequence deep learning models are particularly well-suited for generating a textual response from a query. In this paper, I describe various generative models that are capable of having open-domain conversations. Toward the end, I present a null result I obtained in an attempt to train a chatbot from a small dataset and propose the use of a deep memory based machine translation model for training chatbots on small datasets

Meta-path Augmented Response Generation

We propose a chatbot, namely Mocha to make good use of relevant entities when generating responses. Augmented with meta-path information, Mocha is able to mention proper entities following the conversation flow.Comment: AAAI 201

Learning Multi-Level Information for Dialogue Response Selection by Highway Recurrent Transformer

With the increasing research interest in dialogue response generation, there is an emerging branch formulating this task as selecting next sentences, where given the partial dialogue contexts, the goal is to determine the most probable next sentence. Following the recent success of the Transformer model, this paper proposes (1) a new variant of attention mechanism based on multi-head attention, called highway attention, and (2) a recurrent model based on transformer and the proposed highway attention, so-called Highway Recurrent Transformer. Experiments on the response selection task in the seventh Dialog System Technology Challenge (DSTC7) show the capability of the proposed model of modeling both utterance-level and dialogue-level information; the effectiveness of each module is further analyzed as well