Generative Conversational Agents- The State-of-the-Art and the Future of Intelligent Conversational Systems

Intelligent conversational agents that generate responses from scratch are rapidly gaining in popularity. Sequence-to-sequence deep learning models are particularly well-suited for generating a textual response from a query. In this paper, I describe various generative models that are capable of having open-domain conversations. Toward the end, I present a null result I obtained in an attempt to train a chatbot from a small dataset and propose the use of a deep memory based machine translation model for training chatbots on small datasets

Generating descriptions that summarize geospatial and temporal data

Effective data summarization methods that use AI techniques can help humans understand large sets of data. In this paper, we describe a knowledge-based method for automatically generating summaries of geospatial and temporal data, i.e. data with geographical and temporal references. The method is useful for summarizing data streams, such as GPS traces and traffic information, that are becoming more prevalent with the increasing use of sensors in computing devices. The method presented here is an initial architecture for our ongoing research in this domain. In this paper we describe the data representations we have designed for our method, our implementations of components to perform data abstraction and natural language generation. We also discuss evaluation results that show the ability of our method to generate certain types of geospatial and temporal descriptions

A knowledge-based method for generating summaries of spatial movement in geographic areas

In this article we describe a method for automatically generating text summaries of data corresponding to traces of spatial movement in geographical areas. The method can help humans to understand large data streams, such as the amounts of GPS data recorded by a variety of sensors in mobile phones, cars, etc. We describe the knowledge representations we designed for our method and the main components of our method for generating the summaries: a discourse planner, an abstraction module and a text generator. We also present evaluation results that show the ability of our method to generate certain types of geospatial and temporal descriptions

GenLeNa: Sistema para la construcción de Aplicaciones de Generación de Lenguaje Natural

In this article the proposal is made for the division of the process of construction of natural language generation (NLG) systems into two stages: content planning (CP), which is dependent on the mastery of the application to be developed, and document structuring (DS). This division allows people who are not expert in NLG to develop natural language generation systems, concentrating on building abstract representations of the information to be communicated (called messages). Specific architecture for the DS stage is also presented. This enables NLG researchers to work ortogonally on specific techniques and methodologies for the conversion of messages into text which is grammatically and syntactically correct.En este artículo se propone la división del proceso de construcción de sistemas de Generación de Lenguajes Natural (GLN) en dos etapas: planificación del contenido (EPC), que es dependiente del dominio de la aplicación a desarrollar, y estructuración del documento (EED). Esta división permite que personas no expertas en GLN puedan desarrollar sistemas de generación de lenguajes natural enfocándose en construir representaciones abstractas de la información que se desea comunicar (denominadas mensajes). Adicionalmente se presenta una arquitectura específica para la etapa EED que permite a investigadores en GLN trabajar ortogonalmente en técnicas y metodologías específicas para la transformación de los mensajes en texto gramatical y sintácticamente correcto

Designing Service-Oriented Chatbot Systems Using a Construction Grammar-Driven Natural Language Generation System

Service oriented chatbot systems are used to inform users in a conversational manner about a particular service or product on a website. Our research shows that current systems are time consuming to build and not very accurate or satisfying to users. We find that natural language understanding and natural language generation methods are central to creating an e�fficient and useful system. In this thesis we investigate current and past methods in this research area and place particular emphasis on Construction Grammar and its computational implementation. Our research shows that users have strong emotive reactions to how these systems behave, so we also investigate the human computer interaction component. We present three systems (KIA, John and KIA2), and carry out extensive user tests on all of them, as well as comparative tests. KIA is built using existing methods, John is built with the user in mind and KIA2 is built using the construction grammar method. We found that the construction grammar approach performs well in service oriented chatbots systems, and that users preferred it over other systems

Natural language response generation in mixed-initiative dialogs.

Yip Wing Lin Winnie.Thesis (M.Phil.)--Chinese University of Hong Kong, 2004.Includes bibliographical references (leaves 102-105).Abstracts in English and Chinese.Chapter 1 --- Introduction --- p.1Chapter 1.1 --- Overview --- p.1Chapter 1.2 --- Thesis Goals --- p.3Chapter 1.3 --- Thesis Outline --- p.5Chapter 2 --- Background --- p.6Chapter 2.1 --- Natural Language Generation --- p.6Chapter 2.1.1 --- Template-based Approach --- p.7Chapter 2.1.2 --- Rule-based Approach --- p.8Chapter 2.1.3 --- Statistical Approach --- p.9Chapter 2.1.4 --- Hybrid Approach --- p.10Chapter 2.1.5 --- Machine Learning Approach --- p.11Chapter 2.2 --- Evaluation Method --- p.12Chapter 2.2.1 --- Cooperative Principles --- p.13Chapter 2.3 --- Chapter Summary --- p.13Chapter 3 --- Natural Language Understanding --- p.14Chapter 3.1 --- The CUHK Restaurant Domain --- p.15Chapter 3.2 --- "Task Goals, Dialog Acts, Concept Categories and Annotation" --- p.17Chapter 3.2.1 --- Task Goals (TGs) and Dialog Acts (DAs) --- p.17Chapter 3.2.2 --- Concept Categories (CTG/CDA) --- p.20Chapter 3.2.3 --- Utterance Segmentation and Annotation --- p.21Chapter 3.3 --- Task Goal and Dialog Act Identification --- p.22Chapter 3.3.1 --- Belief Networks Development --- p.22Chapter 3.3.2 --- Task Goal and Dialog Act Inference --- p.24Chapter 3.3.3 --- Network Dimensions --- p.25Chapter 3.4 --- Chapter Summary --- p.29Chapter 4 --- Automatic Utterance Segmentation --- p.30Chapter 4.1 --- Utterance Definition --- p.31Chapter 4.2 --- Segmentation Procedure --- p.33Chapter 4.2.1 --- Tokenization --- p.35Chapter 4.2.2 --- POS Tagging --- p.36Chapter 4.2.3 --- Multi-Parser Architecture (MPA) Language Parsing --- p.38Chapter 4.2.4 --- Top-down Generalized Representation --- p.40Chapter 4.3 --- Evaluation --- p.47Chapter 4.3.1 --- Results --- p.47Chapter 4.3.2 --- Analysis --- p.48Chapter 4.4 --- Chapter Summary --- p.50Chapter 5 --- Natural Language Response Generation --- p.52Chapter 5.1 --- System Overview --- p.52Chapter 5.2 --- Corpus-derived Dialog State Transition Rules --- p.55Chapter 5.3 --- Hand-designed Text Generation Templates --- p.56Chapter 5.4 --- Performance Evaluation --- p.59Chapter 5.4.1 --- Task Completion Rate --- p.61Chapter 5.4.2 --- Grice's Maxims and Perceived User Satisfaction --- p.62Chapter 5.4.3 --- Error Analysis --- p.64Chapter 5.5 --- Chapter Summary --- p.65Chapter 6 --- Bilingual Response Generation using Semi-Automatically- Induced Response Templates --- p.67Chapter 6.1 --- Response Data --- p.68Chapter 6.2 --- Semi-Automatic Grammar Induction --- p.69Chapter 6.2.1 --- Agglomerative Clustering --- p.69Chapter 6.2.2 --- Parameters Selection --- p.70Chapter 6.3 --- Application to Response Grammar Induction --- p.71Chapter 6.3.1 --- Parameters Selection --- p.73Chapter 6.3.2 --- Unsupervised Grammar Induction --- p.76Chapter 6.3.3 --- Post-processing --- p.80Chapter 6.3.4 --- Prior Knowledge Injection --- p.82Chapter 6.4 --- Response Templates Generation --- p.84Chapter 6.4.1 --- Induced Response Grammar --- p.84Chapter 6.4.2 --- Template Formation --- p.84Chapter 6.4.3 --- Bilingual Response Templates --- p.89Chapter 6.5 --- Evaluation --- p.89Chapter 6.5.1 --- "Task Completion Rate, Grice's Maxims and User Sat- isfaction" --- p.91Chapter 6.6 --- Chapter Summary --- p.94Chapter 7 --- Conclusion --- p.96Chapter 7.1 --- Summary --- p.96Chapter 7.2 --- Contributions --- p.98Chapter 7.3 --- Future Work --- p.100Bibliography --- p.102Chapter A --- Domain-Specific Task Goals in the CUHK Restaurants Do- main --- p.107Chapter B --- Full List of VERBMOBIL-2 Dialog Acts --- p.109Chapter C --- Dialog Acts for Customer Requests and Waiter Responsesin the CUHK Restaurants Domain --- p.111Chapter D --- Grammar for Task Goal and Dialog Act Identification --- p.116Chapter E --- Utterance Definition --- p.119Chapter F --- Dialog State Transition Rules --- p.121Chapter G --- Full List of Templates Selection Conditions --- p.125Chapter H --- Hand-designed Text Generation Templates --- p.130Chapter I --- Evaluation Test Questionnaire for Dialog System in the CUHK Restaurant Domain --- p.135Chapter J --- POS Tags --- p.137Chapter K --- Full List of Lexicon and contextual rule modifications --- p.139Chapter L --- Top-down Generalized Representations --- p.141Chapter M --- Sample Outputs for Automatic Utterance Segmentation --- p.144Chapter N --- Induced Grammar --- p.145Chapter O --- Seeded Categories --- p.148Chapter P --- Semi-Automatically-Induced Response Templates --- p.150Chapter Q --- Details of the Statistical Testing Regarding Grice's Maxims and User Satisfaction --- p.15

Harvesting and summarizing user-generated content for advanced speech-based human-computer interaction

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (p. 155-164).There have been many assistant applications on mobile devices, which could help people obtain rich Web content such as user-generated data (e.g., reviews, posts, blogs, and tweets). However, online communities and social networks are expanding rapidly and it is impossible for people to browse and digest all the information via simple search interface. To help users obtain information more efficiently, both the interface for data access and the information representation need to be improved. An intuitive and personalized interface, such as a dialogue system, could be an ideal assistant, which engages a user in a continuous dialogue to garner the user's interest and capture the user's intent, and assists the user via speech-navigated interactions. In addition, there is a great need for a type of application that can harvest data from the Web, summarize the information in a concise manner, and present it in an aggregated yet natural way such as direct human dialogue. This thesis, therefore, aims to conduct research on a universal framework for developing speech-based interface that can aggregate user-generated Web content and present the summarized information via speech-based human-computer interaction. To accomplish this goal, several challenges must be met. Firstly, how to interpret users' intention from their spoken input correctly? Secondly, how to interpret the semantics and sentiment of user-generated data and aggregate them into structured yet concise summaries? Lastly, how to develop a dialogue modeling mechanism to handle discourse and present the highlighted information via natural language? This thesis explores plausible approaches to tackle these challenges. We will explore a lexicon modeling approach for semantic tagging to improve spoken language understanding and query interpretation. We will investigate a parse-and-paraphrase paradigm and a sentiment scoring mechanism for information extraction from unstructured user-generated data. We will also explore sentiment-involved dialogue modeling and corpus-based language generation approaches for dialogue and discourse. Multilingual prototype systems in multiple domains have been implemented for demonstration.by Jingjing Liu.Ph.D

Natural Language Generation in Dialog Systems

Recent advances in Automatic Speech Recognition technology have put the goal of naturally sounding dialog systems within reach. However, the improved speech recognition has brought to light a new problem: as dialog systems understand more of what the user tells them, they need to be more sophisticated at responding to the user. The issue of system response to users has been extensively studied by the natural language generation community, though rarely in the context of dialog systems. We show how research in generation can be adapted to dialog systems, and how the high cost of hand-crafting knowledge-based generation systems can be overcome by employing machine learning techniques