Deep Reinforcement Learning for Dialogue Generation

Recent neural models of dialogue generation offer great promise for generating responses for conversational agents, but tend to be shortsighted, predicting utterances one at a time while ignoring their influence on future outcomes. Modeling the future direction of a dialogue is crucial to generating coherent, interesting dialogues, a need which led traditional NLP models of dialogue to draw on reinforcement learning. In this paper, we show how to integrate these goals, applying deep reinforcement learning to model future reward in chatbot dialogue. The model simulates dialogues between two virtual agents, using policy gradient methods to reward sequences that display three useful conversational properties: informativity (non-repetitive turns), coherence, and ease of answering (related to forward-looking function). We evaluate our model on diversity, length as well as with human judges, showing that the proposed algorithm generates more interactive responses and manages to foster a more sustained conversation in dialogue simulation. This work marks a first step towards learning a neural conversational model based on the long-term success of dialogues

Issues in integrating existing multi-agent systems for power engineering applications

Multi-agent systems (MAS) have proven to be an effective platform for diagnostic and condition monitoring applications in the power industry. For example, a multi-agent system architecture, entitled condition monitoring multi-agent system (COMMAS) (McArthur et al., 2004), has been applied to the ultra high frequency (UHF) monitoring of partial discharge activity inside transformers. Additionally, a multi-agent system, entitled protection engineering diagnostic agents (PEDA) (Hossack et al., 2003), has demonstrated the use of MAS technology for automated and enhanced post-fault analysis of power systems disturbances based on SCADA and digital fault recorder (DFR) data. In this paper, the authors propose the integration of COMMAS and PEDA as a means of offering enhanced decision support to engineers tasked with managing transformer assets. By providing automatically interpreted data related to condition monitoring and power system disturbances, the proposed integrated system offer engineers a more comprehensive picture of the health of a given transformer. Defects and deterioration in performance can be correlated with the operating conditions it experiences. The integration of COMMAS and PEDA has highlighted the issues inherent to the inter-operation of existing multi-agent systems and, in particular, the issues surrounding the use of differing ontologies. The authors believe that these issues need to be addressed if there is to be widespread deployment of MAS technology within the power industry. This paper presents research undertaken to integrate the two MAS and to deal with ontology issues

COTA: Improving the Speed and Accuracy of Customer Support through Ranking and Deep Networks

For a company looking to provide delightful user experiences, it is of paramount importance to take care of any customer issues. This paper proposes COTA, a system to improve speed and reliability of customer support for end users through automated ticket classification and answers selection for support representatives. Two machine learning and natural language processing techniques are demonstrated: one relying on feature engineering (COTA v1) and the other exploiting raw signals through deep learning architectures (COTA v2). COTA v1 employs a new approach that converts the multi-classification task into a ranking problem, demonstrating significantly better performance in the case of thousands of classes. For COTA v2, we propose an Encoder-Combiner-Decoder, a novel deep learning architecture that allows for heterogeneous input and output feature types and injection of prior knowledge through network architecture choices. This paper compares these models and their variants on the task of ticket classification and answer selection, showing model COTA v2 outperforms COTA v1, and analyzes their inner workings and shortcomings. Finally, an A/B test is conducted in a production setting validating the real-world impact of COTA in reducing issue resolution time by 10 percent without reducing customer satisfaction

Greeks and Trojans Together

This paper describes a comprehensive solution for the integration of object oriented ontology representation frameworks with logic-based agent communication frameworks. The proposed solution addresses the problem at both the agent communication level and the agent implementation level. At the agent communication level, we propose to extend logic content languages with some domain independent operators that allow building logic constructs as propositions from domain dependent entities defined in an object oriented ontology. At the implementation level, we propose to use object-oriented databases as the support for the agent information. Finally, we propose an automatic mechanism for translating agent messages using the extended content language into ODMG OQL commands, which are then used to interact with the object-oriented database. This binding mechanism relies on a special purpose data dictionary representing the mapping between the domain ontology and the agent internal database