Dialogue Simulation and Context Dynamics for Dialogue Management

Proceedings of the 16th Nordic Conference of Computational Linguistics NODALIDA-2007. Editors: Joakim Nivre, Heiki-Jaan Kaalep, Kadri Muischnek and Mare Koit. University of Tartu, Tartu, 2007. ISBN 978-9985-4-0513-0 (online) ISBN 978-9985-4-0514-7 (CD-ROM) pp. 310-317

Strategic dialogue management via deep reinforcement learning

Artificially intelligent agents equipped with strategic skills that can negotiate during their interactions with other natural or artificial agents are still underdeveloped. This paper describes a successful application of Deep Reinforcement Learning (DRL) for training intelligent agents with strategic conversational skills, in a situated dialogue setting. Previous studies have modelled the behaviour of strategic agents using supervised learning and traditional reinforcement learning techniques, the latter using tabular representations or learning with linear function approximation. In this study, we apply DRL with a high-dimensional state space to the strategic board game of Settlers of Catan---where players can offer resources in exchange for others and they can also reply to offers made by other players. Our experimental results report that the DRL-based learnt policies significantly outperformed several baselines including random, rule-based, and supervised-based behaviours. The DRL-based policy has a 53% win rate versus 3 automated players (`bots'), whereas a supervised player trained on a dialogue corpus in this setting achieved only 27%, versus the same 3 bots. This result supports the claim that DRL is a promising framework for training dialogue systems, and strategic agents with negotiation abilities

Downward compatible revision of dialogue annotation

This paper discusses some aspects of revising the ISO standard for dialogue act annotation (ISO 24617-2). The revision is aimed at making annotations using the ISO scheme more accurate and at providing more powerful tools for building natural language based dialogue systems, without invalidating the annotated resources that have been built, with the current version of the standard. In support of the revision of the standard, an analysis is provided of the downward compatibility of a revised annotation scheme with the original scheme at the levels of abstract syntax, concrete syntax, and semantics of annotations

Understanding enormous redshifts in highly concentrated Mn2+ phosphors

Broad band near infrared (NIR) emission has recently been reported for a wide variety of concentrated Mn2+ phosphors. Typically, Mn2+ emits in the green to red spectral region, depending on local coordination. The enormous redshift to the NIR was explained by exchange coupling between Mn2+ neighbours at high Mn2+ dopant concentrations. However, the reported redshifts are an order of magnitude larger than expected for exchange coupling and also the absence of a shift in excitation spectra suggests that exchange coupling cannot explain the observations. Here, extensive concentration, temperature and time dependent luminescence studies are reported for Mg1−xMnxAl2O4 (x = 0.01-0.5). The results show that the broad band NIR emission originates from NIR emitting trap centers, possibly Mn3+. High Mn2+ dopant concentrations enable efficient energy migration over the Mn2+ sublattice to these traps, consistent with the same excitation spectra for the green Mn2+ and NIR trap emission. Upon cooling to cryogenic temperatures energy migration is hampered and the green Mn2+ emission increases, especially in the most concentrated systems. Finally, the relative intensity of the NIR emission was varied by changing synthesis conditions providing further support that the NIR emission in concentrated Mn2+ phosphors originates from NIR emitting centers and not exchange coupled Mn2+ pairs

EQUIVALENCES BETWEEN STOCHASTIC SYSTEMS

Time-dependent correlation functions of (unstable) particles undergoing biased or unbiased diffusion, coagulation and annihilation are calculated. This is achieved by similarity transformations between different stochastic models and between stochastic and soluble {\em non-stochastic} models. The results agree with experiments on one-dimensional annihilation-coagulation processes.Comment: 15 pages, Latex. Some corrections made and an appendix adde

Survey Evidence on Conditional Norm Enforcement

We discuss survey evidence on individuals' willingness to sanction norm violations - such as evading taxes, drunk driving, fare dodging, or skiving o work - by expressing disapproval or social exclusion. Our data suggest that people condition their sanctioning behavior on their belief about the frequency of norm violations. The more commonly a norm violation is believed to occur, the lower the individuals' inclination to punish it. Based on an instrumental variable approach, we demonstrate that this pattern reflects a causal relationship

Evaluating Persuasion Strategies and Deep Reinforcement Learning methods for Negotiation Dialogue agents

In this paper we present a comparative evaluation of various negotiation strategies within an online version of the game “Settlers of Catan”. The comparison is based on human subjects playing games against artificial game-playing agents (‘bots’) which implement different negotiation dialogue strategies, using a chat dialogue interface to negotiate trades. Our results suggest that a negotiation strategy that uses persuasion, as well as a strategy that is trained from data using Deep Reinforcement Learning, both lead to an improved win rate against humans, compared to previous rule-based and supervised learning baseline dialogue negotiators