Incremental LSTM-based Dialog State Tracker

A dialog state tracker is an important component in modern spoken dialog systems. We present an incremental dialog state tracker, based on LSTM networks. It directly uses automatic speech recognition hypotheses to track the state. We also present the key non-standard aspects of the model that bring its performance close to the state-of-the-art and experimentally analyze their contribution: including the ASR confidence scores, abstracting scarcely represented values, including transcriptions in the training data, and model averaging

Robust Dialog State Tracking for Large Ontologies

The Dialog State Tracking Challenge 4 (DSTC 4) differentiates itself from the previous three editions as follows: the number of slot-value pairs present in the ontology is much larger, no spoken language understanding output is given, and utterances are labeled at the subdialog level. This paper describes a novel dialog state tracking method designed to work robustly under these conditions, using elaborate string matching, coreference resolution tailored for dialogs and a few other improvements. The method can correctly identify many values that are not explicitly present in the utterance. On the final evaluation, our method came in first among 7 competing teams and 24 entries. The F1-score achieved by our method was 9 and 7 percentage points higher than that of the runner-up for the utterance-level evaluation and for the subdialog-level evaluation, respectively.Comment: Paper accepted at IWSDS 201

Dialogue state tracking accuracy improvement by distinguishing slot-value pairs and dialogue behaviour

Dialog state tracking (DST) plays a critical role in cycle life of a task-oriented dialogue system. DST represents the goals of the consumer at each step by dialogue and describes such objectives as a conceptual structure comprising slot-value pairs and dialogue actions that specifically improve the performance and effectiveness of dialogue systems. DST faces several challenge

The use of discriminative belief tracking in POMDP-based dialogue systems

Statistical spoken dialogue systems based on Partially Ob-servable Markov Decision Processes (POMDPs) have been shown to be more robust to speech recognition errors by main-taining a belief distribution over multiple dialogue states and making policy decisions based on the entire distribution rather than the single most likely hypothesis. To date most POMDP-based systems have used generative trackers. However, con-cerns about modelling accuracy have created interest in dis-criminative methods, and recent results from the second Dia-log State Tracking Challenge (DSTC2) have shown that dis-criminative trackers can significantly outperform generative models in terms of tracking accuracy. The aim of this pa-per is to investigate the extent to which these improvements translate into improved task completion rates when incorpo-rated into a spoken dialogue system. To do this, the Recur-rent Neural Network (RNN) tracker described by Henderson et al in DSTC2 was integrated into the Cambridge statistical dialogue system and compared with the existing generative Bayesian network tracker. Using a Gaussian Process (GP) based policy, the experimental results indicate that the system using the RNN tracker performs significantly better than the system with the original Bayesian network tracker. Index Terms — dialogue management, spoken dialogue systems, recurrent neural networks, belief tracking, POMDP 1