36 research outputs found
Contextual Out-of-Domain Utterance Handling With Counterfeit Data Augmentation
Neural dialog models often lack robustness to anomalous user input and
produce inappropriate responses which leads to frustrating user experience.
Although there are a set of prior approaches to out-of-domain (OOD) utterance
detection, they share a few restrictions: they rely on OOD data or multiple
sub-domains, and their OOD detection is context-independent which leads to
suboptimal performance in a dialog. The goal of this paper is to propose a
novel OOD detection method that does not require OOD data by utilizing
counterfeit OOD turns in the context of a dialog. For the sake of fostering
further research, we also release new dialog datasets which are 3 publicly
available dialog corpora augmented with OOD turns in a controllable way. Our
method outperforms state-of-the-art dialog models equipped with a conventional
OOD detection mechanism by a large margin in the presence of OOD utterances.Comment: ICASSP 201
Sources of Noise in Dialogue and How to Deal with Them
Training dialogue systems often entails dealing with noisy training examples
and unexpected user inputs. Despite their prevalence, there currently lacks an
accurate survey of dialogue noise, nor is there a clear sense of the impact of
each noise type on task performance. This paper addresses this gap by first
constructing a taxonomy of noise encountered by dialogue systems. In addition,
we run a series of experiments to show how different models behave when
subjected to varying levels of noise and types of noise. Our results reveal
that models are quite robust to label errors commonly tackled by existing
denoising algorithms, but that performance suffers from dialogue-specific
noise. Driven by these observations, we design a data cleaning algorithm
specialized for conversational settings and apply it as a proof-of-concept for
targeted dialogue denoising.Comment: 23 pages, 6 Figures, 5 tables. Accepted at SIGDIAL 202
Knowledge-Enhanced Multi-Label Few-Shot Product Attribute-Value Extraction
Existing attribute-value extraction (AVE) models require large quantities of
labeled data for training. However, new products with new attribute-value pairs
enter the market every day in real-world e-Commerce. Thus, we formulate AVE in
multi-label few-shot learning (FSL), aiming to extract unseen attribute value
pairs based on a small number of training examples. We propose a
Knowledge-Enhanced Attentive Framework (KEAF) based on prototypical networks,
leveraging the generated label description and category information to learn
more discriminative prototypes. Besides, KEAF integrates with hybrid attention
to reduce noise and capture more informative semantics for each class by
calculating the label-relevant and query-related weights. To achieve
multi-label inference, KEAF further learns a dynamic threshold by integrating
the semantic information from both the support set and the query set. Extensive
experiments with ablation studies conducted on two datasets demonstrate that
KEAF outperforms other SOTA models for information extraction in FSL. The code
can be found at: https://github.com/gjiaying/KEAFComment: 6 pages, 2 figures, published in CIKM 202
MUAHAH: Taking the Most out of Simple Conversational Agents
Dialog engines based on multi-agent architectures usually select a single agent, deemed to be the most suitable for a given scenario or for responding to a specific request, and disregard the answers from all of the other available agents. In this work, we present a multi-agent plug-and-play architecture that: (i) enables the integration of different agents; (ii) includes a decision maker module, responsible for selecting a suitable answer out of the responses of different agents. As usual, a single agent can be chosen to provide the final answer, but the latter can also be obtained from the responses of several agents, according to a voting scheme. We also describe three case studies in which we test several agents and decision making strategies; and show how new agents and a new decision strategy can be easily plugged in and take advantage of this platform in different ways. Experimentation also confirms that considering several agents contributes to better responses
Data-efficient methods for dialogue systems
Conversational User Interface (CUI) has become ubiquitous in everyday life, in consumer-focused products like Siri and Alexa or more business-oriented customer support automation
solutions. Deep learning underlies many recent breakthroughs in dialogue systems but requires
very large amounts of training data, often annotated by experts — and this dramatically increases the cost of deploying such systems in production setups and reduces their flexibility as
software products. Trained with smaller data, these methods end up severely lacking robustness
to various phenomena of spoken language (e.g. disfluencies), out-of-domain input, and often
just have too little generalisation power to other tasks and domains.
In this thesis, we address the above issues by introducing a series of methods for bootstrapping
robust dialogue systems from minimal data. Firstly, we study two orthogonal approaches to dialogue: a linguistically informed model (DyLan) and a machine learning-based one (MemN2N) —
from the data efficiency perspective, i.e. their potential to generalise from minimal data and
robustness to natural spontaneous input. We outline the steps to obtain data-efficient solutions
with either approach and proceed with the neural models for the rest of the thesis.
We then introduce the core contributions of this thesis, two data-efficient models for dialogue
response generation: the Dialogue Knowledge Transfer Network (DiKTNet) based on transferable latent dialogue representations, and the Generative-Retrieval Transformer (GRTr) combining response generation logic with a retrieval mechanism as the fallback. GRTr ranked first at
the Dialog System Technology Challenge 8 Fast Domain Adaptation task.
Next, we the problem of training robust neural models from minimal data. As such, we look at
robustness to disfluencies and propose a multitask LSTM-based model for domain-general disfluency detection. We then go on to explore robustness to anomalous, or out-of-domain (OOD)
input. We address this problem by (1) presenting Turn Dropout, a data-augmentation technique
facilitating training for anomalous input only using in-domain data, and (2) introducing VHCN
and AE-HCN, autoencoder-augmented models for efficient training with turn dropout based on
the Hybrid Code Networks (HCN) model family.
With all the above work addressing goal-oriented dialogue, our final contribution in this thesis
focuses on social dialogue where the main objective is maintaining natural, coherent, and engaging conversation for as long as possible. We introduce a neural model for response ranking
in social conversation used in Alana, the 3rd place winner in the Amazon Alexa Prize 2017 and
2018. For our model, we employ a novel technique of predicting the dialogue length as the main
objective for ranking. We show that this approach matches the performance of its counterpart
based on the conventional, human rating-based objective — and surpasses it given more raw
dialogue transcripts, thus reducing the dependence on costly and cumbersome dialogue annotations.EPSRC project BABBLE (grant EP/M01553X/1)