2 research outputs found
An End-to-End Goal-Oriented Dialog System with a Generative Natural Language Response Generation
Recently advancements in deep learning allowed the development of end-to-end
trained goal-oriented dialog systems. Although these systems already achieve
good performance, some simplifications limit their usage in real-life
scenarios.
In this work, we address two of these limitations: ignoring positional
information and a fixed number of possible response candidates. We propose to
use positional encodings in the input to model the word order of the user
utterances. Furthermore, by using a feedforward neural network, we are able to
generate the output word by word and are no longer restricted to a fixed number
of possible response candidates. Using the positional encoding, we were able to
achieve better accuracies in the Dialog bAbI Tasks and using the feedforward
neural network for generating the response, we were able to save computation
time and space consumption.Comment: 11 pages, 4 figures, forthcoming in IWSDS 2018; added quantitative
analysis of sensitivity to modified user utterances and minor improvement
Multi-task learning to improve natural language understanding
Recently advancements in sequence-to-sequence neural network architectures
have led to an improved natural language understanding. When building a neural
network-based Natural Language Understanding component, one main challenge is
to collect enough training data. The generation of a synthetic dataset is an
inexpensive and quick way to collect data. Since this data often has less
variety than real natural language, neural networks often have problems to
generalize to unseen utterances during testing. In this work, we address this
challenge by using multi-task learning. We train out-of-domain real data
alongside in-domain synthetic data to improve natural language understanding.
We evaluate this approach in the domain of airline travel information with two
synthetic datasets. As out-of-domain real data, we test two datasets based on
the subtitles of movies and series. By using an attention-based encoder-decoder
model, we were able to improve the F1-score over strong baselines from 80.76 %
to 84.98 % in the smaller synthetic dataset.Comment: 11 pages, 4 figures, 2 tables, forthcoming in IWSDS 201