3,174 research outputs found
XL-NBT: A Cross-lingual Neural Belief Tracking Framework
Task-oriented dialog systems are becoming pervasive, and many companies
heavily rely on them to complement human agents for customer service in call
centers. With globalization, the need for providing cross-lingual customer
support becomes more urgent than ever. However, cross-lingual support poses
great challenges---it requires a large amount of additional annotated data from
native speakers. In order to bypass the expensive human annotation and achieve
the first step towards the ultimate goal of building a universal dialog system,
we set out to build a cross-lingual state tracking framework. Specifically, we
assume that there exists a source language with dialog belief tracking
annotations while the target languages have no annotated dialog data of any
form. Then, we pre-train a state tracker for the source language as a teacher,
which is able to exploit easy-to-access parallel data. We then distill and
transfer its own knowledge to the student state tracker in target languages. We
specifically discuss two types of common parallel resources: bilingual corpus
and bilingual dictionary, and design different transfer learning strategies
accordingly. Experimentally, we successfully use English state tracker as the
teacher to transfer its knowledge to both Italian and German trackers and
achieve promising results.Comment: 13 pages, 5 figures, 3 tables, accepted to EMNLP 2018 conferenc
MULTI3NLU++: A Multilingual, Multi-Intent, Multi-Domain Dataset for Natural Language Understanding in Task-Oriented Dialogue
Task-oriented dialogue (TOD) systems have been widely deployed in many
industries as they deliver more efficient customer support. These systems are
typically constructed for a single domain or language and do not generalise
well beyond this. To support work on Natural Language Understanding (NLU) in
TOD across multiple languages and domains simultaneously, we constructed
MULTI3NLU++, a multilingual, multi-intent, multi-domain dataset. MULTI3NLU++
extends the English only NLU++ dataset to include manual translations into a
range of high, medium, and low resource languages (Spanish, Marathi, Turkish
and Amharic), in two domains (BANKING and HOTELS). Because of its multi-intent
property, MULTI3NLU++ represents complex and natural user goals, and therefore
allows us to measure the realistic performance of TOD systems in a varied set
of the world's languages. We use MULTI3NLU++ to benchmark state-of-the-art
multilingual models for the NLU tasks of intent detection and slot labelling
for TOD systems in the multilingual setting. The results demonstrate the
challenging nature of the dataset, particularly in the low-resource language
setting, offering ample room for future experimentation in multi-domain
multilingual TOD setups.Comment: ACL 2023 (Findings) Camera Read
An Open-Domain Dialog Act Taxonomy
This document defines the taxonomy of dialog acts that are necessary to encode domain-independent dialog moves in the context of a task-oriented, open-domain dialog. Such taxonomy is formulated to satisfy two complementary requirements: on the one hand, domain independence, i.e. the power to cover all the range of possible interactions in any type of conversation (particularly conversation oriented to the performance of tasks). On the other hand, the ability to instantiate a concrete set of tasks as defined by a specific knowledge base (such as an ontology of domain concepts and actions) and within a particular language. For the modeling of dialog acts, inspiration is taken from several well-known dialog annotation schemes, such as DAMSL (Core & Allen, 1997), TRAINS (Traum, 1996) and VERBMOBIL (Alexandersson et al., 1997)
Multi3NLU++: A Multilingual, Multi-Intent, Multi-Domain Dataset for Natural Language Understanding in Task-Oriented Dialogue
Task-oriented dialogue (ToD) systems have been widely deployed in many industries as they deliver more efficient customer support. These systems are typically constructed for a single domain or language and do not generalise well beyond this. To support work on Natural Language Understanding (NLU) in ToD across multiple languages and domains simultaneously, we constructed Multi3NLU++, a multilingual, multi-intent, multi-domain dataset. Multi3NLU++ extends the English-only NLU++ dataset to include manual translations into a range of high, medium, and low resource languages (Spanish, Marathi, Turkish and Amharic), in two domains (banking and hotels). Because of its multi-intent property, Multi3NLU++ represents complex and natural user goals, and therefore allows us to measure the realistic performance of ToD systems in a varied set of the world's languages. We use Multi3NLU++ to benchmark state-of-the-art multilingual models for the NLU tasks of intent detection and slot labeling for ToD systems in the multilingual setting. The results demonstrate the challenging nature of the dataset, particularly in the low-resource language setting, offering ample room for future experimentation in multi-domain multilingual ToD setups
A Systematic Study of Performance Disparities in Multilingual Task-Oriented Dialogue Systems
Achieving robust language technologies that can perform well across the
world's many languages is a central goal of multilingual NLP. In this work, we
take stock of and empirically analyse task performance disparities that exist
between multilingual task-oriented dialogue (ToD) systems. We first define new
quantitative measures of absolute and relative equivalence in system
performance, capturing disparities across languages and within individual
languages. Through a series of controlled experiments, we demonstrate that
performance disparities depend on a number of factors: the nature of the ToD
task at hand, the underlying pretrained language model, the target language,
and the amount of ToD annotated data. We empirically prove the existence of the
adaptation and intrinsic biases in current ToD systems: e.g., ToD systems
trained for Arabic or Turkish using annotated ToD data fully parallel to
English ToD data still exhibit diminished ToD task performance. Beyond
providing a series of insights into the performance disparities of ToD systems
in different languages, our analyses offer practical tips on how to approach
ToD data collection and system development for new languages.Comment: Accepted to EMNLP 202
- …