14 research outputs found
Conversation Style Transfer using Few-Shot Learning
Conventional text style transfer approaches for natural language focus on
sentence-level style transfer without considering contextual information, and
the style is described with attributes (e.g., formality). When applying style
transfer on conversations such as task-oriented dialogues, existing approaches
suffer from these limitations as context can play an important role and the
style attributes are often difficult to define in conversations. In this paper,
we introduce conversation style transfer as a few-shot learning problem, where
the model learns to perform style transfer by observing only the target-style
dialogue examples. We propose a novel in-context learning approach to solve the
task with style-free dialogues as a pivot. Human evaluation shows that by
incorporating multi-turn context, the model is able to match the target style
while having better appropriateness and semantic correctness compared to
utterance-level style transfer. Additionally, we show that conversation style
transfer can also benefit downstream tasks. Results on multi-domain intent
classification tasks show improvement in F1 scores after transferring the style
of training data to match the style of test data
Pre-training Intent-Aware Encoders for Zero- and Few-Shot Intent Classification
Intent classification (IC) plays an important role in task-oriented dialogue
systems as it identifies user intents from given utterances. However, models
trained on limited annotations for IC often suffer from a lack of
generalization to unseen intent classes. We propose a novel pre-training method
for text encoders that uses contrastive learning with intent psuedo-labels to
produce embeddings that are well-suited for IC tasks. By applying this
pre-training strategy, we also introduce the pre-trained intent-aware encoder
(PIE). Specifically, we first train a tagger to identify key phrases within
utterances that are crucial for interpreting intents. We then use these
extracted phrases to create examples for pre-training a text encoder in a
contrastive manner. As a result, our PIE model achieves up to 5.4% and 4.0%
higher accuracy than the previous state-of-the-art pre-trained sentence encoder
for the N-way zero- and one-shot settings on four IC datasets
Semantic Parsing in Task-Oriented Dialog with Recursive Insertion-Based Encoder
We introduce a Recursive INsertion-based Encoder (RINE), a novel approach for semantic parsing in task-oriented dialog. Our model consists of an encoder network that incrementally builds the semantic parse tree by predicting the non-terminal label and its positions in the linearized tree. At the generation time, the model constructs the semantic parse tree by recursively inserting the predicted non-terminal labels at the predicted positions until termination. RINE achieves state-of-the-art exact match accuracy on low- and high-resource versions of the conversational semantic parsing benchmark TOP, outperforming strong sequence-to-sequence models and transition-based parsers. We also show that our model design is applicable to nested named entity recognition task, where it performs on par with state-of-the-art approach designed for that task. Finally, we demonstrate that our approach is 2-3.5 times faster than the sequence-to-sequence model at inference time