466 research outputs found
Domain transfer for deep natural language generation from abstract meaning representations
Stochastic natural language generation systems that are trained from labelled datasets are often domainspecific in their annotation and in their mapping from semantic input representations to lexical-syntactic outputs. As a result, learnt models fail to generalize across domains, heavily restricting their usability beyond single applications. In this article, we focus on the problem of domain adaptation for natural language generation. We show how linguistic knowledge from a source domain, for which labelled data is available, can be adapted to a target domain by reusing training data across domains. As a key to this, we propose to employ abstract meaning representations as a common semantic representation across domains. We model natural language generation as a long short-term memory recurrent neural network encoderdecoder, in which one recurrent neural network learns a latent representation of a semantic input, and a second recurrent neural network learns to decode it to a sequence of words. We show that the learnt representations can be transferred across domains and can be leveraged effectively to improve training on new unseen domains. Experiments in three different domains and with six datasets demonstrate that the lexical-syntactic constructions learnt in one domain can be transferred to new domains and achieve up to 75-100% of the performance of in-domain training. This is based on objective metrics such as BLEU and semantic error rate and a subjective human rating study. Training a policy from prior knowledge from a different domain is consistently better than pure in-domain training by up to 10%
Efficient Cross-Task Prompt Tuning for Few-Shot Conversational Emotion Recognition
Emotion Recognition in Conversation (ERC) has been widely studied due to its
importance in developing emotion-aware empathetic machines. The rise of
pre-trained language models (PLMs) has further pushed the limit of ERC
performance. However, most recent works on ERC using PLMs are heavily
data-driven, and requires fine-tuning the entire PLMs. To improve both sample
and computational efficiency, we propose a derivative-free optimization method
called Cross-Task Prompt Tuning (CTPT) for few-shot conversational emotion
recognition. Unlike existing methods that learn independent knowledge from
individual tasks, CTPT leverages sharable cross-task knowledge by exploiting
external knowledge from other source tasks to improve learning performance
under the few-shot setting. Moreover, CTPT only needs to optimize a vector
under the low intrinsic dimensionality without gradient, which is highly
parameter-efficient compared with existing approaches. Experiments on five
different contextual conversation datasets demonstrate that our CTPT method has
superior results on both few-shot scenarios and zero-shot transfers.Comment: Findings of EMNLP 202
Off the Beaten Path: Let's Replace Term-Based Retrieval with k-NN Search
Retrieval pipelines commonly rely on a term-based search to obtain candidate
records, which are subsequently re-ranked. Some candidates are missed by this
approach, e.g., due to a vocabulary mismatch. We address this issue by
replacing the term-based search with a generic k-NN retrieval algorithm, where
a similarity function can take into account subtle term associations. While an
exact brute-force k-NN search using this similarity function is slow, we
demonstrate that an approximate algorithm can be nearly two orders of magnitude
faster at the expense of only a small loss in accuracy. A retrieval pipeline
using an approximate k-NN search can be more effective and efficient than the
term-based pipeline. This opens up new possibilities for designing effective
retrieval pipelines. Our software (including data-generating code) and
derivative data based on the Stack Overflow collection is available online
- …