13 research outputs found
Topic-informed neural machine translation
In recent years, neural machine translation (NMT) has demonstrated state-of-the-art machine
translation (MT) performance. It is a new approach to MT, which tries to learn a set of parameters
to maximize the conditional probability of target sentences given source sentences. In this paper,
we present a novel approach to improve the translation performance in NMT by conveying topic
knowledge during translation. The proposed topic-informed NMT can increase the likelihood of
selecting words from the same topic and domain for translation. Experimentally, we demonstrate
that topic-informed NMT can achieve a 1.15 (3.3% relative) and 1.67 (5.4% relative) absolute
improvement in BLEU score on the Chinese-to-English language pair using NIST 2004 and 2005
test sets, respectively, compared to NMT without topic information
Domain Control for Neural Machine Translation
Machine translation systems are very sensitive to the domains they were
trained on. Several domain adaptation techniques have been deeply studied. We
propose a new technique for neural machine translation (NMT) that we call
domain control which is performed at runtime using a unique neural network
covering multiple domains. The presented approach shows quality improvements
when compared to dedicated domains translating on any of the covered domains
and even on out-of-domain data. In addition, model parameters do not need to be
re-estimated for each domain, making this effective to real use cases.
Evaluation is carried out on English-to-French translation for two different
testing scenarios. We first consider the case where an end-user performs
translations on a known domain. Secondly, we consider the scenario where the
domain is not known and predicted at the sentence level before translating.
Results show consistent accuracy improvements for both conditions.Comment: Published in RANLP 201
Translating Phrases in Neural Machine Translation
Phrases play an important role in natural language understanding and machine
translation (Sag et al., 2002; Villavicencio et al., 2005). However, it is
difficult to integrate them into current neural machine translation (NMT) which
reads and generates sentences word by word. In this work, we propose a method
to translate phrases in NMT by integrating a phrase memory storing target
phrases from a phrase-based statistical machine translation (SMT) system into
the encoder-decoder architecture of NMT. At each decoding step, the phrase
memory is first re-written by the SMT model, which dynamically generates
relevant target phrases with contextual information provided by the NMT model.
Then the proposed model reads the phrase memory to make probability estimations
for all phrases in the phrase memory. If phrase generation is carried on, the
NMT decoder selects an appropriate phrase from the memory to perform phrase
translation and updates its decoding state by consuming the words in the
selected phrase. Otherwise, the NMT decoder generates a word from the
vocabulary as the general NMT decoder does. Experiment results on the Chinese
to English translation show that the proposed model achieves significant
improvements over the baseline on various test sets.Comment: Accepted by EMNLP 201
Improving Word Sense Disambiguation in Neural Machine Translation with Salient Document Context
Lexical ambiguity is a challenging and pervasive problem in machine
translation (\mt). We introduce a simple and scalable approach to resolve
translation ambiguity by incorporating a small amount of extra-sentential
context in neural \mt. Our approach requires no sense annotation and no change
to standard model architectures. Since actual document context is not available
for the vast majority of \mt training data, we collect related sentences for
each input to construct pseudo-documents. Salient words from pseudo-documents
are then encoded as a prefix to each source sentence to condition the
generation of the translation. To evaluate, we release \docmucow, a challenge
set for translation disambiguation based on the English-German \mucow
\cite{raganato-etal-2020-evaluation} augmented with document IDs. Extensive
experiments show that our method translates ambiguous source words better than
strong sentence-level baselines and comparable document-level baselines while
reducing training costs
Exploring the use of Acoustic Embeddings in Neural Machine Translation
Neural Machine Translation (NMT) has recently demonstrated
improved performance over statistical machine translation
and relies on an encoder-decoder framework for translating
text from source to target. The structure of NMT makes
it amenable to add auxiliary features, which can provide complementary
information to that present in the source text. In
this paper, auxiliary features derived from accompanying
audio, are investigated for NMT and are compared and combined
with text-derived features. These acoustic embeddings
can help resolve ambiguity in the translation, thus improving
the output. The following features are experimented with:
Latent Dirichlet Allocation (LDA) topic vectors and GMM
subspace i-vectors derived from audio. These are contrasted
against: skip-gram/Word2Vec features and LDA features
derived from text. The results are encouraging and show
that acoustic information does help with NMT, leading to an
overall 3.3% relative improvement in BLEU scores
Dynamic Topic Tracker for KB-to-Text Generation
Recently, many KB-to-text generation tasks have been proposed to bridge the gap between knowledge bases and natural language by directly converting a group of knowledge base triples into human-readable sentences. However, most of the existing models suffer from the off-topic the problem, namely, the models are prone to generate some unrelated clauses that are somehow
involved with certain input terms regardless of the given input data. This problem seriously degrades the quality of the generation results. In this paper, we propose a novel dynamic topic tracker for solving this problem. Different from existing models, our proposed model learns a global hidden representation for topics and recognizes the corresponding topic during each
generation step. The recognized topic is used as additional information to guide the generation process and thus alleviates the off-topic problem. The experimental results show that our proposed model can enhance the performance of sentence generation and the off-topic problem is
significantly mitigated
Evaluation of Terminology Translation in Instance-Based Neural MT Adaptation
We address the issues arising when a neural machine translation engine trained on generic data receives requests from a new domain that contains many specific technical terms. Given training data of the new domain, we consider two alternative methods to adapt the generic system: corpus-based and instance-based adaptation. While the first approach is computationally more intensive in generating a domain-customized network, the latter operates more efficiently at translation time and can handle on-the-fly adaptation to multiple domains. Besides evaluating the generic and the adapted networks with conventional translation quality metrics, in this paper we focus on their ability to properly handle domain-specific terms. We show that instance-based adaptation, by fine-tuning the model on-the-fly, is capable to significantly boost the accuracy of translated terms, producing translations of quality comparable to the expensive corpus-based method