5 research outputs found
A Survey of Word Reordering in Statistical Machine Translation: Computational Models and Language Phenomena
Word reordering is one of the most difficult aspects of statistical machine
translation (SMT), and an important factor of its quality and efficiency.
Despite the vast amount of research published to date, the interest of the
community in this problem has not decreased, and no single method appears to be
strongly dominant across language pairs. Instead, the choice of the optimal
approach for a new translation task still seems to be mostly driven by
empirical trials. To orientate the reader in this vast and complex research
area, we present a comprehensive survey of word reordering viewed as a
statistical modeling challenge and as a natural language phenomenon. The survey
describes in detail how word reordering is modeled within different
string-based and tree-based SMT frameworks and as a stand-alone task, including
systematic overviews of the literature in advanced reordering modeling. We then
question why some approaches are more successful than others in different
language pairs. We argue that, besides measuring the amount of reordering, it
is important to understand which kinds of reordering occur in a given language
pair. To this end, we conduct a qualitative analysis of word reordering
phenomena in a diverse sample of language pairs, based on a large collection of
linguistic knowledge. Empirical results in the SMT literature are shown to
support the hypothesis that a few linguistic facts can be very useful to
anticipate the reordering characteristics of a language pair and to select the
SMT framework that best suits them.Comment: 44 pages, to appear in Computational Linguistic
Investigating the Relationship between Classification Quality and SMT Performance in Discriminative Reordering Models
Reordering is one of the most important factors affecting the quality of the output in
statistical machine translation (SMT). A considerable number of approaches that proposed addressing
the reordering problem are discriminative reordering models (DRM). The core component of the
DRMs is a classifier which tries to predict the correct word order of the sentence. Unfortunately,
the relationship between classification quality and ultimate SMT performance has not been
investigated to date. Understanding this relationship will allow researchers to select the classifier that
results in the best possible MT quality. It might be assumed that there is a monotonic relationship
between classification quality and SMT performance, i.e., any improvement in classification
performance will be monotonically reflected in overall SMT quality. In this paper, we experimentally
show that this assumption does not always hold, i.e., an improvement in classification performance
might actually degrade the quality of an SMT system, from the point of view of MT automatic
evaluation metrics. However, we show that if the improvement in the classification performance is
high enough, we can expect the SMT quality to improve as well. In addition to this, we show that
there is a negative relationship between classification accuracy and SMT performance in imbalanced
parallel corpora. For these types of corpora, we provide evidence that, for the evaluation of the
classifier, macro-averaged metrics such as macro-averaged F-measure are better suited than accuracy,
the metric commonly used to date
An empirical analysis of phrase-based and neural machine translation
Two popular types of machine translation (MT) are phrase-based and neural
machine translation systems. Both of these types of systems are composed of
multiple complex models or layers. Each of these models and layers learns
different linguistic aspects of the source language. However, for some of these
models and layers, it is not clear which linguistic phenomena are learned or
how this information is learned. For phrase-based MT systems, it is often clear
what information is learned by each model, and the question is rather how this
information is learned, especially for its phrase reordering model. For neural
machine translation systems, the situation is even more complex, since for many
cases it is not exactly clear what information is learned and how it is
learned.
To shed light on what linguistic phenomena are captured by MT systems, we
analyze the behavior of important models in both phrase-based and neural MT
systems. We consider phrase reordering models from phrase-based MT systems to
investigate which words from inside of a phrase have the biggest impact on
defining the phrase reordering behavior. Additionally, to contribute to the
interpretability of neural MT systems we study the behavior of the attention
model, which is a key component in neural MT systems and the closest model in
functionality to phrase reordering models in phrase-based systems. The
attention model together with the encoder hidden state representations form the
main components to encode source side linguistic information in neural MT. To
this end, we also analyze the information captured in the encoder hidden state
representations of a neural MT system. We investigate the extent to which
syntactic and lexical-semantic information from the source side is captured by
hidden state representations of different neural MT architectures.Comment: PhD thesis, University of Amsterdam, October 2020.
https://pure.uva.nl/ws/files/51388868/Thesis.pd