Machine Translation and the Evaluation of Its Quality

Machine translation has already become part of our everyday life. This chapter gives an overview of machine translation approaches. Statistical machine translation was a dominant approach over the past 20 years. It brought many cases of practical use. It is described in more detail in this chapter. Statistical machine translation is not equally successful for all language pairs. Highly inflectional languages are hard to process, especially as target languages. As statistical machine translation has almost reached the limits of its capacity, neural machine translation is becoming the technology of the future. This chapter also describes the evaluation of machine translation quality. It covers manual and automatic evaluations. Traditional and recently proposed metrics for automatic machine translation evaluation are described. Human translation still provides the best translation quality, but it is, in general, time-consuming and expensive. Integration of human and machine translation is a promising workflow for the future. Machine translation will not replace human translation, but it can serve as a tool to increase productivity in the translation process

Multiword expression aware neural machine translation

Multiword Expressions (MWEs) are a frequently occurring phenomenon found in all natural languages that is of great importance to linguistic theory, natural language processing applications, and machine translation systems. Neural Machine Translation (NMT) architectures do not handle these expression well and previous studies have not explicitly addressed MWEs in this framework. In this work, we show that using external linguistic resources and data augmentation we can improve both translations of MWEs that occur in the source, and the generation of MWEs on the target, and improve performance by up to 5.09 BLEU points on MWE test sets. We also devise a MWE score to specifically assess the quality of MWE translation which agrees with human evaluation. We make available the MWEscore implementation – along with MWE-annotated training sets and corpus-based lists of MWEs – for reproduction and extension

Adaptive Translation : Finding Interlingual Mappings Using Self-Organizing Maps

Volume: 5163This paper presents a method for creating interlingual word-to-word or phrase-to-phrase mappings between any two languages using the self-organizing map algorithm. The method can be used as a component in a statistical machine translation system. The conceptual space created by the self-organizing map serves as a kind of interlingual representation. The specific problems of machine translation are discussed in some detail. The proposed method serves in alleviating two problems. The main problem addressed here is the fact that different languages divide the conceptual space differently. The approach can also help in dealing with lexical ambiguity.Peer reviewe

Referential translation machines for quality estimation

We introduce referential translation machines (RTM) for quality estimation of translation outputs. RTMs are a computational model for identifying the translation acts between any two data sets with respect to a reference corpus selected in the same domain, which can be used for estimating the quality of translation outputs, judging the semantic similarity between text, and evaluating the quality of student answers. RTMs achieve top performance in automatic, accurate, and language independent prediction of sentence-level and word-level statistical machine translation (SMT) quality. RTMs remove the need to access any SMT system specific information or prior knowledge of the training data or models used when generating the translations. We develop novel techniques for solving all subtasks in the WMT13 quality estimation (QE) task (QET 2013) based on individual RTM models. Our results achieve improvements over last year’s QE task results (QET 2012), as well as our previous results, provide new features and techniques for QE, and rank 1st or 2nd in all of the subtasks

Continuous spaces in statistical machine Translation

[EN] Classically, statistical machine translation relied on representations of words in a discrete space. Words and phrases were atomically represented as indices in a vector. In the last years, techniques for representing words and phrases in a continuous space have arisen. In this scenario, a word is represented in the continuous space as a real-valued, dense and low-dimensional vector. Statistical models can profit from this richer representation, since it is able to naturally take into account concepts such as semantic or syntactic relationships between words and phrases. This approach is encouraging, but it also entails new challenges. In this work, a language model which relies on continuous representations of words is developed. Such model makes use of a bidirectional recurrent neural network, which is able to take into account both the past and the future context of words. Since the model is costly to train, the training dataset is reduced by using bilingual sentence selection techniques. Two selection methods are used and compared. The language model is then used to rerank translation hypotheses. Results show improvements on the translation quality. Moreover, a new approach for machine translation has been recently proposed: The so-called neural machine translation. It consists in the sole use of a large neural network for carrying out the translation process. In this work, such novel model is compared to the existing phrase-based approaches of statistical machine translation. Finally, the neural translation models are combined with diverse machine translation systems, in order to provide a consensus translation, which aim to improve the translation given by each single system.[ES] Los sistemas clásicos de traducción automática estadística están basados en representaciones de palabras en un espacio discreto. Palabras y segmentos se representan como índices en un vector. Durante los últimos años han surgido técnicas para realizar la representación de palabras y segmentos en un espacio continuo. En este escenario, una palabra se representa en el espacio continuo como un vector de valores reales, denso y de baja dimensión. Los modelos estadísticos pueden aprovecharse de esta representación más rica, puesto que incluye de forma natural conceptos semánticos o relaciones sintácticas entre palabras y segmentos. Esta aproximación es prometedora, pero también conlleva nuevos retos. En este trabajo se desarrolla un modelo de lenguaje basado en representaciones continuas de palabras. Dicho modelo emplea una red neuronal recurrente bidireccional, la cual es capaz de considerar tanto el contexto pasado como el contexto futuro de las palabras. Debido a que este modelo es costoso de entrenar, se emplea un conjunto de entrenamiento reducido mediante técnicas de selección de frases bilingües. Se emplean y comparan dos métodos de selección. Una vez entrenado, el modelo se emplea para reordenar hipótesis de traducción. Los resultados muestran mejoras en la calidad de la traducción. Por otro lado, recientemente se propuso una nueva aproximación a la traducción automática: la llamada traducción automática neuronal. Consiste en el uso exclusivo de una gran red neuronal para llevar a cabo el proceso de traducción. En este trabajo, este nuevo modelo se compara al paradigma actual de traducción basada en segmentos. Finalmente, los modelos de traducción neuronales son combinados con otros sistemas de traducción automática, para ofrecer una traducción consensuada, que busca mejorar las traducciones individuales que cada sistema ofrecePeris Abril, Á. (2015). Continuous spaces in statistical machine Translation. http://hdl.handle.net/10251/68448Archivo delegad

The Recurrent Temporal Discriminative Restricted Boltzmann Machines

Classification of sequence data is the topic of interest for dynamic Bayesian models and Recurrent Neural Networks (RNNs). While the former can explicitly model the temporal dependencies between class variables, the latter have a capability of learning representations. Several attempts have been made to improve performance by combining these two approaches or increasing the processing capability of the hidden units in RNNs. This often results in complex models with a large number of learning parameters. In this paper, a compact model is proposed which offers both representation learning and temporal inference of class variables by rolling Restricted Boltzmann Machines (RBMs) and class variables over time. We address the key issue of intractability in this variant of RBMs by optimising a conditional distribution, instead of a joint distribution. Experiments reported in the paper on melody modelling and optical character recognition show that the proposed model can outperform the state-of-the-art. Also, the experimental results on optical character recognition, part-of-speech tagging and text chunking demonstrate that our model is comparable to recurrent neural networks with complex memory gates while requiring far fewer parameters

Apprentissage discriminant des modèles continus en traduction automatique

Over the past few years, neural network (NN) architectures have been successfully applied to many Natural Language Processing (NLP) applications, such as Automatic Speech Recognition (ASR) and Statistical Machine Translation (SMT).For the language modeling task, these models consider linguistic units (i.e words and phrases) through their projections into a continuous (multi-dimensional) space, and the estimated distribution is a function of these projections. Also qualified continuous-space models (CSMs), their peculiarity hence lies in this exploitation of a continuous representation that can be seen as an attempt to address the sparsity issue of the conventional discrete models. In the context of SMT, these echniques have been applied on neural network-based language models (NNLMs) included in SMT systems, and oncontinuous-space translation models (CSTMs). These models have led to significant and consistent gains in the SMT performance, but are also considered as very expensive in training and inference, especially for systems involving large vocabularies. To overcome this issue, Structured Output Layer (SOUL) and Noise Contrastive Estimation (NCE) have been proposed; the former modifies the standard structure on vocabulary words, while the latter approximates the maximum-likelihood estimation (MLE) by a sampling method. All these approaches share the same estimation criterion which is the MLE ; however using this procedure results in an inconsistency between theobjective function defined for parameter stimation and the way models are used in the SMT application. The work presented in this dissertation aims to design new performance-oriented and global training procedures for CSMs to overcome these issues. The main contributions lie in the investigation and evaluation of efficient training methods for (large-vocabulary) CSMs which aim~:(a) to reduce the total training cost, and (b) to improve the efficiency of these models when used within the SMT application. On the one hand, the training and inference cost can be reduced (using the SOUL structure or the NCE algorithm), or by reducing the number of iterations via a faster convergence. This thesis provides an empirical analysis of these solutions on different large-scale SMT tasks. On the other hand, we propose a discriminative training framework which optimizes the performance of the whole system containing the CSM as a component model. The experimental results show that this framework is efficient to both train and adapt CSM within SMT systems, opening promising research perspectives.Durant ces dernières années, les architectures de réseaux de neurones (RN) ont été appliquées avec succès à de nombreuses applications en Traitement Automatique de Langues (TAL), comme par exemple en Reconnaissance Automatique de la Parole (RAP) ainsi qu'en Traduction Automatique (TA).Pour la tâche de modélisation statique de la langue, ces modèles considèrent les unités linguistiques (c'est-à-dire des mots et des segments) à travers leurs projections dans un espace continu (multi-dimensionnel), et la distribution de probabilité à estimer est une fonction de ces projections.Ainsi connus sous le nom de "modèles continus" (MC), la particularité de ces derniers se trouve dans l'exploitation de la représentation continue qui peut être considérée comme une solution au problème de données creuses rencontré lors de l'utilisation des modèles discrets conventionnels.Dans le cadre de la TA, ces techniques ont été appliquées dans les modèles de langue neuronaux (MLN) utilisés dans les systèmes de TA, et dans les modèles continus de traduction (MCT).L'utilisation de ces modèles se sont traduit par d'importantes et significatives améliorations des performances des systèmes de TA. Ils sont néanmoins très coûteux lors des phrases d'apprentissage et d'inférence, notamment pour les systèmes ayant un grand vocabulaire.Afin de surmonter ce problème, l'architecture SOUL (pour "Structured Output Layer" en anglais) et l'algorithme NCE (pour "Noise Contrastive Estimation", ou l'estimation contrastive bruitée) ont été proposés: le premier modifie la structure standard de la couche de sortie, alors que le second cherche à approximer l'estimation du maximum de vraisemblance (MV) par une méthode d’échantillonnage.Toutes ces approches partagent le même critère d'estimation qui est la log-vraisemblance; pourtant son utilisation mène à une incohérence entre la fonction objectif définie pour l'estimation des modèles, et la manière dont ces modèles seront utilisés dans les systèmes de TA.Cette dissertation vise à concevoir de nouvelles procédures d'entraînement des MC, afin de surmonter ces problèmes.Les contributions principales se trouvent dans l'investigation et l'évaluation des méthodes d'entraînement efficaces pour MC qui visent à: (i) réduire le temps total de l'entraînement, et (ii) améliorer l'efficacité de ces modèles lors de leur utilisation dans les systèmes de TA.D'un côté, le coût d'entraînement et d'inférence peut être réduit (en utilisant l'architecture SOUL ou l'algorithme NCE), ou la convergence peut être accélérée.La dissertation présente une analyse empirique de ces approches pour des tâches de traduction automatique à grande échelle.D'un autre côté, nous proposons un cadre d'apprentissage discriminant qui optimise la performance du système entier ayant incorporé un modèle continu.Les résultats expérimentaux montrent que ce cadre d'entraînement est efficace pour l'apprentissage ainsi que pour l'adaptation des MC au sein des systèmes de TA, ce qui ouvre de nouvelles perspectives prometteuses