Transductive data-selection algorithms for fine-tuning neural machine translation

Machine Translation models are trained to translate a variety of documents from one language into another. However, models specifically trained for a particular characteristics of the documents tend to perform better. Fine-tuning is a technique for adapting an NMT model to some domain. In this work, we want to use this technique to adapt the model to a given test set. In particular, we are using transductive data selection algorithms which take advantage the information of the test set to retrieve sentences from a larger parallel set

Combining SMT and NMT back-translated data for efficient NMT

Neural Machine Translation (NMT) models achieve their best performance when large sets of parallel data are used for training. Consequently, techniques for augmenting the training set have become popular recently. One of these methods is back-translation (Sennrich et al., 2016), which consists on generating synthetic sentences by translating a set of monolingual, target-language sentences using a Machine Translation (MT) model. Generally, NMT models are used for back-translation. In this work, we analyze the performance of models when the training data is extended with synthetic data using different MT approaches. In particular we investigate back-translated data generated not only by NMT but also by Statistical Machine Translation (SMT) models and combinations of both. The results reveal that the models achieve the best performances when the training set is augmented with back-translated data created by merging different MT approaches

Feature decay algorithms for neural machine translation

Neural Machine Translation (NMT) systems require a lot of data to be competitive. For this reason, data selection techniques are used only for finetuning systems that have been trained with larger amounts of data. In this work we aim to use Feature Decay Algorithms (FDA) data selection techniques not only to fine-tune a system but also to build a complete system with less data. Our findings reveal that it is possible to find a subset of sentence pairs, that outperforms by 1.11 BLEU points the full training corpus, when used for training a German-English NMT system

Data selection with feature decay algorithms using an approximated target side

AbstractData selection techniques applied to neural machine trans-lation (NMT) aim to increase the performance of a model byretrieving a subset of sentences for use as training data.One of the possible data selection techniques are trans-ductive learning methods, which select the data based on thetest set, i.e. the document to be translated. A limitation ofthese methods to date is that using the source-side test setdoes not by itself guarantee that sentences are selected withcorrect translations, or translations that are suitable given thetest-set domain. Some corpora, such as subtitle corpora, maycontain parallel sentences with inaccurate translations causedby localization or length restrictions.In order to try to fix this problem, in this paper we pro-pose to use an approximated target-side in addition to thesource-side when selecting suitable sentence-pairs for train-ing a model. This approximated target-side is built by pre-translating the source-side.In this work, we explore the performance of this generalidea for one specific data selection approach called FeatureDecay Algorithms (FDA).We train German-English NMT models on data selectedby using the test set (source), the approximated target side,and a mixture of both. Our findings reveal that models builtusing a combination of outputs of FDA (using the test setand an approximated target side) perform better than thosesolely using the test set. We obtain a statistically significantimprovement of more than 1.5 BLEU points over a modeltrained with all data, and more than 0.5 BLEU points over astrong FDA baseline that uses source-side information only

Data selection with feature decay algorithms using an approximated target side

Data selection techniques applied to neural machine translation (NMT) aim to increase the performance of a model by retrieving a subset of sentences for use as training data. One of the possible data selection techniques are transductive learning methods, which select the data based on the test set, i.e. the document to be translated. A limitation of these methods to date is that using the source-side test set does not by itself guarantee that sentences are selected with correct translations, or translations that are suitable given the test-set domain. Some corpora, such as subtitle corpora, may contain parallel sentences with inaccurate translations caused by localization or length restrictions. In order to try to fix this problem, in this paper we propose to use an approximated target-side in addition to the source-side when selecting suitable sentence-pairs for training a model. This approximated target-side is built by pretranslating the source-side. In this work, we explore the performance of this general idea for one specific data selection approach called Feature Decay Algorithms (FDA). We train German-English NMT models on data selected by using the test set (source), the approximated target side, and a mixture of both. Our findings reveal that models built using a combination of outputs of FDA (using the test set and an approximated target side) perform better than those solely using the test set. We obtain a statistically significant improvement of more than 1.5 BLEU points over a model trained with all data, and more than 0.5 BLEU points over a strong FDA baseline that uses source-side information only

Elastic-substitution decoding for hierarchical SMT: efficiency, richer search and double labels

Elastic-substitution decoding (ESD), first introduced by Chiang (2010), can be important for obtaining good results when applying labels to enrich hierarchical statistical machine translation (SMT). However, an efficient implementation is essential for scalable application. We describe how to achieve this, contributing essential details that were missing in the original exposition. We compare ESD to strict matching and show its superiority for both reordering and syntactic labels. To overcome the sub-optimal performance due to the late evaluation of features marking label substitution types, we increase the diversity of the rules explored during cube pruning initialization with respect to labels their labels. This approach gives significant improvements over basic ESD and performs favorably compared to extending the search by increasing the cube pruning pop-limit. Finally, we look at combining multiple labels. The combination of reordering labels and target-side boundary-tags yields a significant improvement in terms of the word-order sensitive metrics Kendall reordering score and METEOR. This confirms our intuition that the combination of reordering labels and syntactic labels can yield improvements over either label by itself, despite increased sparsity

Adaptation of machine translation models with back-translated data using transductive data selection methods

Data selection has proven its merit for improving Neural Machine Translation (NMT), when applied to authentic data. But the benefit of using synthetic data in NMT training, produced by the popular back-translation technique, raises the question if data selection could also be useful for synthetic data? In this work we use Infrequent n-gram Recovery (INR) and Feature Decay Algorithms (FDA), two transductive data selection methods to obtain subsets of sentences from synthetic data. These methods ensure that selected sentences share n-grams with the test set so the NMT model can be adapted to translate it. Performing data selection on back-translated data creates new challenges as the source-side may contain noise originated by the model used in the back-translation. Hence, finding ngrams present in the test set become more difficult. Despite that, in our work we show that adapting a model with a selection of synthetic data is an useful approach

Biliary Bicarbonate Secretion Constitutes a Protective Mechanism against Bile Acid-Induced Injury in Man

Background: Cholangiocytes expose a striking resistance against bile acids: while other cell types, such as hepatocytes, are susceptible to bile acid-induced toxicity and apoptosis already at micromolar concentrations, cholangiocytes are continuously exposed to millimolar concentrations as present in bile. We present a hypothesis suggesting that biliary secretion of HCO(3)(-) in man serves to protect cholangiocytes against bile acid-induced damage by fostering the deprotonation of apolar bile acids to more polar bile salts. Here, we tested if bile acid-induced toxicity is pH-dependent and if anion exchanger 2 (AE2) protects against bile acid-induced damage. Methods: A human cholangiocyte cell line was exposed to chenodeoxycholate (CDC), or its glycine conjugate, from 0.5 mM to 2.0 mM at pH 7.4, 7.1, 6.7 or 6.4, or after knockdown of AE2. Cell viability and apoptosis were determined by WST and caspase-3/-7 assays, respectively. Results: Glycochenodeoxycholate (GCDC) uptake in cholangiocytes is pH-dependent. Furthermore, CDC and GCDC (pK(a) 4-5) induce cholangiocyte toxicity in a pH-dependent manner: 0.5 mM CDC and 1 mM GCDC at pH 7.4 had no effect on cell viability, but at pH 6.4 decreased viability by >80% and increased caspase activity almost 10- and 30-fold, respectively. Acidification alone had no effect. AE2 knockdown led to 3- and 2-fold enhanced apoptosis induced by 0.75 mM CDC or 2 mM GCDC at pH 7.4. Discussion: These data support our hypothesis of a biliary HCO(3)(-) umbrella serving to protect human cholangiocytes against bile acid-induced injury. AE2 is a key contributor to this protective mechanism. The development and progression of cholangiopathies, such as primary biliary cirrhosis, may be a consequence of genetic and acquired functional defects of genes involved in maintaining the biliary HCO(3)(-) umbrella. Copyright (C) 2011 S. Karger AG, Base

Agglomération et hétéroagglomération des nanoparticules d'argent en eaux douces

Les nanomatériaux sont une classe de contaminants qui est de plus en plus présent dans l’environnement. Leur impact sur l’environnement dépendra de leur persistance, mobilité, toxicité et bioaccumulation. Chacun de ces paramètres dépendra de leur comportement physicochimique dans les eaux naturelles (i.e. dissolution et agglomération). L’objectif de cette étude est de comprendre l’agglomération et l’hétéroagglomération des nanoparticules d’argent dans l’environnement. Deux différentes sortes de nanoparticules d’argent (nAg; avec enrobage de citrate et avec enrobage d’acide polyacrylique) de 5 nm de diamètre ont été marquées de manière covalente à l’aide d’un marqueur fluorescent et ont été mélangées avec des colloïdes d’oxyde de silice (SiO2) ou d’argile (montmorillonite). L’homo- et hétéroagglomération des nAg ont été étudiés dans des conditions représentatives d’eaux douces naturelles (pH 7,0; force ionique 10 7 à 10-1 M de Ca2+). Les tailles ont été mesurées par spectroscopie de corrélation par fluorescence (FCS) et les résultats ont été confirmés à l’aide de la microscopie en champ sombre avec imagerie hyperspectrale (HSI). Les résultats ont démontrés que les nanoparticules d’argent à enrobage d’acide polyacrylique sont extrêmement stables sous toutes les conditions imposées, incluant la présence d’autres colloïdes et à des forces ioniques très élevées tandis que les nanoparticules d’argent avec enrobage de citrate ont formées des hétéroagrégats en présence des deux particules colloïdales.Nanomaterials are a class of contaminants that are increasingly found in the natural environment. Their environmental risk will depend on their persistence, mobility, toxicity and bioaccumulation. Each of these parameters will depend strongly upon their physicochemical fate (dissolution, agglomeration) in natural waters. The goal of this paper is to understand the agglomeration and heteroagglomeration of silver nanoparticles in the environment. Two different silver nanoparticles (nAg; citrate coated and polyacrylic acid coated) with a diameter of 5 nm were covalently labelled with a fluorescent dye and then mixed with colloidal silicon oxides (SiO2) and clays (montmorillonite). The homo- and heteroagglomeration of the silver nanoparticles were then studied in waters that were representative of natural freshwaters (pH 7.0; ionic strength 10-7 to 10-1 M of Ca2+). Sizes were followed by fluorescence correlation spectroscopy (FCS) and results were validated using enhanced darkfield microscopy with hyperspectral imaging (HSI). Results have demonstrated that the polyacrylic acid coated nAg was extremely stable under all conditions, including in the presence of other colloids and at high ionic strength, whereas the citrate coated nAg formed heteroagregates in the presence of both natural colloidal particles