Unsupervised Neural Machine Translation with SMT as Posterior Regularization

Without real bilingual corpus available, unsupervised Neural Machine Translation (NMT) typically requires pseudo parallel data generated with the back-translation method for the model training. However, due to weak supervision, the pseudo data inevitably contain noises and errors that will be accumulated and reinforced in the subsequent training process, leading to bad translation performance. To address this issue, we introduce phrase based Statistic Machine Translation (SMT) models which are robust to noisy data, as posterior regularizations to guide the training of unsupervised NMT models in the iterative back-translation process. Our method starts from SMT models built with pre-trained language models and word-level translation tables inferred from cross-lingual embeddings. Then SMT and NMT models are optimized jointly and boost each other incrementally in a unified EM framework. In this way, (1) the negative effect caused by errors in the iterative back-translation process can be alleviated timely by SMT filtering noises from its phrase tables; meanwhile, (2) NMT can compensate for the deficiency of fluency inherent in SMT. Experiments conducted on en-fr and en-de translation tasks show that our method outperforms the strong baseline and achieves new state-of-the-art unsupervised machine translation performance.Comment: To be presented at AAAI 2019; 9 pages, 4 figure

From R_AA via correlations to jets - the long road to tomography

The main motivation to investigate hard probes in heavy ion collisions is to do tomography, i.e. to infer medium properties from the in-medium modification of hard processes. Yet while the suppression of high P_T hadrons has been measured for some time, solid tomographic information is slow to emerge. This can be traced back to theoretical uncertainties and ambiguities in modelling both medium evolution and parton-medium interaction. Ways to overcome these difficulties are to constrain models better and to focus on more differential observables. Correlations of high P_T hadrons offer non-trivial information beyond what can be deduced from single hadron suppression. They reflect not only the hard reaction being modified by the medium, but also the back reaction of the medium to the hard probe. Models for hard back-to-back correlations are now very well constrained by a wealth of data and allow insights into the nature of the parton-medium interaction as well as first true tomographic results. Models of full in-medium jet evolution are being actively developed, but have yet to make substantial contact with data. Progress is slower in the understanding of low P_T correlations, the ridge and the cone, although a qualitative understanding of the nature of the physics behind these correlations starts to emerge.Comment: 8 pages, 3 figures- To appear in the conference proceedings for Quark Matter 2009, March 30 - April 4, Knoxville, Tennesse

Alternating Back-Propagation for Generator Network

This paper proposes an alternating back-propagation algorithm for learning the generator network model. The model is a non-linear generalization of factor analysis. In this model, the mapping from the continuous latent factors to the observed signal is parametrized by a convolutional neural network. The alternating back-propagation algorithm iterates the following two steps: (1) Inferential back-propagation, which infers the latent factors by Langevin dynamics or gradient descent. (2) Learning back-propagation, which updates the parameters given the inferred latent factors by gradient descent. The gradient computations in both steps are powered by back-propagation, and they share most of their code in common. We show that the alternating back-propagation algorithm can learn realistic generator models of natural images, video sequences, and sounds. Moreover, it can also be used to learn from incomplete or indirect training data

Joint Training for Neural Machine Translation Models with Monolingual Data

Monolingual data have been demonstrated to be helpful in improving translation quality of both statistical machine translation (SMT) systems and neural machine translation (NMT) systems, especially in resource-poor or domain adaptation tasks where parallel data are not rich enough. In this paper, we propose a novel approach to better leveraging monolingual data for neural machine translation by jointly learning source-to-target and target-to-source NMT models for a language pair with a joint EM optimization method. The training process starts with two initial NMT models pre-trained on parallel data for each direction, and these two models are iteratively updated by incrementally decreasing translation losses on training data. In each iteration step, both NMT models are first used to translate monolingual data from one language to the other, forming pseudo-training data of the other NMT model. Then two new NMT models are learnt from parallel data together with the pseudo training data. Both NMT models are expected to be improved and better pseudo-training data can be generated in next step. Experiment results on Chinese-English and English-German translation tasks show that our approach can simultaneously improve translation quality of source-to-target and target-to-source models, significantly outperforming strong baseline systems which are enhanced with monolingual data for model training including back-translation.Comment: Accepted by AAAI 201

Age and growth of spiny dogfish (Squalus acanthias) in the Gulf of Alaska: analysis of alternative growth models

Ten growth models were fitted to age and growth data for spiny dogfish (Squalus acanthias) in the Gulf of Alaska. Previous studies of spiny dogfish growth have all fitted the t0 formulation of the von Bertalanffy model without examination of alternative models. Among the alternatives, we present a new two-phase von Bertalanffy growth model formulation with a logistically scaled k parameter and which estimates L0. A total of 1602 dogfish were aged from opportunistic collections with longline, rod and reel, set net, and trawling gear in the eastern and central Gulf of Alaska between 2004 and 2007. Ages were estimated from the median band count of three independent readings of the second dorsal spine plus the estimated number of worn bands for worn spines. Owing to a lack of small dogfish in the samples, lengths at age of small individuals were back-calculated from a subsample of 153 dogfish with unworn spines. The von Bertalanffy, two-parameter von Bertalanffy, two-phase von Bertalanffy, Gompertz, two-parameter Gompertz, and logistic models were fitted to length-at-age data for each sex separately, both with and without back-calculated lengths at age. The two-phase von Bertalanffy growth model produced the statistically best fit for both sexes of Gulf of Alaska spiny dogfish, resulting in L∞ = 87.2 and 102.5 cm and k= 0.106 and 0.058 for males and females, respectively

Enhanced suffix arrays as language models: Virtual k-testable languages

In this article, we propose the use of suffix arrays to efficiently implement n-gram language models with practically unlimited size n. This approach, which is used with synchronous back-off, allows us to distinguish between alternative sequences using large contexts. We also show that we can build this kind of models with additional information for each symbol, such as part-of-speech tags and dependency information. The approach can also be viewed as a collection of virtual k-testable automata. Once built, we can directly access the results of any k-testable automaton generated from the input training data. Synchronous back- off automatically identies the k-testable automaton with the largest feasible k. We have used this approach in several classification tasks