3 research outputs found
Enhanced Neural Machine Translation by Learning from Draft
Neural machine translation (NMT) has recently achieved impressive results. A
potential problem of the existing NMT algorithm, however, is that the decoding
is conducted from left to right, without considering the right context. This
paper proposes an two-stage approach to solve the problem. In the first stage,
a conventional attention-based NMT system is used to produce a draft
translation, and in the second stage, a novel double-attention NMT system is
used to refine the translation, by looking at the original input as well as the
draft translation. This drafting-and-refinement can obtain the right-context
information from the draft, hence producing more consistent translations. We
evaluated this approach using two Chinese-English translation tasks, one with
44k pairs and 1M pairs respectively. The experiments showed that our approach
achieved positive improvements over the conventional NMT system: the
improvements are 2.4 and 0.9 BLEU points on the small-scale and large-scale
tasks, respectively
Neural Networks for Modeling Source Code Edits
Programming languages are emerging as a challenging and interesting domain
for machine learning. A core task, which has received significant attention in
recent years, is building generative models of source code. However, to our
knowledge, previous generative models have always been framed in terms of
generating static snapshots of code. In this work, we instead treat source code
as a dynamic object and tackle the problem of modeling the edits that software
developers make to source code files. This requires extracting intent from
previous edits and leveraging it to generate subsequent edits. We develop
several neural networks and use synthetic data to test their ability to learn
challenging edit patterns that require strong generalization. We then collect
and train our models on a large-scale dataset of Google source code, consisting
of millions of fine-grained edits from thousands of Python developers. From the
modeling perspective, our main conclusion is that a new composition of
attentional and pointer network components provides the best overall
performance and scalability. From the application perspective, our results
provide preliminary evidence of the feasibility of developing tools that learn
to predict future edits.Comment: Deanonymized version of ICLR 2019 submissio
Neural Machine Translation: A Review and Survey
The field of machine translation (MT), the automatic translation of written
text from one natural language into another, has experienced a major paradigm
shift in recent years. Statistical MT, which mainly relies on various
count-based models and which used to dominate MT research for decades, has
largely been superseded by neural machine translation (NMT), which tackles
translation with a single neural network. In this work we will trace back the
origins of modern NMT architectures to word and sentence embeddings and earlier
examples of the encoder-decoder network family. We will conclude with a survey
of recent trends in the field.Comment: Extended version of "Neural Machine Translation: A Review" accepted
by the Journal of Artificial Intelligence Research (JAIR