1 research outputs found
Modeling Future Cost for Neural Machine Translation
Existing neural machine translation (NMT) systems utilize
sequence-to-sequence neural networks to generate target translation word by
word, and then make the generated word at each time-step and the counterpart in
the references as consistent as possible. However, the trained translation
model tends to focus on ensuring the accuracy of the generated target word at
the current time-step and does not consider its future cost which means the
expected cost of generating the subsequent target translation (i.e., the next
target word). To respond to this issue, we propose a simple and effective
method to model the future cost of each target word for NMT systems. In detail,
a time-dependent future cost is estimated based on the current generated target
word and its contextual information to boost the training of the NMT model.
Furthermore, the learned future context representation at the current time-step
is used to help the generation of the next target word in the decoding.
Experimental results on three widely-used translation datasets, including the
WMT14 German-to-English, WMT14 English-to-French, and WMT17 Chinese-to-English,
show that the proposed approach achieves significant improvements over strong
Transformer-based NMT baseline