In this paper we study the impact of using
images to machine-translate user-generated ecommerce product listings. We study how
a multi-modal Neural Machine Translation
(NMT) model compares to two text-only approaches: a conventional state-of-the-art attentional NMT and a Statistical Machine Translation (SMT) model. User-generated product
listings often do not constitute grammatical
or well-formed sentences. More often than
not, they consist of the juxtaposition of short
phrases or keywords. We train our models
end-to-end as well as use text-only and multimodal NMT models for re-ranking n-best lists
generated by an SMT model. We qualitatively evaluate our user-generated training data
also analyse how adding synthetic data impacts the results. We evaluate our models
quantitatively using BLEU and TER and find
that (i) additional synthetic data has a general
positive impact on text-only and multi-modal
NMT models, and that (ii) using a multi-modal
NMT model for re-ranking n-best lists improves TER significantly across different nbest list sizes