12 research outputs found
Unsupervised Neural Dialect Translation with Commonality and Diversity Modeling
As a special machine translation task, dialect translation has two main
characteristics: 1) lack of parallel training corpus; and 2) possessing similar
grammar between two sides of the translation. In this paper, we investigate how
to exploit the commonality and diversity between dialects thus to build
unsupervised translation models merely accessing to monolingual data.
Specifically, we leverage pivot-private embedding, layer coordination, as well
as parameter sharing to sufficiently model commonality and diversity among
source and target, ranging from lexical, through syntactic, to semantic levels.
In order to examine the effectiveness of the proposed models, we collect 20
million monolingual corpus for each of Mandarin and Cantonese, which are
official language and the most widely used dialect in China. Experimental
results reveal that our methods outperform rule-based simplified and
traditional Chinese conversion and conventional unsupervised translation models
over 12 BLEU scores.Comment: AAAI 202
Generating Diverse Translation by Manipulating Multi-Head Attention
Transformer model has been widely used on machine translation tasks and
obtained state-of-the-art results. In this paper, we report an interesting
phenomenon in its encoder-decoder multi-head attention: different attention
heads of the final decoder layer align to different word translation
candidates. We empirically verify this discovery and propose a method to
generate diverse translations by manipulating heads. Furthermore, we make use
of these diverse translations with the back-translation technique for better
data augmentation. Experiment results show that our method generates diverse
translations without severe drop in translation quality. Experiments also show
that back-translation with these diverse translations could bring significant
improvement on performance on translation tasks. An auxiliary experiment of
conversation response generation task proves the effect of diversity as well.Comment: Accepted by AAAI 202