6 research outputs found
Revisiting Iterative Back-Translation from the Perspective of Compositional Generalization
Human intelligence exhibits compositional generalization (i.e., the capacity
to understand and produce unseen combinations of seen components), but current
neural seq2seq models lack such ability. In this paper, we revisit iterative
back-translation, a simple yet effective semi-supervised method, to investigate
whether and how it can improve compositional generalization. In this work: (1)
We first empirically show that iterative back-translation substantially
improves the performance on compositional generalization benchmarks (CFQ and
SCAN). (2) To understand why iterative back-translation is useful, we carefully
examine the performance gains and find that iterative back-translation can
increasingly correct errors in pseudo-parallel data. (3) To further encourage
this mechanism, we propose curriculum iterative back-translation, which better
improves the quality of pseudo-parallel data, thus further improving the
performance.Comment: accepted in AAAI 202