Generate-then-rank is a widely used mechanism for text generation, where a
generator produces multiple text candidates and a ranker chooses the best one
among the text candidates. However, existing methods usually train the
generator and the ranker individually, neglecting the mutual feedback that
could further enhance the generation quality. To tackle this limitation, we
propose JGR, a novel joint training algorithm that integrates the generator and
the ranker in a single framework. JGR optimizes the generator with a hybrid
objective that combines data likelihood and ranker reward, and trains the
ranker with a contrastive loss that compares the generator outputs. By
iteratively updating the generator and the ranker, JGR can effectively
harmonize their learning and enhance their quality jointly. We evaluate JGR on
various text generation tasks and demonstrate that it surpasses existing
methods on four public datasets across three common generation scenarios. Our
code and models are publicly available at
https://github.com/microsoft/ProphetNet/tree/master/JGR