32 research outputs found
Leveraging Auxiliary Domain Parallel Data in Intermediate Task Fine-tuning for Low-resource Translation
NMT systems trained on Pre-trained Multilingual Sequence-Sequence (PMSS)
models flounder when sufficient amounts of parallel data is not available for
fine-tuning. This specifically holds for languages missing/under-represented in
these models. The problem gets aggravated when the data comes from different
domains. In this paper, we show that intermediate-task fine-tuning (ITFT) of
PMSS models is extremely beneficial for domain-specific NMT, especially when
target domain data is limited/unavailable and the considered languages are
missing or under-represented in the PMSS model. We quantify the domain-specific
results variations using a domain-divergence test, and show that ITFT can
mitigate the impact of domain divergence to some extent.Comment: Accepted for poster presentation at the Practical Machine Learning
for Developing Countries (PML4DC) workshop, ICLR 202