1 research outputs found
Memory-efficient NLLB-200: Language-specific Expert Pruning of a Massively Multilingual Machine Translation Model
The recently released NLLB-200 is a set of multilingual Neural Machine
Translation models that cover 202 languages. The largest model is based on a
Mixture of Experts architecture and achieves SoTA results across many language
pairs. It contains 54.5B parameters and requires at least four 32GB GPUs just
for inference. In this work, we propose a pruning method that enables the
removal of up to 80% of experts without further finetuning and with a
negligible loss in translation quality, which makes it feasible to run the
model on a single 32GB GPU. Further analysis suggests that our pruning metrics
can identify language-specific experts