1 research outputs found
Multilingual Byte2Speech Models for Scalable Low-resource Speech Synthesis
To scale neural speech synthesis to various real-world languages, we present
a multilingual end-to-end framework that maps byte inputs to spectrograms, thus
allowing arbitrary input scripts. Besides strong results on 40+ languages, the
framework demonstrates capabilities to adapt to new languages under extreme
low-resource and even few-shot scenarios of merely 40s transcribed recording,
without the need of per-language resources like lexicon, extra corpus,
auxiliary models, or linguistic expertise, thus ensuring scalability. While it
retains satisfactory intelligibility and naturalness matching rich-resource
models. Exhaustive comparative and ablation studies are performed to reveal the
potential of the framework for low-resource languages. Furthermore, we propose
a novel method to extract language-specific sub-networks in a multilingual
model for a better understanding of its mechanism.Comment: 17 page