It is challenging to build a multi-singer high-fidelity singing voice
synthesis system with cross-lingual ability by only using monolingual singers
in the training stage. In this paper, we propose CrossSinger, which is a
cross-lingual singing voice synthesizer based on Xiaoicesing2. Specifically, we
utilize International Phonetic Alphabet to unify the representation for all
languages of the training data. Moreover, we leverage conditional layer
normalization to incorporate the language information into the model for better
pronunciation when singers meet unseen languages. Additionally, gradient
reversal layer (GRL) is utilized to remove singer biases included in lyrics
since all singers are monolingual, which indicates singer's identity is
implicitly associated with the text. The experiment is conducted on a
combination of three singing voice datasets containing Japanese Kiritan
dataset, English NUS-48E dataset, and one internal Chinese dataset. The result
shows CrossSinger can synthesize high-fidelity songs for various singers with
cross-lingual ability, including code-switch cases.Comment: Accepted by ASRU202