6 research outputs found
An Effective Mixture-Of-Experts Approach For Code-Switching Speech Recognition Leveraging Encoder Disentanglement
With the massive developments of end-to-end (E2E) neural networks, recent
years have witnessed unprecedented breakthroughs in automatic speech
recognition (ASR). However, the codeswitching phenomenon remains a major
obstacle that hinders ASR from perfection, as the lack of labeled data and the
variations between languages often lead to degradation of ASR performance. In
this paper, we focus exclusively on improving the acoustic encoder of E2E ASR
to tackle the challenge caused by the codeswitching phenomenon. Our main
contributions are threefold: First, we introduce a novel disentanglement loss
to enable the lower-layer of the encoder to capture inter-lingual acoustic
information while mitigating linguistic confusion at the higher-layer of the
encoder. Second, through comprehensive experiments, we verify that our proposed
method outperforms the prior-art methods using pretrained dual-encoders,
meanwhile having access only to the codeswitching corpus and consuming half of
the parameterization. Third, the apparent differentiation of the encoders'
output features also corroborates the complementarity between the
disentanglement loss and the mixture-of-experts (MoE) architecture.Comment: ICASSP 202
Language-specific Acoustic Boundary Learning for Mandarin-English Code-switching Speech Recognition
Code-switching speech recognition (CSSR) transcribes speech that switches
between multiple languages or dialects within a single sentence. The main
challenge in this task is that different languages often have similar
pronunciations, making it difficult for models to distinguish between them. In
this paper, we propose a method for solving the CSSR task from the perspective
of language-specific acoustic boundary learning. We introduce language-specific
weight estimators (LSWE) to model acoustic boundary learning in different
languages separately. Additionally, a non-autoregressive (NAR) decoder and a
language change detection (LCD) module are employed to assist in training.
Evaluated on the SEAME corpus, our method achieves a state-of-the-art mixed
error rate (MER) of 16.29% and 22.81% on the test_man and test_sge sets. We
also demonstrate the effectiveness of our method on a 9000-hour in-house
meeting code-switching dataset, where our method achieves a relatively 7.9% MER
reduction