133 research outputs found
Unified model for code-switching speech recognition and language identification based on a concatenated tokenizer
Code-Switching (CS) multilingual Automatic Speech Recognition (ASR) models
can transcribe speech containing two or more alternating languages during a
conversation. This paper proposes (1) a new method for creating code-switching
ASR datasets from purely monolingual data sources, and (2) a novel Concatenated
Tokenizer that enables ASR models to generate language ID for each emitted text
token while reusing existing monolingual tokenizers. The efficacy of these
approaches for building CS ASR models is demonstrated for two language pairs,
English-Hindi and English-Spanish, where we achieve new state-of-the-art
results on the Miami Bangor CS evaluation corpus. In addition to competitive
ASR performance, the proposed Concatenated Tokenizer models are highly
effective for spoken language identification, achieving 98%+ accuracy on the
out-of-distribution FLEURS dataset
Multilingual self-supervised speech representations improve the speech recognition of low-resource African languages with codeswitching
While many speakers of low-resource languages regularly code-switch between
their languages and other regional languages or English, datasets of
codeswitched speech are too small to train bespoke acoustic models from scratch
or do language model rescoring. Here we propose finetuning self-supervised
speech representations such as wav2vec 2.0 XLSR to recognize code-switched
data. We find that finetuning self-supervised multilingual representations and
augmenting them with n-gram language models trained from transcripts reduces
absolute word error rates by up to 20% compared to baselines of hybrid models
trained from scratch on code-switched data. Our findings suggest that in
circumstances with limited training data finetuning self-supervised
representations is a better performing and viable solution.Comment: 5 pages, 1 figure. Computational Approaches to Linguistic
Code-Switching, CALCS 2023 (co-located with EMNLP 2023
Code-Switched Urdu ASR for Noisy Telephonic Environment using Data Centric Approach with Hybrid HMM and CNN-TDNN
Call Centers have huge amount of audio data which can be used for achieving
valuable business insights and transcription of phone calls is manually tedious
task. An effective Automated Speech Recognition system can accurately
transcribe these calls for easy search through call history for specific
context and content allowing automatic call monitoring, improving QoS through
keyword search and sentiment analysis. ASR for Call Center requires more
robustness as telephonic environment are generally noisy. Moreover, there are
many low-resourced languages that are on verge of extinction which can be
preserved with help of Automatic Speech Recognition Technology. Urdu is the
most widely spoken language in the world, with 231,295,440 worldwide
still remains a resource constrained language in ASR. Regional call-center
conversations operate in local language, with a mix of English numbers and
technical terms generally causing a "code-switching" problem. Hence, this paper
describes an implementation framework of a resource efficient Automatic Speech
Recognition/ Speech to Text System in a noisy call-center environment using
Chain Hybrid HMM and CNN-TDNN for Code-Switched Urdu Language. Using Hybrid
HMM-DNN approach allowed us to utilize the advantages of Neural Network with
less labelled data. Adding CNN with TDNN has shown to work better in noisy
environment due to CNN's additional frequency dimension which captures extra
information from noisy speech, thus improving accuracy. We collected data from
various open sources and labelled some of the unlabelled data after analysing
its general context and content from Urdu language as well as from commonly
used words from other languages, primarily English and were able to achieve WER
of 5.2% with noisy as well as clean environment in isolated words or numbers as
well as in continuous spontaneous speech.Comment: 32 pages, 19 figures, 2 tables, preprin
Integrating Language Identification to improve Multilingual Speech Recognition
The process of determining the language of a speech utterance is called Language Identification (LID). This task can be very challenging as it has to take into account various language-specific aspects, such as phonetic, phonotactic, vocabulary and grammar-related cues. In multilingual speech recognition we try to find the most likely word sequence that corresponds to an utterance where the language is not known a priori. This is a considerably harder task compared to monolingual speech recognition and it is common to use LID to estimate the current language. In this project we present two general approaches for LID and describe how to integrate them into multilingual speech recognizers. The first approach uses hierarchical multilayer perceptrons to estimate language posterior probabilities given the acoustics in combination with hidden Markov models. The second approach evaluates the output of a multilingual speech recognizer to determine the spoken language. The research is applied to the MediaParl speech corpus that was recorded at the Parliament of the canton of Valais, where people switch from Swiss French to Swiss German or vice versa. Our experiments show that, on that particular data set, LID can be used to significantly improve the performance of multilingual speech recognizers. We will also point out that ASR dependent LID approaches yield the best performance due to higher-level cues and that our systems perform much worse on non-native dat
Development of Bilingual ASR System for MediaParl Corpus
The development of an Automatic Speech Recognition (ASR) system for the bilingual MediaParl corpus is challenging for several reasons: (1) reverberant recordings, (2) accented speech, and (3) no prior information about the language. In that context, we employ frequency domain linear prediction-based (FDLP) features to reduce the effect of reverberation, exploit bilingual deep neural networks applied in Tandem and hybrid acoustic modeling approaches to significantly improve ASR for accented speech and develop a fully bilingual ASR system using entropy-based decoding-graph selection. Our experiments indicate that the proposed bilingual ASR system performs similar to a language-specific ASR system if approximately five seconds of speech are available
Implicit Self-supervised Language Representation for Spoken Language Diarization
In a code-switched (CS) scenario, the use of spoken language diarization (LD)
as a pre-possessing system is essential. Further, the use of implicit
frameworks is preferable over the explicit framework, as it can be easily
adapted to deal with low/zero resource languages. Inspired by speaker
diarization (SD) literature, three frameworks based on (1) fixed segmentation,
(2) change point-based segmentation and (3) E2E are proposed to perform LD. The
initial exploration with synthetic TTSF-LD dataset shows, using x-vector as
implicit language representation with appropriate analysis window length ()
can able to achieve at per performance with explicit LD. The best implicit LD
performance of in terms of Jaccard error rate (JER) is achieved by using
the E2E framework. However, considering the E2E framework the performance of
implicit LD degrades to while using with practical Microsoft CS (MSCS)
dataset. The difference in performance is mostly due to the distributional
difference between the monolingual segment duration of secondary language in
the MSCS and TTSF-LD datasets. Moreover, to avoid segment smoothing, the
smaller duration of the monolingual segment suggests the use of a small value
of . At the same time with small , the x-vector representation is unable
to capture the required language discrimination due to the acoustic similarity,
as the same speaker is speaking both languages. Therefore, to resolve the issue
a self-supervised implicit language representation is proposed in this study.
In comparison with the x-vector representation, the proposed representation
provides a relative improvement of and achieved a JER of using
the E2E framework.Comment: Planning to Submit in IEEE-JSTS
- âŠ