12 research outputs found
Cross Lingual Transfer Learning for Zero-Resource Domain Adaptation
We propose a method for zero-resource domain adaptation of DNN acoustic
models, for use in low-resource situations where the only in-language training
data available may be poorly matched to the intended target domain. Our method
uses a multi-lingual model in which several DNN layers are shared between
languages. This architecture enables domain adaptation transforms learned for
one well-resourced language to be applied to an entirely different low-resource
language. First, to develop the technique we use English as a well-resourced
language and take Spanish to mimic a low-resource language. Experiments in
domain adaptation between the conversational telephone speech (CTS) domain and
broadcast news (BN) domain demonstrate a 29% relative WER improvement on
Spanish BN test data by using only English adaptation data. Second, we
demonstrate the effectiveness of the method for low-resource languages with a
poor match to the well-resourced language. Even in this scenario, the proposed
method achieves relative WER improvements of 18-27% by using solely English
data for domain adaptation. Compared to other related approaches based on
multi-task and multi-condition training, the proposed method is able to better
exploit well-resource language data for improved acoustic modelling of the
low-resource target domain.Comment: Submitted to ICASSP 2020. Main updates wrt previous versions: same
network config in all experiments, added Babel/Material LR target language
experiments, added comparison with alternative/similar methods of
cross-lingual adaptatio
Phonetic Error Analysis Beyond Phone Error Rate
In this article, we analyse the performance of the TIMIT-based phone recognition systems beyond the overall phone error rate (PER) metric. We consider three broad phonetic classes (BPCs): {affricate, diphthong, fricative, nasal, plosive, semi-vowel, vowel, silence}, {consonant, vowel, silence} and {voiced, unvoiced, silence} and, calculate the contribution of each phonetic class in terms of the substitution, deletion, insertion and PER. Furthermore, for each BPC we investigate the following: evolution of PER during training, effect of noise (NTIMIT), importance of different spectral subbands (1, 2, 4, and 8 kHz), usefulness of bidirectional vs unidirectional sequential modelling, transfer learning from WSJ and regularisation via monophones. In addition, we construct a confusion matrix for each BPC and analyse the confusions via dimensionality reduction to 2D at the input (acoustic features) and output (logits) levels of the acoustic model. We also compare the performance and confusion matrices of the BLSTM-based hybrid baseline system with those of the GMM-HMM based hybrid, Conformer and wav2vec 2.0 based end-to-end phone recognisers. Finally, the relationship of the unweighted and weighted PERs with the broad phonetic class priors is studied for both the hybrid and end-to-end systems
The Edinburgh International Accents of English Corpus: Towards the Democratization of English ASR
English is the most widely spoken language in the world, used daily by
millions of people as a first or second language in many different contexts. As
a result, there are many varieties of English. Although the great many advances
in English automatic speech recognition (ASR) over the past decades, results
are usually reported based on test datasets which fail to represent the
diversity of English as spoken today around the globe. We present the first
release of The Edinburgh International Accents of English Corpus (EdAcc). This
dataset attempts to better represent the wide diversity of English,
encompassing almost 40 hours of dyadic video call conversations between
friends. Unlike other datasets, EdAcc includes a wide range of first and
second-language varieties of English and a linguistic background profile of
each speaker. Results on latest public, and commercial models show that EdAcc
highlights shortcomings of current English ASR models. The best performing
model, trained on 680 thousand hours of transcribed data, obtains an average of
19.7% word error rate (WER) -- in contrast to the 2.7% WER obtained when
evaluated on US English clean read speech. Across all models, we observe a drop
in performance on Indian, Jamaican, and Nigerian English speakers. Recordings,
linguistic backgrounds, data statement, and evaluation scripts are released on
our website (https://groups.inf.ed.ac.uk/edacc/) under CC-BY-SA license.Comment: Accepted to IEEE ICASSP 202
Why is My Social Robot so Slow? How a Conversational Listener can Revolutionize Turn-Taking
Current machine dialog systems are predominantly implemented using a sequential, utterance based, two-party, speak-wait/speak-wait approach. human-human dialog is 1) not sequential, with overlap, interruption and back channels; 2) processes utterances before they are complete and 3) are often multi-party. The current approach is stifling innovation in social robots were long delays(often several seconds) is the current norm for dialog response time, leading to stilted and unnatural dialog flow. In this paper, by referencing a light weight word spotting speech recognition system - Chatty SDK, we present a practical engineering strategy for developing what we term a conversational listener that would allow systems to mimic natural human turn-taking in dialogue