Recent work at the Centre for Speech Technology Re- search (CSTR) at the University of Edinburgh has de- veloped an accent-independent lexicon for speech synthesis (the Unisyn project). The main purpose of this lexicon is to avoid the problems and cost of writing a new lexicon for every new accent needed for synthesis. Only recently, a first attempt has been made to use the Keyword Lexicon for automatic speech recognition

King, Simon

Van Bael, Christophe

Edinburgh Research Archive

The Keyword Lexicon - An accent-independent lexiconfor automatic speech recognitionChristophe Van Bael∗ and Simon King†∗ A2RT, Department of Language and Speech, University of Nijmegen, The NetherlandsC.v.Bael@let.kun.nl† Centre for Speech Technology Research (CSTR), University of Edinburgh, UKSimon.King@ed.ac.ukABSTRACTRecent work at the Centre for Speech Technology Re-search (CSTR) at the University of Edinburgh has de-veloped an accent-independent lexicon for speech syn-thesis (the Unisyn project). The main purpose of thislexicon is to avoid the problems and cost of writing anew lexicon for every new accent needed for synthesis.Only recently [1], a first attempt has been made touse the Keyword Lexicon for automatic speech recog-nition.1 INTRODUCTIONModelling pronunciation variation is a hot topic inspeech recognition. Previous attempts to handle pro-nunciation variation at the lexical level, such as [2, 3],largely focused on adding multiple pronunciations perlexeme in the lexica to fit the acoustic data better.Even though it has been proven that this procedurecan improve recognition accuracy, it might also lead toa higher degree of lexical confusability, and hence tohigher Word Error Rates (WERs).Assuming that speech characteristics are (at least ap-proximately) known for the training data, the testdata or both, the confusability problem may be partlysolved by generating accent- or speaker-specific lex-ica. Our approach exploits the fact that there is atleast some invariability in a speaker’s accent or inseveral speakers’ accents. For example, when a per-son starts talking about /b A: T/ (SAMPA-notationis used throughout this article), the chances are highthat person will prefer the pronunciation /p A: T/ over/p { T/. This consistency may be exhibited within onespeaker or a group of speakers such as those from aparticular town, region or country. As accent-specificlexica covering these consistencies can be easily builtwith the Keyword Lexicon, the main aim of this re-search was investigating its abilities to model pronun-ciation variation for ASR.Town hierarchiesper ruleTranscriptionskeysymbols(underlyingrepresentation)surface KEYWORD LEXICON RULES APPLIEDusingrule 1rule 2rule nLexicon Brepresentation countryregiontownspeakerscores1010RULES DEFINEDLexicon AFigure 1: The Keyword Lexicon - Survey1.1 The Keyword LexiconThe Keyword Lexicon [4, 5] (figure 1) is an abstractlexicon consisting of keyvowels and keyconsonants fol-lowing the original keyvowel idea of Wells [6]. Eachkeysymbol defines a phoneme in a class of words pro-nounced in a similar way within any accent of English.Hence, the keyvowel A defines the vowel in a group ofwords containing both path and bath. In the examplegiven, this means that in Leeds path and bath will bothbe pronounced with the short vowel /{/, hence /p { T/and /b { T/. In a typical Southern English accent how-ever, those words will both be pronounced with thelong vowel /A:/, hence /p A: T/ and /b A: T/.Accent-specific lexica can be built by applying hierar-chically ordered rewrite rules to the underlying Key-word Lexicon. Figure 2 shows how different lexica canbe built by defining whether the rewrite rules apply atthe country, region, town and person level. If a ruleapplies (score 1 in the figure) at a higher level, thatrule will be applied when building lexica for that andall underlying levels. Rule scores from higher levelscan be explicitly overriden, though, at lower levels.Hence, in figure 2, the h-drop rule, stating that in somephonetic contexts in some accents of English an ini-tial /h/ may not be pronounced, is not applied to theunderlying Keyword Lexicon when building a North-ern English (N ENG) lexicon, while it does apply forall speakers in Newcastle (Newc.), except for speakerNewc.1.KEYWORD LEXICONdo_h_drop. . .                          { h * o t }           $ rules {do_h_drop} RULES APPLIEDRULES DEFINED{ h * o t }Lexicon Newc.1           {REG}   {N_ENG} =  0           {PER}   {Newc.1}  =  0  { * o t }          hot { h * o t }                       key         surface{ * o t }Lexicon Newc.{ h * o t }Lexicon N_ENG          {TWN}  {Newc.}    =  1 Figure 2: The Keyword Lexicon - Method2 MATERIAL AND METHOD2.1 MaterialWSJCAM0 [7], the British English version of part ofthe Wall Street Journal (WSJ) corpus, was used fortraining, tuning and testing. The training set com-prised 7860 utterances, the tuning set (to determinethe optimal language model scaling factor to scale theinfluence of the language model and that of the phonemodels with regard to each other, the optimal wordinsertion log probability to control insertions and dele-tions, and the optimal pruning) 200 utterances, andthe test set 1090 utterances, altogether speech of 140subjects speaking with various British English accentsand defining a 5K closed vocabulary task.The recogniser was built using Hidden Markov Mod-els (HMMs) with HTK [8]. Word internal tied-state context-dependent models with multiple Gaus-sian mixture components per state were used. A stan-dard backed-off bigram language model was used atrecognition time.The speech files were sampled at 16kHz and 16 bitsper sample. They were parameterised as MFCCs with12 coefficients to capture the spectral properties ofthe waveform, one coefficient to capture the ampli-tude, 13 delta and 13 acceleration coefficients. A stan-dard left-right 3-state topology was maintained for allphone models and a separate silence model, except fora 1-state left-right short pause model used to modelthe optional silence between words. As no bootstrapdata were available, all phone models were built fromscratch.The British English Example Pronunciation (BEEP)dictionary was used. It is a typical multiple pronunci-ation dictionary of the type usually used for ASR. OutOf Vocabulary words (OOVs) were inserted from theUS English Carnegie Mellon University (CMU) mul-tiple pronunciation dictionary after a suitable phoneset mapping. The use of the CMU dictionary in thiscontext was clearly a suboptimal solution as the CMUlexicon is an American English lexicon and as BEEPand the WSJCAM0 task cover British English. Due totime restrictions, though, this proved to be the easiestway for us to cover the OOVs.2.2 MethodThe focus of this study is the lexicon. In all experi-ments, the lexica and the phonetic transcriptions de-rived from them were the only variables. This meansthat the data, the language model and the recognitiontask as a whole remained the same at all times.To test whether the Keyword Lexicon was suitable forrecognition, its performance was compared to a base-line. This baseline represents the traditional approachof modelling pronunciation variation in ASR at the lex-ical level: the use of lexica with multiple pronuncia-tions per lexical entry (multiple pronunciation lexica).This baseline was obtained using the BEEP lexicon.The same recognition task was then performed witha conglomerate of all 8 pre-defined English lexica cov-ered by the rule-sets in the first release of the KeywordLexicon. This will be called the merged keyword lexiconhereafter. In this way the Keyword Lexicon was testedas a multiple pronunciation lexicon, thus neglecting itspossible gain for pronunciation variation modelling (itsability to easily create accent-specific lexica) for thetime being.After testing the Keyword Lexicon’s inherent suitabil-ity for ASR by using it as a multiple pronunciationlexicon, its true contribution to ASR was investigatedin 2 experiments using the Lexicon as an abstract lex-icon from which accent- or speaker-specific lexica weregenerated. In the previous experiments the same lexi-con was used both for training (to generate phone tran-scriptions from the training data through a lookup pro-cedure) and for recognition (as a top-down constraint).However, the lexica used for training and recognitionmay also differ per experiment. In that way more spe-cific lexica can be used that better fit the train andtest data. We adopted this procedure in the followingexperiments.In the first experiment testing the Keyword Lexicon’squality and ease of use, the merged keyword lexiconwas used for training, while at recognition time accent-specific lexica generated from the Keyword Lexiconwere used. In the second experiment the trainingspeakers were roughly divided into 7 accent groups,and for each one of these groups, an accent-specific lexi-con was generated. These lexica were used to train theacoustic models with, while the same accent-specificlexica of the previous experiment were used to test therecogniser.3 BASELINE AND PRELIMINARYEXPERIMENT3.1 Baseline: recognition with the BEEP lexi-con1512 words of the training corpus and 66 words of thetest corpus did not appear in BEEP. Therefore, the(multiple) pronunciations of these words were insertedfrom the CMU lexicon. The total phone set comprised46 phones. The least frequent phone occured 57 timesin the training lexicon, the most frequent phone 5583times. The first baseline was 31.4% WER.3.2 Testing the Keyword Lexicon as a multiplepronunciation lexiconThe merged keyword lexicon (see 2.2) was used bothfor training and testing the recogniser. Again, themissing words were inserted from the CMU dictio-nary. The total phone set, however, now comprised 83phones. This is a large set, given that the same amountof training data was available as for the baseline experi-ment (with the BEEP lexicon and only 46 phones). Allphones were needed, though, to capture the most im-portant pronunciation variants in the merged keywordlexicon. After conducting the same recognition taska WER of 32.5% was obtained. A two-tailed t-testproved that this WER was not significantly differentfrom the 31.4% obtained when testing with the BEEPlexicon.4 TESTING THE KEYWORDLEXICONFor these experiments, 7 accent-specific lexica werebuilt from the Keyword Lexicon to cover the trainingdata, 4 accent-specific lexica to cover the developmenttest set and 7 more accent-specific lexica to cover thespeakers from the test set not covered by any of theother lexica. The accents covered are presented in ta-ble 1. Whereas the division of the training speakersin accent-groups was a coarse one based on geographi-cal information of the speakers, the division of the testspeakers into accent groups was based on the judge-ments of two expert listeners.Train set Tuning set Test setSouthern, Northern, Southern, Northern, RP 1,2,3,4Central 1, Western, Central 1, RP 1 Central 1,2,3,4Welsh, Irish, Scottish Western, Irish,Scottish (Edi)Table 1: Accent groups in the data sets.4.1 Training with a multiple pronunciation lex-icon, testing with accent-specific lexicaTraining was performed with the merged keyword lex-icon, as in 3.2. For recognition, the test speakers weredivided in 11 accent-groups, and per accent-group anaccent-specific lexicon was generated. With a WER of39.7%, in this experiment the use of the Keyword Lexi-con led to a decrease in accuracy compared to the sametask performed with multiple pronunciation lexica.4.2 Training and testing with accent-specificlexicaFor this experiment, the training data were subdividedin 7 accent groups, and new acoustic models weretrained from phonetic transcriptions generated with 7accent-specific lexica. However, as the developmenttest set only comprised 3 different accents whereasthe test set comprised 11 different accents, and as therecogniser had to be tuned again because new phonemodels were used, in this and in the following experi-ment the recogniser was tuned on the test set. There-fore we are careful interpreting the results, but we be-lieve that the 32.5% WER obtained in this experimentat least indicates that the use of the Keyword Lexiconcan lead to equally good recognition results as the useof multiple pronunciation lexica.5 DISCUSSIONIn the first experiment BEEP was used both for train-ing and for recognition. Figure 3 illustrates the recog-niser’s baseline performance (31.4% WER) on the sep-arate accent-groups.051015202530354045scot(edi)rp2 mid4 rp1 rp3 mid3 mid1 rp4 west mid2 irishAccentsWER (%)Figure 3: Baseline: accent-specific resultsThe second experiment tested the Keyword lexicon asa multiple pronunciation lexicon. The performance(32.5% WER) was not significantly different from thebaseline. When comparing the results of the baselineexperiment and the results of this experiment per ac-cent, the WERs resembled. Therefore we conclude thatthe Keyword Lexicon is a worthy equivalent of BEEPwhen used as a multiple pronunciation lexicon for ASR.Using a multiple pronunciation lexicon for training andaccent-specific lexica for recognition resulted in a sig-nificantly worse performance (39.7% WER) than thebaseline. Again a similar distribution was discoveredwhen comparing this performance with the baseline ata speaker- and accent level. The results could meanthree things: 1) the pooled training lexicon wasn’t suit-able to train phone models (either it didn’t fit the dataor the degree of confusability between the differentpronunciation variants was too high), 2) the accent-specific test lexica didn’t fit the data or 3) our newapproach to ASR proposed here had to be reconsid-ered.Finally, accent-specific lexica were used both for train-ing and for recognition. Using accent-specific traininglexica does not involve any extra effort in our approach,as those lexica can be made swiftly based on informa-tion of the training speakers. Now a WER of 32.5% wasobtained. Putting more time and effort in determin-ing the training speakers’accents and preferably eventheir personal characteristics and incorporating thisknowledge into speaker-specific rule sets to generatemore specific training lexica, might further improve therecogniser’s performance, thus outperforming multiplepronunciation lexica on the same task. Even thoughone has to be careful interpreting these results (in thisexperiment the recogniser was tuned on the test set),we conclude that the problem in the previous experi-ment was the multiple pronunciation training lexicon,and not the accent-specific test lexica, nor our new ap-proach as such. This experiment proves that our newapproach can at least compete with the use of multiplepronunciation lexica on the same task.6 CONCLUSIONWe have introduced a new approach to model pronun-ciation variation for ASR at the lexical level. Untilnow, research has largely focussed on creating largemultiple pronunciation lexica for speech recognition.However, when too many pronunciation variants areincluded, the confusability increases, and the recog-niser’s performance decreases. Our new approach in-troduces an abstract lexicon from which accent- or evenspeaker-specific training and test lexica can be gener-ated via rule-settings that can be defined with greatease.The experiments showed that the Keyword Lexiconserving as a multiple pronunciation lexicon performsequally well as a standard multiple pronunciation lex-icon like the BEEP dictionary. Moreover, using stillquite general accent-specific lexica both for trainingand for recognition gave a similar performance on thesame task. We believe that adding more time and ef-fort in building speaker-specific training lexica mighteasily improve the recognition accuracy, without as-suming more knowledge of the test speaker’s speechcharacteristics.ACKNOWLEDGEMENTSThe research reported was performed as part of a Mas-ters program between May and September 2002 at theDepartment of Theoretical and Applied Linguistics atthe University of Edinburgh, UK.REFERENCES[1] C.P.J. Van Bael, “Using the Keyword Lexi-con for speech recognition,” M.Sc. thesis, TheDepartment of Theoretical and Applied Linguis-tics, University of Edinburgh, 2002, available at:http://lands.let.kun.nl/literature/vanbael.2002.2.pdf.[2] J.M. Kessens, Making a difference. On automatictranscription and modeling of Dutch pronunciationvariation for automatic speech recognition, Ph.D.thesis, University of Nijmegen, The Netherlands,2002.[3] M. Wester, Pronunciation variation modeling forDutch Automatic Speech Recognition, Ph.D. thesis,University of Nijmegen, The Netherlands, 2002.[4] S. Fitt and S. Isard, “Synthesis of regional Englishusing a keyword lexicon,” in Eurospeech 99, 1999.[5] S. Fitt, “Documentation and user guide to UNISYNlexicon and post-lexical rules,” Tech. Rep., Centrefor Speech Technology Research, University of Ed-inburgh, 2000.[6] J.C. Wells, Accents of English, Cambridge Univer-sity Press, Cambridge, 1982.[7] J. Fransen et al., “WSJCAM0 corpus and de-scription,” Tech. Rep. CUED/F-INFENG/TR.192,Cambridge University Engineering Department(CUED) Speech Group, Trumpington Street Cam-bridge CB2 1PZ U.K., 1994.[8] S. Young et al., “The HTK book (for HTK version3.1),” Tech. Rep., Cambridge University Engineer-ing Department, 2001.

An accent-independent lexicon for automatic speech recognition.

Accents of English,

Documentation and user guide to UNISYN lexicon and post-lexical rules,”

Making a diﬀerence. On automatic transcription and modeling of Dutch pronunciation variation for automatic speech recognition,

Pronunciation variation modeling for Dutch Automatic Speech Recognition,

Synthesis of regional English using a keyword lexicon,”

The HTK book (for HTK version 3.1),” Tech. Rep.,

Using the Keyword Lexicon for speech recognition,” M.Sc. thesis, The Department of Theoretical and Applied Linguistics,

WSJCAM0 corpus and description,”

https://era.ed.ac.uk/bitstream/1842/1094/1/VanBael_King_icphs2003.pdf

An accent-independent lexicon for automatic speech recognition.

Abstract

Similar works

Full text

Available Versions

Edinburgh Research Archive