Hild, Hermann

Waibel, Alex

English

KITopen

RECOGNITION OF SPELLED NAMES OVER THE TELEPHONEHermann Hild and Alex Waibelhhild@ira.uka.de,ahw@cs.cmu.eduInteractive Systems LaboratoriesUniversity of Karlsruhe | 76128 Karlsruhe, GermanyCarnegie Mellon University | Pittsburgh, USAABSTRACTRecognition of spelled names over the telephone line is essen-tial for applications such as telephone directory assistance,or automatic mail ordering. We present recognition resultson the spelling section of the OGI Spelled and Spoken WordTelephone Corpus, using a Multi-State Time Delay NeuralNetwork (MS-TDNN). Many applications allow for stronglanguage modeling constraints. In our experiments we ex-amined the benecial eects of reducing the search space toa list of last names, ranging from about 1000 to 14 millionentries. We compare tree search methods and show thatsignicant improvements can be achieved by enriching thesearch trees with probabilities.International Conference on Speech and Language Processing (ICSLP '96),Philadelphia, USA, October 3-6, 19961. INTRODUCTIONThis paper presents recognition results on the OGI Spelledand Spoken Word Telephone Corpus, using a Multi-StateTime Delay Neural Network (MS-TDNN). While it is de-sirable to recognize spelled strings which are embedded inspontaneous speech [1], (\Smith please, thats S-M-I-T-H"),the task here is to recognize letters only. Spelled letters overthe telephone are easily confused. Current systems achievein the order of 90% letter accuracy, which results in string(i.e. name) accuracies far below practical usefullness. For-tunately, many applications, most prominently telephone di-rectory assistance, allow for strong language modeling con-straints. We show how very large, but conceptionally simpletree structured nite state grammars (FSG) can achieve verygood recognition results ranging from 89 to 98% name accu-racy for lists of 1000 up to 14 million last names.2. THE LETTER RECOGNIZERA connectionist recognizer, the Multi-State Time Delay Neu-ral Network (MS-TDNN) [3, 5] is used for connected spelledletter recognition. The MS-TDNN integrates the time-shiftinvariant architecture of a TDNN and a nonlinear time align-ment procedure (DTW) into a word-level classier. Thefront-end TDNN uses sliding windows with time-delayedconnections to compute a score for each phoneme-like statein every frame. Each word to be recognized is modeled bya sequence of phonemes; in a dynamic time warping proce-dure, an optimal alignment path is found for each word. Theactivations along these paths are then collected in the wordoutput units. The error derivatives are backpropagated fromthe word units through the alignment path and the front-end TDNN. For continuous recognition using no or n-gramlanguage models, the standard \one stage dynamic program-ming search" is used. Algorithms for search in nite stategrammars are described below. Training starts with a boot-strapping phase, which involves only the rst three layers ofthe net and establishes a phoneme classication. Then thesystem is trained on \letter level", using the \classicationgure of merit" (CFM) [4] error function for discriminativetraining.3. LANGUAGE MODELSLet S = fs1; : : : ; sNg be a set of names or strings. In thefollowing we examine techniques which conne the recogni-tion to the names in S. The advantage is a high increasein recognition accuracy. The drawback is that names notin S can not be recognized. In [2], a score (interpreted asprobability) is computed for each letter. These scores areused in a tree search to retrieve names from a set of 50,000names. [1] compares several techniques, which use the con-straints either within the search or in a postprocessing stepsuch as nearest neighbor search. In [6], a complex procedureusing a mixture of techniques and several recognition passesis proposed. The approach presented below uses very large,but conceptionally simple nite state grammars (FSG) toexactly represent the names in S.3.1. Search in Finite State GraphsEach word to be recognized is represented by one acousticmodel. Conventional search techniques handle sequences ofwords by concatenating the models as shown to the left ingure 1. In a FSG, word (in our case letter) sequences canbe explicitly coded in a graph structure. Although the sameletters use the same acoustic modeling, it is necessary to keepan individual copy for each node in the tree, since it repre-sents a dierent search history. Each node consumes memoryABZBOBYAYBOBYAFigure 1: Conventional search (left) uses one instance ofeach word model. Search in nite state grammars needs mul-tiple copies, as examplied for a tree (middle) and minimalgraph (right).and computing resources, therefore, the FSG should be keptas small as possible, which can be achieved by constructing aminimal graph. Although a tree can be several times as largeas a minimal graph, it features one major advantage: Sinceeach represented string si is uniquely determined by its nalnode, no backpointers are needed to identify the best lettersequence at the end of the search.The largest tree we are employing uses almost 2 million nodesto represent about 800,000 names, which makes a beamsearch strategy indispensable. A simple scheme is used in thetime synchronous search: If s has been the highest observedscore, all nodes with scores s < s   beam are deactivated.Depending on the beam size and the position in the tree,about 50 - 1000 nodes are active at each frame in time. Thefact that no backpointers are needed keeps the search treevery simple. Essentially, in each node only one cell for eachstate of the corresponding acoustic model is needed to storethe accumulated search score. Scores are forwarded withina node and to successor nodes, as illustrated in gure 2.BAOBFigure 2: Tree search: Acoustic scores are forwarded withinnodes and across nodes.3.2. Search Tree and ProbabilitiesConning the search to a given set S is a strong constraint,but there is one other source of information yet unconsid-ered. Common names such as \Smith" are very frequent,others rare. By counting their relative frequency, a prob-ability p(si) can be assigned to each name si 2 S. P (si)is most naturally incorporated into the tree by associatingeach nal node representing si with p(si). Theoretically, itshould not matter where the probability mass is distributedin the tree, as long as the probability along a path to stringsi accumulates to p(si). However, as the language model(LM) knowledge becomes only available at the very end ofthe search, the beam search may cut o eventually good can-didates too early. An example for a probability assignmentof this and two other methods described below is illustratedin gure 3.3.3. Local ProbabilitiesLet the string si 2 S consist of ni letterssi = li1 li2 : : : liniA partial path li1 li2 : : : lik uniquely denes a node in the tree.We denote the unique transition into this node as ti;k, i.e.ti;k  li1 li2 : : : lik 1ti;k ! likInstead of assigning p(si) to a nal node, we can involvethe LM earlier in the search process by dening a \local"probability local(t) for each transition t:local(ti;k) := p(lik jli1 li2 : : : lik 1)local(ti;k) can be computed as the relative frequency bywhich a path is extended from its parent node into ti;k asopposed to its sibling transitions. We note that the probabil-ities along the path to the nal node representing si correctlyaccumulate to p(si):iniYk=1local(ti;k) = p(li1)  p(li2 jli1 )     p(lini jli1 ::lini 1)= p(li1 li2 : : : lini ) = p(si)3.4. Early ProbabilitiesThe most likely string (i.e. nal node) that can be reachedfrom a partial path can serve as a measure for the potential\importance" of the path. We dene (t) as the highestprobability p(si) which can be reached from a transition t.(:) can be computed by propagating the maxima from theleaves to the root of the tree:(ti;k) :=p(si) k = ni (\nal node")maxt2suc(ti;k)f(t)g elseNow starting from the root, the transition probabilities aredened as the \missing probability" towards p(si):early(ti;k) :=(ti;1) k = 1(ti;k)(ti;k 1)k > 1Again we note that the nal probabilities are correct:iniYk=1early(ti;k) = (ti;1) (ti;2)(ti;1)    (ti;ni)(ti;ni 1)= (ti;ni) = p(si)Figure 3 shows an example for each of the three methods,which in the experiments to be described will be refered toas \nal", \local" and \early". So far we have ignored thata string si can be a prex of another string sj , which invali-dates some of the above formulas. The problem can be xedby using an explicit \end-of-string" marker lini+1 for eachstring si.BOBYY1/21/41/4BOBYY3/42/31/31/4BOBYY1/211/21/4Figure 3: Final (left), local (middle) and early (right) as-signment of probabilities for a tree representing the names(Bob, Boy, By) with the probabilities ( 12;14;14).4. EXPERIMENTAL RESULTS4.1. Data BaseThe \Oregon Graduate Institute (OGI) Spelled and SpokenWord Telephone Corpus" provides recordings from about4000 calls over the public telephone line. Among otherprompts (\What city are you calling from?", \what is yourlast name" etc.), callers were asked to spell their last nameswith and without short pauses between letters (SLP/SLN),their rst names with pauses (SFP), and the alphabet (ALP).About 8% of the spellings contain out-of-alphabet words, e.g."c h e [sorry] c h a v [as in victor] e z". As our letter rec-ognizer can not yet handle such cases, they were excluded1from the experiments. With the partition into training, de-velopment and test set provided with the data base, the dataused for our experiments amounted to the numbers listed intable 1.Set Strings LettersTraining (SLN, SLP, SFP, ALP) 4132 39687Dev. Test (SLN, SLP, SFP, ALP) 2063 15612Test 1 (SLN) 685 4419Test 2 (SLP) 305 1935Table 1: Sizes of Training, Crossvalidation and Test Sets4.2. The Name ListsA set of lists ranging from 1000 to 14 million names wasused to evaluate the tree search procedure. To ensure thatall names in the test set are represented2, we created thelists by lling up the SLN/SLP sets with randomly (withoutreplacement) selected entries from a list of 14 million entries,1together with some very infrequent cut-os. Utterances con-taminated with any of the 6 transcribed noise classes are used inthe training and test sets.2Over 40% of the 800,000 unique names in the 14 million listoccur only once, 49 Names of the SLN test set are not in the list!which was obtained from directory listings from the north-east of the United States. Of course the lists contain manydouble names. The sizes of all lists, the number of uniqueentries and the perplexity of the SLN test set given a treewith and without probabilties is shown in table 2.Total unique PP PP(probs)685 596 3.28 2.651,000 870 3.52 2.792,500 1,990 4.17 3.135,000 4,078 4.50 3.3910,000 7,445 5.20 3.6625,000 15,571 6.11 3.9750,000 26,787 6.89 4.20100,000 44,714 7.71 4.40250,000 85,258 8.93 4.61500,000 135,550 9.93 4.751,000,000 209,301 10.94 4.872,000,000 313,320 11.97 4.974,000,000 452,903 12.93 5.068,000,000 630,718 13.83 5.1314,000,000 807,013 14.53 5.17Table 2: Sizes of the names lists, and perplexity on the SLNtest set using plain trees and trees with probabilities.4.3. Baseline Results16 Melscale FFT coecients are computed every 10 msec.The MS-TDNN uses a hidden layer with 100 units, which cor-responds to a total of only about 34000 parameters (weights).Minimum phoneme duration constraints are computed fromstatistics on the training data and encoded in the acousticletter models. The baseline recognition results using no LM,bi- and trigrams are shown in table 3.4.4. Tree SearchA recognition test with a tree constructed from a list of 1million (209,301 unique) names was performed to comparethe three dierent methods of assigning probabilities (nal,local, early). Figure 4 demonstrates that for small beamsizes, assigning \local" probabilities achieves the best results.As expected, with increasing beam size all three methodsperform equally well, but the recognition becomes more timeexpensive. Figure 5 shows the recognition results on theSLN and SLP sets for lists of various sizes. The usage ofprobabilities in the tree comes with an astonishing perplexityreduction (see table 2), which is also reected in signicantlybetter recognition results.Language Model LA SASLP, No LM 90.6 60.0SLN, No LM 88.2 53.7SLN, Bigrams 91.0 62.8SLN, Trigrams 92.5 70.2Table 3: Baseline results using no LM, bi- and trigrams.70.075.080.085.090.095.01 3 5 7 9 11 13 15 17 19 21 23 25Beam size0123456789sec6local probs ssss ss s s s s s searly probs ccccc cc c c c c cnal probs 4444 4 44 4 4 4 4time per name ?? ? ? ? ? ?? ??????Figure 4: String accuracy for a tree search in a list of1 million (200,000 unique) names. Three dierent methodsfor assigning probabilities in the tree are compared for dif-ferent beam sizes. The lower curve is the recognition time inseconds for one string.76.078.080.082.084.086.088.090.092.094.096.098.0100.0Total 1000 10k 100k 1 Mio 10 100Unique 4078 45k 209k 807kSLP eee e ee e e eeee ee eeSLN uuu u uuu uuuuu uu u uSLP no Probs cccc cccccccc c cccSLN no Probs ss ssssssssss ssssFigure 5: String Accuracy using trees with and withoutprobabilities on the SLN and SLP test sets.5. SUMMARYWe have tested the MS-TDNN letter recognizer on thespelled names provided in the OGI Spelled and Spoken WordTelephone Corpus. Without using any language modeling,the baseline result on SLP test set is 90.6% LA. As expectedthe SLN set is more dicult, 88.2% LA was achieved. Theseresults are only about 2 - 3% worse than our results on highquality speech spellings [5, 1] probably because people tendto spell more careful under adverse telephone condition. Bi-and trigrams achieve only moderate improvements. If thesearch is constrained to a given set of names, high letterand string accuracies can be achieved. Enriching the searchtree with probabilities proved to be astonishingly helpful.In lists in the order of 1000 names well above 95% stringaccuracy can be reached. Even in our largest list with 14million (800,000 unique) names, almost 90% string accuracyis achieved. The results are summarized in table 4.Language Model SLNLA SANo LM 88.2 53.7Bigrams 91.0 62.8Trigrams 92.5 70.2Tree 1,000 98.1 97.7Tree 100,000 97.5 94.4Tree 1,000,000 97.1 91.5Tree 14,000,000 96.5 89.3" 14,000,000, no prob. 91.4 75.2Table 4: Letter and string accuracies for the SLN and SLPtest set, given no language model, bi- and trigrams and treesenriched with probabilities.AcknowledgementsThe authors would like to thank the members of the Inter-active System Laboratories, especially Klaus Ries and Bern-hard Suhm for their assistance with the bi- and trigram lan-guage models, and Michael Finke for contributing the ideaof the \early" tree probabilities.6. REFERENCES1. Martin Betz and Hermann Hild. Language Models for aSpelled Letter Recognizer. In Proc. IEEE InternationalConference on Acoustics, Speech, and Signal Processing,pages 856{859. IEEE, May 9-12 1995.2. Roland A. Cole, Mark Fanty, Gopalakrishnan, andRik D.T. Janssen. Speaker-Independent Name Retrivalfrom Spellings using a Database of 50,000 Names. In Pro-ceedings of the International Conference on Acoustics,Speech and Signal Processing, Toronto, Ontario, Canada,May 1991. IEEE.3. P. Haner, M. Franzini, and A. Waibel. Integrating TimeAlignment and Neural Networks for High PerformanceContinuous Speech Recognition. In Proc. InternationalConference on Acoustics, Speech, and Signal Processing.IEEE, May 1991.4. J. Hampshire and A. Waibel. A Novel Objective Functionfor Improved Phoneme Recognition Using Time-DelayNeural Networks. In Proceedings of the 1989 InternationalJoint Conference on Neural Networks, June 1989.5. Hermann Hild and Alex Waibel. Speaker-IndependentConnected Letter Recognition With a Multi-StateTime Delay Neural Network. In 3rd European Confer-ence on Speech, Communication and Technology (EU-ROSPEECH) 93, pages 1481 { 1484, September 1993.6. Jean-Claude Junqua, Stephane Valente, Dominique Fohr,and Jean-Francois Mari. An N-Best Strategy, DynamicGrammars and Selectively Trained Neural Networks forReal-Time Recognition of Continuously Spelled Namesover the Telephone. In Proc. IEEE International Confer-ence on Acoustics, Speech, and Signal Processing, pages852{855. IEEE, May 9-12 1995.

Recognition of spelled names over the telephone

https://publikationen.bibliothek.kit.edu/328096/2478

Recognition of spelled names over the telephone

Abstract

Similar works

Full text

Available Versions

KITopen