This paper presents an automatic strategy to decide how to pronounce a Capital Letter Sequence (CLS) in a Text to Speech system (TTS). If CLS is well known by the TTS, it can be expanded in several words. But when the CLS is unknown, the system has two alternatives: spelling it (abbreviation) or pronouncing it as a new word (acronym). In Spanish, there is a high relationship between letters and phonemes. Because of this, when a CLS is similar to other words in Spanish, there is a high tendency to pronounce it as a standard word. This paper proposes an automatic method for detecting acronyms. Additionaly, this paper analyses the discrimination capability of some features, and several strategies for combining them in order to obtain the best classifier. For the best classifier, the classification error is 8.45%. About the feature analysis, the best features have been the Letter Sequence Perplexity and the Average N-gram order

King, Simon

Lopez Ludeña, Veronica

Montero Martínez, Juan Manuel

San Segundo Hernández, Rubén

Edinburgh Research Explorer

     Edinburgh Research Explorer                                      Detecting Acronyms from Capital Letter Sequences in SpanishCitation for published version:San-Segundo, R, Montero, JM, Lopez-Luden, V & King, S 2012, Detecting Acronyms from Capital LetterSequences in Spanish. in Proc. Interspeech.Link:Link to publication record in Edinburgh Research ExplorerDocument Version:Peer reviewed versionPublished In:Proc. InterspeechPublisher Rights Statement:© San-Segundo, R., Montero, J. M., Lopez-Luden, V., & King, S. (2012). Detecting Acronyms from Capital LetterSequences in Spanish. In Proc. Interspeech.General rightsCopyright for the publications made accessible via the Edinburgh Research Explorer is retained by the author(s)and / or other copyright owners and it is a condition of accessing these publications that users recognise andabide by the legal requirements associated with these rights.Take down policyThe University of Edinburgh has made every reasonable effort to ensure that Edinburgh Research Explorercontent complies with UK legislation. If you believe that the public display of this file breaches copyright pleasecontact openaccess@ed.ac.uk providing details, and we will remove access to the work immediately andinvestigate your claim.Download date: 05. Apr. 2019Detecting Acronyms from Capital Letter Sequences in SpanishRube´n San-Segundo1, Juan M. Montero1, Vero´nica Lo´pez-Luden˜a1, Simon King21Speech Technology Group, ETSI Telecomunicacio´n. UPM. Spain.2Centre for Speech Technology Research, University of Edinburgh, UK.lapiz@die.upm.esAbstractThis paper presents an automatic strategy to decide howto pronounce a Capital Letter Sequence (CLS) in a Text toSpeech system (TTS). If CLS is well known by the TTS,it can be expanded in several words. But when the CLS isunknown, the system has two alternatives: spelling it (ab-breviation) or pronouncing it as a new word (acronym).In Spanish, there is a high relationship between lettersand phonemes. Because of this, when a CLS is similar toother words in Spanish, there is a high tendency to pro-nounce it as a standard word. This paper proposes anautomatic method for detecting acronyms. Additionaly,this paper analyses the discrimination capability of somefeatures, and several strategies for combining them in or-der to obtain the best classifier. For the best classifier, theclassification error is 8.45%. About the feature analysis,the best features have been the Letter Sequence Perplex-ity and the Average N-gram order.Index Terms: Capital letter sequence pronunciation,Speech synthesis, Spelling, Spanish, Acronyms, Abbre-viations1. IntroductionWhen developing a Text to Speech (TTS) system, Capi-tal Letter Sequences (CLS) can be pronounced in severalways. In many TTS, there is a list with the most frequentCLSs and their corresponding expansion. In this case,the TTS expands the CLS into several words (i.e. MIT:the Massachusetts Institute of Technology). The problemarises when the CLS is unknown. In this case, the systemhas two alternatives. The general one is to spell the CLSletter by letter (i.e. FBI: F B I). On the other hand, insome cases, the CLS can be pronounced as a new word(i.e. NATO), the CLS is an acronym. In Spanish, due tothe high relationship between letters and phonemes, thereis a high tendency to pronounce any CLS as a standardword when the CLS is similar to other words in Span-ish. In this sense, there is not a general rule to decidewhen pronouncing it as word or letter by letter. This pa-per proposes an automatic method for selecting the bestalternative when dealing with unknown CLSs. The maintarget is to detect acronyms (that can be pronounced as astandard word) rejecting the abbreviations (that must bespelled). This paper includes also an analysis of the dis-crimination power of several features considered in theclassification task.The rest of the paper is organized as follows. Sec-tion 2 summaries the state of the art in acronym pro-nunciation. Section 3 describes the database used in thiswork. Section 4 presents the main features proposed inthis study. Section 5 presents the feature analysis usingthis database. Finally, section 6 shows the main classi-fication experiments and section 7 summaries the mainconclusions of this work.2. State of the artThe problem addressed in this paper is one of the prob-lems included in the research line of text normalization[1]. In this reference, the whole text normalization prob-lem is described. Sproat and al present a Non-StandardWord (NSW) taxonomy with several proposals of auto-matic strategies for classifying every NSW within one ofthe taxonomy classes. According to acronyms, Sproatand al propose to consider acronyms as any other stan-dard words and to use a dictionary (list of standard words)for discriminating between standard words and NSWs.This paper proposes an automatic method for selectingthe best prononciation of unknow CLSs (not included inthe dictionary).In the literature, there are some research efforts fo-cused on how to extract acronyms from raw text auto-matically [2] [3] [4]. In this studies, the main target is toincrease the list of acronyms, in order to reduce the prob-ability of being unknown. On the other hand, there areseveral works trying to model the creation of acronyms[5] or abbreviations [6]. From these efforts, it is possi-ble to conclude that the syllable and letter structure areimportant aspect in the creation of acronyms and abbre-viations. Based on these ideas, this paper will considerfeatures based on a letter language model.3. Database descriptionTable 1 summaries the main characteristics of thedatabase considered in this work. This database has beenextracted from several months of the ”El Mundo” news-paper [7]. The extraction process has been performedautomatically (looking for words in capital letters) with aposterior manual revision.Table 1: Database characteristics.Characteristics Acronyms AbbreviationsNumber of examples 653 684Percentage 48.8% 51.2%Average length 3.8 3.3Maximum length 8 5The first characteristic is that the percentage ofacronyms is very high, very close to the percentage ofabbreviations. This is due to the high relationship be-tween letters and phonemes in Spanish. Another charac-teristic is that the average length of acronyms is higher. Asmaller letter sequence (2 or 3 letters) has less probabilityof generating an acronym.4. Main features based on letter sequencesThis work has considered several features based on a let-ter language model (LLM) in order to decide if a CapitalLetter Sequence (CLS) must be spelled or pronounced asa standard word. This language model has been gener-ated considering the acronyms from the training set (in aCross-Validation strategy described in section 6). Con-sidering the maximun lengths reported in table 1, a 6-gram letter language model has been considered. ThisLLM has been trained using the IRSTLM toolkit [8]. Inorder to train this model, two marks for indicating the be-ginning and the end of the letter sequence have been con-sidered (ini and end). For example, the letter sequencecorresponding to the abbreviation IND, is ”ini I N D end”.This way, it is possible to incorporate information aboutthe boundaries. The features derivated from the LLM are:• Letter Sequence Perplexity (LS Perplexity). Thisfeature is the perplexity of a CLS given a LLM.Considering that the LLM was generated usingacronyms from the training set, it is expected thatacronyms have lower perplexity than abbrevia-tions.• Minimum Probability (Min Prob). In some cases,part of the CLS can be very common in Span-ish words while there are other parts forbidden instandard Spanish words. The perplexity, in thesecases, can be small but the CLS should be re-jected (not considered as an acronym). For exam-ple: MNAC, NAC has a very high probability andonly the sequence MN generates a low probabil-ity. Considering the whole perplexity some localdetails are missed. Computing the minimum prob-ability along the letter sequence can report addi-tional information.• Maximum Probability (Max Prob). Similar to theprevious feature, in this case, the maximum prob-ability computed along the letter sequence is con-sidered.• Average N-gram (Ave N-gram). This measure-ment is the average N-gram order for computingthe probability of every letter. This feature tries tocomplement the information reported with the per-plexity. Acronyms are expected to have a higheraverage N-gram order than abbreviations. For ex-ample, the N-gram orders for computing the proba-bilities in the MNAC sequence are: ini M(2-gram)N(1-gram) A(2-gram) C(3-gram) end(2-gram). Inthis case, the Average N-gram order is 10 / 5 = 2.• Minimum N-gram (Min N-gram). Similar to theminimum probability, the minimum N-gram orderis considered for reporting more details about localbehaviours.• Maximum N-gram (Max N-gram). In order tocomplete the analysis, the maximum N-gram orderis also considered.5. Feature AnalysisFig. 1 shows the ROC curves for all the features. False ac-ceptation is the percentage of cases where the system con-siders an abbreviation as an acronym by mistake, tryingto pronounce it as a standard word. False rejection is thepercentage of cases where the system classifies (wrongly)an acronym as an abbreviation, spelling it instead of pro-nouncing it as a standard word. As it is shown, the bestfeatures are the Letter Sequence Perplexity and the Aver-age N-gram order. In both cases, it is possible to reacha 10% EER (Equal Error Rate). Minimun and MaximunN-gram order features have a discrete number of possiblevalues, so the ROC curves present straight lines in someplaces.Additional to the ROC curves, table 2 shows the In-formation Gain for each feature. Similar to the ROCcurves, the best features are LS Perplexity and Ave N-gram. The worst ones are the Max Prob and the Max N-gram. This analysis has been performed using the WEKAtoolkit [9].Table 2: Information Gain AnalysisFeature Information GainAve N-gram 0.67LS Perplexity 0.66Min Prob 0.47Min N-gram 0.42Max N-gram 0.40Max Prob 0.36ROC0%10%20%30%40%50%60%70%80%90%100%0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%False AcceptationF al se R ej e ct i onLS PerplexityMin ProbMax ProbAve N-gramMin N-gramMax N-gramFigure 1: ROC for the different features.6. Classification experimentsFor the classification experiments, a Cross-Validationstrategy has been considered. The whole database hasbeen ramdonly divided into 10 subsets: eight for trainingthe LLM, one for tunning the classifier parameters andone for testing. The experiments are repeated 10 timesand the results are the average of all runs.6.1. Grouping FeaturesThe first analysis is about how to group the different fea-tures. Several alternatives have been considered and eval-uated in table 3. These grouping alternatives are: the twobest features, the LLM probability based features, the N-gram order based features, and considering all of them.Table 3 shows the classification error and the ROC area(area under the ROC curve) for the different alternatives.In this case, a simple classifier has been considered: anaive Bayes classifier. This classifier is a simple proba-bilistic classifier based on applying Bayes’ theorem as-suming independence between features.Table 3: Classification experiments considering severalgrouping alternativesFeatures group ROC area Class. errorAve N-gram + LS Pp 0.968 12.04%LS Pp, Min + Max Prob 0.962 11.90%Ave + Min + Max N-gram 0.966 13.98%All the features 0.968 11.20%As it is shown in table 3, LLM probability based fea-tures perform better than N-gram based ones. The bestresults are obtained considering all the features.6.2. Using Several ClassifiersThis sub-section presents experiments considering differ-ent classifiers for combining all the features. Table 4shows the classification error and the ROC area (area un-der the ROC curve) for the different alternatives:• Naive Bayes. Same as the classifier considered inthe previous section.• Bayesian Network. This classifier is based onBayesian Networks. A Bayesian Network is aprobabilistic graphical model that represents a setof random variables and their conditional depen-dencies via a directed acyclic graph (DAG).• Multi-Layer Perceptron. A Multi Layer Perceptronis a feedforward artificial neural network consist-ing of multiple layers of nodes in a directed graph(3 layers in this case), with each layer fully con-nected to the next one. Except for the input nodes,each node is a neuron (or processing element) witha nonlinear activation function (sigmoide).• Decision Tree using the C4.5 algorithm [10].• Attribute Selected Classifier. This is meta-classifier that takes a search algorithm and evalua-tor next to the base classifier. In this case, the baseclassifier is the decision tree generated using theC4.5 algorithm.Table 4: Classification experiments with different classi-fiersFeatures group ROC area Class. errorNaive Bayes 0.968 11.20%Bayesian Network 0.969 10.40%Multi-Layer Perceptron 0.967 9.57%Decision Tree C4.5 0.940 8.67%Attribute Selected Classifier 0.943 8.45%In these experiments (table 4), the best classifier hasbeen the decision tree obtained with the C4.5 algorithm.In this case, the classification error is 8.67%. Using themeta-classifier (Attribute Selected Classifier) the classifi-cation error is 8.45%.Fig. 2 shows the top nodes of the decision tree ob-tained (using WEKA). As it is shown, decision over Av-erage N-gram, LS perplexity and Min N-gram featuresare situated at the top nodes.Fig. 3 shows the ROC curves for all the classifiers.False acceptation represents the percentage of abbrevia-tions classified as acronym by mistake, and the false re-jection is the percentage of acronyms classified as abbre-viations. When classifying an acronym as an abbreviation(false rejection), the system will spell the CLS instead ofpronouncing it as a standard word. On the other hand,when the system classifies an abbreviation as an acronym(false acceptance), the system will try to pronounce it asFigure 2: Top nodes of the decision treeROC for all classifiers0%5%10%15%20%25%30%0% 5% 10% 15% 20%False AcceptationF al se Rej e ct i onNaive BayesBayesian NetworkMulti-layer PerceptronDecision TreeMeta classifierWorking pointFigure 3: ROC curves considering the different classi-fiers.a standard word generating a wrong pronunciation. Thissecond type of error is the most dangerous one. Becauseof this, in order to define a working point, the false accep-tance must be lower than 5%. Table 5 shows the false re-jection for several working points (false acceptation per-centages), considering the best system in this range: theBayesian Network.Table 5: Different working pointsFalse Acceptance False Rejection5.0% 15.1%2.5% 18.5%1.0% 23.5%7. ConclusionsThis paper has presented an automatic strategy to clas-sify Capital Letter Sequences (CLSs) as abbreviation oracronyms, when these CLSs are unknown by the TTS(they are not in the system vocabulary). The abbrevi-ations will be spelled while the acronyms will be pro-nounced as standar words. In Spanish, there is a high per-centage of acronyms that makes this work very interest-ing. This paper has also analysed the discrimination capa-bility of some features based on a Letter Language Model(LLM): letter sequence perplexity, minimum and maxi-mum probability and, average, minimum and maximumN-gram order when computing the probability along theletter sequence. The best features have been the lettersequence perplexity and the average N-gram order. Fi-nally, the paper has evaluated several classifiers for com-bining the differents features. The lowest classificationerror (8.45% in table 4) has been obtained with a metaclassifier that uses a decision tree based on the C4.5 algo-rithm (this has been the best meta classifier in this task).When analysing the working point (a false acceptancelower than 5%), the best classifier, in this range, is theBayesian Network.8. AcknowledgementsThe work leading to these results has received fund-ing from the European Union under grant agree-ment n 287678. It has also been supportedby TIMPANO (TIN2011-28169-C05-03), INAPRA(MICINN, DPI2010-21247-C02-02) and MA2VICMR(CAM, S2009/TIC-1542) projects. Authors also thankthe other members of the Speech Technology Group andSimple4All project for the continuous and fruitful discus-sion on these topics.9. References[1] Sproat, R., et al. ”Normalization of non-standard words.” Com-puter Speech and Language, 15(3), 287-333, 2001.[2] Larkey, Leah, Paul Ogilvie, Andrew Price and Brenden Tamilio.”Acrophile: An Automated Acronym Extractor and Server”, InProceedings of the ACM Digital Libraries conference, pp. 205-214, 2000.[3] Stuart Yeates. ”Automatic Extraction of Acronyms from Text”.New Zealand Computer Science Research Students’ Conference.1999.[4] J.T. Chang, H Schtze, and R.B. Altman. 2002. ”Creating an On-line Dictionary of Abbreviations from MEDLINE” JAMIA.[5] Cannon, Garland. 1989. ”Abbreviations and acronyms in Englishword-formation.” American Speech, 64:99127.[6] Pennell and Liu. 2011. ”Toward text message normalization:Modeling abbreviation generation.” Proceedings of the IEEE. pp.5364-5367.[7] El Mundo Newspaper (1996-2006). New from the ”El Mundo”newspaper during several years.[8] Federico, M. and Cettolo, M. (2007) Efficient Handling of N-gram Language Models for Statistical Machine Translation Proc.of ACL Workshop on SMT. pages 8895, Prague, Czech Republic.[9] Witten, I.H. and Frank, E. (2005) Data Mining: Practical machinelearning tools and techniques. 2nd edition Morgan Kaufmann, SanFrancisco.[10] Quinlan, J. R. C4.5: Programs for Machine Learning. MorganKaufmann Publishers, 1993.

     Edinburgh Research Explorer                                      Detecting Acronyms from Capital Letter Sequences in SpanishCitation for published version:San-Segundo, R, Montero, JM, Lopez-Luden, V & King, S 2012, Detecting Acronyms from Capital LetterSequences in Spanish. in Proc. Interspeech.Link:Link to publication record in Edinburgh Research ExplorerDocument Version:Peer reviewed versionPublished In:Proc. InterspeechPublisher Rights Statement:© San-Segundo, R., Montero, J. M., Lopez-Luden, V., & King, S. (2012). Detecting Acronyms from Capital LetterSequences in Spanish. In Proc. Interspeech.General rightsCopyright for the publications made accessible via the Edinburgh Research Explorer is retained by the author(s)and / or other copyright owners and it is a condition of accessing these publications that users recognise andabide by the legal requirements associated with these rights.Take down policyThe University of Edinburgh has made every reasonable effort to ensure that Edinburgh Research Explorercontent complies with UK legislation. If you believe that the public display of this file breaches copyright pleasecontact openaccess@ed.ac.uk providing details, and we will remove access to the work immediately andinvestigate your claim.Download date: 28. Apr. 2017Detecting Acronyms from Capital Letter Sequences in SpanishRube´n San-Segundo1, Juan M. Montero1, Vero´nica Lo´pez-Luden˜a1, Simon King21Speech Technology Group, ETSI Telecomunicacio´n. UPM. Spain.2Centre for Speech Technology Research, University of Edinburgh, UK.lapiz@die.upm.esAbstractThis paper presents an automatic strategy to decide howto pronounce a Capital Letter Sequence (CLS) in a Text toSpeech system (TTS). If CLS is well known by the TTS,it can be expanded in several words. But when the CLS isunknown, the system has two alternatives: spelling it (ab-breviation) or pronouncing it as a new word (acronym).In Spanish, there is a high relationship between lettersand phonemes. Because of this, when a CLS is similar toother words in Spanish, there is a high tendency to pro-nounce it as a standard word. This paper proposes anautomatic method for detecting acronyms. Additionaly,this paper analyses the discrimination capability of somefeatures, and several strategies for combining them in or-der to obtain the best classifier. For the best classifier, theclassification error is 8.45%. About the feature analysis,the best features have been the Letter Sequence Perplex-ity and the Average N-gram order.Index Terms: Capital letter sequence pronunciation,Speech synthesis, Spelling, Spanish, Acronyms, Abbre-viations1. IntroductionWhen developing a Text to Speech (TTS) system, Capi-tal Letter Sequences (CLS) can be pronounced in severalways. In many TTS, there is a list with the most frequentCLSs and their corresponding expansion. In this case,the TTS expands the CLS into several words (i.e. MIT:the Massachusetts Institute of Technology). The problemarises when the CLS is unknown. In this case, the systemhas two alternatives. The general one is to spell the CLSletter by letter (i.e. FBI: F B I). On the other hand, insome cases, the CLS can be pronounced as a new word(i.e. NATO), the CLS is an acronym. In Spanish, due tothe high relationship between letters and phonemes, thereis a high tendency to pronounce any CLS as a standardword when the CLS is similar to other words in Span-ish. In this sense, there is not a general rule to decidewhen pronouncing it as word or letter by letter. This pa-per proposes an automatic method for selecting the bestalternative when dealing with unknown CLSs. The maintarget is to detect acronyms (that can be pronounced as astandard word) rejecting the abbreviations (that must bespelled). This paper includes also an analysis of the dis-crimination power of several features considered in theclassification task.The rest of the paper is organized as follows. Sec-tion 2 summaries the state of the art in acronym pro-nunciation. Section 3 describes the database used in thiswork. Section 4 presents the main features proposed inthis study. Section 5 presents the feature analysis usingthis database. Finally, section 6 shows the main classi-fication experiments and section 7 summaries the mainconclusions of this work.2. State of the artThe problem addressed in this paper is one of the prob-lems included in the research line of text normalization[1]. In this reference, the whole text normalization prob-lem is described. Sproat and al present a Non-StandardWord (NSW) taxonomy with several proposals of auto-matic strategies for classifying every NSW within one ofthe taxonomy classes. According to acronyms, Sproatand al propose to consider acronyms as any other stan-dard words and to use a dictionary (list of standard words)for discriminating between standard words and NSWs.This paper proposes an automatic method for selectingthe best prononciation of unknow CLSs (not included inthe dictionary).In the literature, there are some research efforts fo-cused on how to extract acronyms from raw text auto-matically [2] [3] [4]. In this studies, the main target is toincrease the list of acronyms, in order to reduce the prob-ability of being unknown. On the other hand, there areseveral works trying to model the creation of acronyms[5] or abbreviations [6]. From these efforts, it is possi-ble to conclude that the syllable and letter structure areimportant aspect in the creation of acronyms and abbre-viations. Based on these ideas, this paper will considerfeatures based on a letter language model.3. Database descriptionTable 1 summaries the main characteristics of thedatabase considered in this work. This database has beenextracted from several months of the ”El Mundo” news-paper [7]. The extraction process has been performedautomatically (looking for words in capital letters) with aposterior manual revision.Table 1: Database characteristics.Characteristics Acronyms AbbreviationsNumber of examples 653 684Percentage 48.8% 51.2%Average length 3.8 3.3Maximum length 8 5The first characteristic is that the percentage ofacronyms is very high, very close to the percentage ofabbreviations. This is due to the high relationship be-tween letters and phonemes in Spanish. Another charac-teristic is that the average length of acronyms is higher. Asmaller letter sequence (2 or 3 letters) has less probabilityof generating an acronym.4. Main features based on letter sequencesThis work has considered several features based on a let-ter language model (LLM) in order to decide if a CapitalLetter Sequence (CLS) must be spelled or pronounced asa standard word. This language model has been gener-ated considering the acronyms from the training set (in aCross-Validation strategy described in section 6). Con-sidering the maximun lengths reported in table 1, a 6-gram letter language model has been considered. ThisLLM has been trained using the IRSTLM toolkit [8]. Inorder to train this model, two marks for indicating the be-ginning and the end of the letter sequence have been con-sidered (ini and end). For example, the letter sequencecorresponding to the abbreviation IND, is ”ini I N D end”.This way, it is possible to incorporate information aboutthe boundaries. The features derivated from the LLM are:• Letter Sequence Perplexity (LS Perplexity). Thisfeature is the perplexity of a CLS given a LLM.Considering that the LLM was generated usingacronyms from the training set, it is expected thatacronyms have lower perplexity than abbrevia-tions.• Minimum Probability (Min Prob). In some cases,part of the CLS can be very common in Span-ish words while there are other parts forbidden instandard Spanish words. The perplexity, in thesecases, can be small but the CLS should be re-jected (not considered as an acronym). For exam-ple: MNAC, NAC has a very high probability andonly the sequence MN generates a low probabil-ity. Considering the whole perplexity some localdetails are missed. Computing the minimum prob-ability along the letter sequence can report addi-tional information.• Maximum Probability (Max Prob). Similar to theprevious feature, in this case, the maximum prob-ability computed along the letter sequence is con-sidered.• Average N-gram (Ave N-gram). This measure-ment is the average N-gram order for computingthe probability of every letter. This feature tries tocomplement the information reported with the per-plexity. Acronyms are expected to have a higheraverage N-gram order than abbreviations. For ex-ample, the N-gram orders for computing the proba-bilities in the MNAC sequence are: ini M(2-gram)N(1-gram) A(2-gram) C(3-gram) end(2-gram). Inthis case, the Average N-gram order is 10 / 5 = 2.• Minimum N-gram (Min N-gram). Similar to theminimum probability, the minimum N-gram orderis considered for reporting more details about localbehaviours.• Maximum N-gram (Max N-gram). In order tocomplete the analysis, the maximum N-gram orderis also considered.5. Feature AnalysisFig. 1 shows the ROC curves for all the features. False ac-ceptation is the percentage of cases where the system con-siders an abbreviation as an acronym by mistake, tryingto pronounce it as a standard word. False rejection is thepercentage of cases where the system classifies (wrongly)an acronym as an abbreviation, spelling it instead of pro-nouncing it as a standard word. As it is shown, the bestfeatures are the Letter Sequence Perplexity and the Aver-age N-gram order. In both cases, it is possible to reacha 10% EER (Equal Error Rate). Minimun and MaximunN-gram order features have a discrete number of possiblevalues, so the ROC curves present straight lines in someplaces.Additional to the ROC curves, table 2 shows the In-formation Gain for each feature. Similar to the ROCcurves, the best features are LS Perplexity and Ave N-gram. The worst ones are the Max Prob and the Max N-gram. This analysis has been performed using the WEKAtoolkit [9].Table 2: Information Gain AnalysisFeature Information GainAve N-gram 0.67LS Perplexity 0.66Min Prob 0.47Min N-gram 0.42Max N-gram 0.40Max Prob 0.36ROC0%10%20%30%40%50%60%70%80%90%100%0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%False AcceptationF al se R ej e ct i onLS PerplexityMin ProbMax ProbAve N-gramMin N-gramMax N-gramFigure 1: ROC for the different features.6. Classification experimentsFor the classification experiments, a Cross-Validationstrategy has been considered. The whole database hasbeen ramdonly divided into 10 subsets: eight for trainingthe LLM, one for tunning the classifier parameters andone for testing. The experiments are repeated 10 timesand the results are the average of all runs.6.1. Grouping FeaturesThe first analysis is about how to group the different fea-tures. Several alternatives have been considered and eval-uated in table 3. These grouping alternatives are: the twobest features, the LLM probability based features, the N-gram order based features, and considering all of them.Table 3 shows the classification error and the ROC area(area under the ROC curve) for the different alternatives.In this case, a simple classifier has been considered: anaive Bayes classifier. This classifier is a simple proba-bilistic classifier based on applying Bayes’ theorem as-suming independence between features.Table 3: Classification experiments considering severalgrouping alternativesFeatures group ROC area Class. errorAve N-gram + LS Pp 0.968 12.04%LS Pp, Min + Max Prob 0.962 11.90%Ave + Min + Max N-gram 0.966 13.98%All the features 0.968 11.20%As it is shown in table 3, LLM probability based fea-tures perform better than N-gram based ones. The bestresults are obtained considering all the features.6.2. Using Several ClassifiersThis sub-section presents experiments considering differ-ent classifiers for combining all the features. Table 4shows the classification error and the ROC area (area un-der the ROC curve) for the different alternatives:• Naive Bayes. Same as the classifier considered inthe previous section.• Bayesian Network. This classifier is based onBayesian Networks. A Bayesian Network is aprobabilistic graphical model that represents a setof random variables and their conditional depen-dencies via a directed acyclic graph (DAG).• Multi-Layer Perceptron. A Multi Layer Perceptronis a feedforward artificial neural network consist-ing of multiple layers of nodes in a directed graph(3 layers in this case), with each layer fully con-nected to the next one. Except for the input nodes,each node is a neuron (or processing element) witha nonlinear activation function (sigmoide).• Decision Tree using the C4.5 algorithm [10].• Attribute Selected Classifier. This is meta-classifier that takes a search algorithm and evalua-tor next to the base classifier. In this case, the baseclassifier is the decision tree generated using theC4.5 algorithm.Table 4: Classification experiments with different classi-fiersFeatures group ROC area Class. errorNaive Bayes 0.968 11.20%Bayesian Network 0.969 10.40%Multi-Layer Perceptron 0.967 9.57%Decision Tree C4.5 0.940 8.67%Attribute Selected Classifier 0.943 8.45%In these experiments (table 4), the best classifier hasbeen the decision tree obtained with the C4.5 algorithm.In this case, the classification error is 8.67%. Using themeta-classifier (Attribute Selected Classifier) the classifi-cation error is 8.45%.Fig. 2 shows the top nodes of the decision tree ob-tained (using WEKA). As it is shown, decision over Av-erage N-gram, LS perplexity and Min N-gram featuresare situated at the top nodes.Fig. 3 shows the ROC curves for all the classifiers.False acceptation represents the percentage of abbrevia-tions classified as acronym by mistake, and the false re-jection is the percentage of acronyms classified as abbre-viations. When classifying an acronym as an abbreviation(false rejection), the system will spell the CLS instead ofpronouncing it as a standard word. On the other hand,when the system classifies an abbreviation as an acronym(false acceptance), the system will try to pronounce it asFigure 2: Top nodes of the decision treeROC for all classifiers0%5%10%15%20%25%30%0% 5% 10% 15% 20%False AcceptationF al se Rej e ct i onNaive BayesBayesian NetworkMulti-layer PerceptronDecision TreeMeta classifierWorking pointFigure 3: ROC curves considering the different classi-fiers.a standard word generating a wrong pronunciation. Thissecond type of error is the most dangerous one. Becauseof this, in order to define a working point, the false accep-tance must be lower than 5%. Table 5 shows the false re-jection for several working points (false acceptation per-centages), considering the best system in this range: theBayesian Network.Table 5: Different working pointsFalse Acceptance False Rejection5.0% 15.1%2.5% 18.5%1.0% 23.5%7. ConclusionsThis paper has presented an automatic strategy to clas-sify Capital Letter Sequences (CLSs) as abbreviation oracronyms, when these CLSs are unknown by the TTS(they are not in the system vocabulary). The abbrevi-ations will be spelled while the acronyms will be pro-nounced as standar words. In Spanish, there is a high per-centage of acronyms that makes this work very interest-ing. This paper has also analysed the discrimination capa-bility of some features based on a Letter Language Model(LLM): letter sequence perplexity, minimum and maxi-mum probability and, average, minimum and maximumN-gram order when computing the probability along theletter sequence. The best features have been the lettersequence perplexity and the average N-gram order. Fi-nally, the paper has evaluated several classifiers for com-bining the differents features. The lowest classificationerror (8.45% in table 4) has been obtained with a metaclassifier that uses a decision tree based on the C4.5 algo-rithm (this has been the best meta classifier in this task).When analysing the working point (a false acceptancelower than 5%), the best classifier, in this range, is theBayesian Network.8. AcknowledgementsThe work leading to these results has received fund-ing from the European Union under grant agree-ment n 287678. It has also been supportedby TIMPANO (TIN2011-28169-C05-03), INAPRA(MICINN, DPI2010-21247-C02-02) and MA2VICMR(CAM, S2009/TIC-1542) projects. Authors also thankthe other members of the Speech Technology Group andSimple4All project for the continuous and fruitful discus-sion on these topics.9. References[1] Sproat, R., et al. ”Normalization of non-standard words.” Com-puter Speech and Language, 15(3), 287-333, 2001.[2] Larkey, Leah, Paul Ogilvie, Andrew Price and Brenden Tamilio.”Acrophile: An Automated Acronym Extractor and Server”, InProceedings of the ACM Digital Libraries conference, pp. 205-214, 2000.[3] Stuart Yeates. ”Automatic Extraction of Acronyms from Text”.New Zealand Computer Science Research Students’ Conference.1999.[4] J.T. Chang, H Schtze, and R.B. Altman. 2002. ”Creating an On-line Dictionary of Abbreviations from MEDLINE” JAMIA.[5] Cannon, Garland. 1989. ”Abbreviations and acronyms in Englishword-formation.” American Speech, 64:99127.[6] Pennell and Liu. 2011. ”Toward text message normalization:Modeling abbreviation generation.” Proceedings of the IEEE. pp.5364-5367.[7] El Mundo Newspaper (1996-2006). New from the ”El Mundo”newspaper during several years.[8] Federico, M. and Cettolo, M. (2007) Efficient Handling of N-gram Language Models for Statistical Machine Translation Proc.of ACL Workshop on SMT. pages 8895, Prague, Czech Republic.[9] Witten, I.H. and Frank, E. (2005) Data Mining: Practical machinelearning tools and techniques. 2nd edition Morgan Kaufmann, SanFrancisco.[10] Quinlan, J. R. C4.5: Programs for Machine Learning. MorganKaufmann Publishers, 1993.

Archivo Digital UPM

Detecting acronyms from capital letter sequences in Spanish

Servicio de Coordinación de Bibliotecas de la Universidad Politécnica de Madrid

Detecting Acronyms from Capital Letter Sequences in Spanish
Rube´n San-Segundo1, Juan M. Montero1, Vero´nica Lo´pez-Luden˜a1, Simon King2
1Speech Technology Group, ETSI Telecomunicacio´n. UPM. Spain.
2Centre for Speech Technology Research, University of Edinburgh, UK.
lapiz@die.upm.es
Abstract
This paper presents an automatic strategy to decide how
to pronounce a Capital Letter Sequence (CLS) in a Text to
Speech system (TTS). If CLS is well known by the TTS,
it can be expanded in several words. But when the CLS is
unknown, the system has two alternatives: spelling it (ab-
breviation) or pronouncing it as a new word (acronym).
In Spanish, there is a high relationship between letters
and phonemes. Because of this, when a CLS is similar to
other words in Spanish, there is a high tendency to pro-
nounce it as a standard word. This paper proposes an
automatic method for detecting acronyms. Additionaly,
this paper analyses the discrimination capability of some
features, and several strategies for combining them in or-
der to obtain the best classifier. For the best classifier, the
classification error is 8.45%. About the feature analysis,
the best features have been the Letter Sequence Perplex-
ity and the Average N-gram order.
Index Terms: Capital letter sequence pronunciation,
Speech synthesis, Spelling, Spanish, Acronyms, Abbre-
viations
1. Introduction
When developing a Text to Speech (TTS) system, Capi-
tal Letter Sequences (CLS) can be pronounced in several
ways. In many TTS, there is a list with the most frequent
CLSs and their corresponding expansion. In this case,
the TTS expands the CLS into several words (i.e. MIT:
the Massachusetts Institute of Technology). The problem
arises when the CLS is unknown. In this case, the system
has two alternatives. The general one is to spell the CLS
letter by letter (i.e. FBI: F B I). On the other hand, in
some cases, the CLS can be pronounced as a new word
(i.e. NATO), the CLS is an acronym. In Spanish, due to
the high relationship between letters and phonemes, there
is a high tendency to pronounce any CLS as a standard
word when the CLS is similar to other words in Span-
ish. In this sense, there is not a general rule to decide
when pronouncing it as word or letter by letter. This pa-
per proposes an automatic method for selecting the best
alternative when dealing with unknown CLSs. The main
target is to detect acronyms (that can be pronounced as a
standard word) rejecting the abbreviations (that must be
spelled). This paper includes also an analysis of the dis-
crimination power of several features considered in the
classification task.
The rest of the paper is organized as follows. Sec-
tion 2 summaries the state of the art in acronym pro-
nunciation. Section 3 describes the database used in this
work. Section 4 presents the main features proposed in
this study. Section 5 presents the feature analysis using
this database. Finally, section 6 shows the main classi-
fication experiments and section 7 summaries the main
conclusions of this work.
2. State of the art
The problem addressed in this paper is one of the prob-
lems included in the research line of text normalization
[1]. In this reference, the whole text normalization prob-
lem is described. Sproat and al present a Non-Standard
Word (NSW) taxonomy with several proposals of auto-
matic strategies for classifying every NSW within one of
the taxonomy classes. According to acronyms, Sproat
and al propose to consider acronyms as any other stan-
dard words and to use a dictionary (list of standard words)
for discriminating between standard words and NSWs.
This paper proposes an automatic method for selecting
the best prononciation of unknow CLSs (not included in
the dictionary).
In the literature, there are some research efforts fo-
cused on how to extract acronyms from raw text auto-
matically [2] [3] [4]. In this studies, the main target is to
increase the list of acronyms, in order to reduce the prob-
ability of being unknown. On the other hand, there are
several works trying to model the creation of acronyms
[5] or abbreviations [6]. From these efforts, it is possi-
ble to conclude that the syllable and letter structure are
important aspect in the creation of acronyms and abbre-
viations. Based on these ideas, this paper will consider
features based on a letter language model.
3. Database description
Table 1 summaries the main characteristics of the
database considered in this work. This database has been
extracted from several months of the ”El Mundo” news-
paper [7]. The extraction process has been performed
automatically (looking for words in capital letters) with a
posterior manual revision.
Table 1: Database characteristics.
Characteristics Acronyms Abbreviations
Number of examples 653 684
Percentage 48.8% 51.2%
Average length 3.8 3.3
Maximum length 8 5
The first characteristic is that the percentage of
acronyms is very high, very close to the percentage of
abbreviations. This is due to the high relationship be-
tween letters and phonemes in Spanish. Another charac-
teristic is that the average length of acronyms is higher. A
smaller letter sequence (2 or 3 letters) has less probability
of generating an acronym.
4. Main features based on letter sequences
This work has considered several features based on a let-
ter language model (LLM) in order to decide if a Capital
Letter Sequence (CLS) must be spelled or pronounced as
a standard word. This language model has been gener-
ated considering the acronyms from the training set (in a
Cross-Validation strategy described in section 6). Con-
sidering the maximun lengths reported in table 1, a 6-
gram letter language model has been considered. This
LLM has been trained using the IRSTLM toolkit [8]. In
order to train this model, two marks for indicating the be-
ginning and the end of the letter sequence have been con-
sidered (ini and end). For example, the letter sequence
corresponding to the abbreviation IND, is ”ini I N D end”.
This way, it is possible to incorporate information about
the boundaries. The features derivated from the LLM are:
• Letter Sequence Perplexity (LS Perplexity). This
feature is the perplexity of a CLS given a LLM.
Considering that the LLM was generated using
acronyms from the training set, it is expected that
acronyms have lower perplexity than abbrevia-
tions.
• Minimum Probability (Min Prob). In some cases,
part of the CLS can be very common in Span-
ish words while there are other parts forbidden in
standard Spanish words. The perplexity, in these
cases, can be small but the CLS should be re-
jected (not considered as an acronym). For exam-
ple: MNAC, NAC has a very high probability and
only the sequence MN generates a low probabil-
ity. Considering the whole perplexity some local
details are missed. Computing the minimum prob-
ability along the letter sequence can report addi-
tional information.
• Maximum Probability (Max Prob). Similar to the
previous feature, in this case, the maximum prob-
ability computed along the letter sequence is con-
sidered.
• Average N-gram (Ave N-gram). This measure-
ment is the average N-gram order for computing
the probability of every letter. This feature tries to
complement the information reported with the per-
plexity. Acronyms are expected to have a higher
average N-gram order than abbreviations. For ex-
ample, the N-gram orders for computing the proba-
bilities in the MNAC sequence are: ini M(2-gram)
N(1-gram) A(2-gram) C(3-gram) end(2-gram). In
this case, the Average N-gram order is 10 / 5 = 2.
• Minimum N-gram (Min N-gram). Similar to the
minimum probability, the minimum N-gram order
is considered for reporting more details about local
behaviours.
• Maximum N-gram (Max N-gram). In order to
complete the analysis, the maximum N-gram order
is also considered.
5. Feature Analysis
Fig. 1 shows the ROC curves for all the features. False ac-
ceptation is the percentage of cases where the system con-
siders an abbreviation as an acronym by mistake, trying
to pronounce it as a standard word. False rejection is the
percentage of cases where the system classifies (wrongly)
an acronym as an abbreviation, spelling it instead of pro-
nouncing it as a standard word. As it is shown, the best
features are the Letter Sequence Perplexity and the Aver-
age N-gram order. In both cases, it is possible to reach
a 10% EER (Equal Error Rate). Minimun and Maximun
N-gram order features have a discrete number of possible
values, so the ROC curves present straight lines in some
places.
Additional to the ROC curves, table 2 shows the In-
formation Gain for each feature. Similar to the ROC
curves, the best features are LS Perplexity and Ave N-
gram. The worst ones are the Max Prob and the Max N-
gram. This analysis has been performed using the WEKA
toolkit [9].
Table 2: Information Gain Analysis
Feature Information Gain
Ave N-gram 0.67
LS Perplexity 0.66
Min Prob 0.47
Min N-gram 0.42
Max N-gram 0.40
Max Prob 0.36
ROC
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
False Acceptation
F a
l s
e
 
R e
j e c
t i o
n
LS Perplexity
Min Prob
Max Prob
Ave N-gram
Min N-gram
Max N-gram
Figure 1: ROC for the different features.
6. Classification experiments
For the classification experiments, a Cross-Validation
strategy has been considered. The whole database has
been ramdonly divided into 10 subsets: eight for training
the LLM, one for tunning the classifier parameters and
one for testing. The experiments are repeated 10 times
and the results are the average of all runs.
6.1. Grouping Features
The first analysis is about how to group the different fea-
tures. Several alternatives have been considered and eval-
uated in table 3. These grouping alternatives are: the two
best features, the LLM probability based features, the N-
gram order based features, and considering all of them.
Table 3 shows the classification error and the ROC area
(area under the ROC curve) for the different alternatives.
In this case, a simple classifier has been considered: a
naive Bayes classifier. This classifier is a simple proba-
bilistic classifier based on applying Bayes’ theorem as-
suming independence between features.
Table 3: Classification experiments considering several
grouping alternatives
Features group ROC area Class. error
Ave N-gram + LS Pp 0.968 12.04%
LS Pp, Min + Max Prob 0.962 11.90%
Ave + Min + Max N-gram 0.966 13.98%
All the features 0.968 11.20%
As it is shown in table 3, LLM probability based fea-
tures perform better than N-gram based ones. The best
results are obtained considering all the features.
6.2. Using Several Classifiers
This sub-section presents experiments considering differ-
ent classifiers for combining all the features. Table 4
shows the classification error and the ROC area (area un-
der the ROC curve) for the different alternatives:
• Naive Bayes. Same as the classifier considered in
the previous section.
• Bayesian Network. This classifier is based on
Bayesian Networks. A Bayesian Network is a
probabilistic graphical model that represents a set
of random variables and their conditional depen-
dencies via a directed acyclic graph (DAG).
• Multi-Layer Perceptron. A Multi Layer Perceptron
is a feedforward artificial neural network consist-
ing of multiple layers of nodes in a directed graph
(3 layers in this case), with each layer fully con-
nected to the next one. Except for the input nodes,
each node is a neuron (or processing element) with
a nonlinear activation function (sigmoide).
• Decision Tree using the C4.5 algorithm [10].
• Attribute Selected Classifier. This is meta-
classifier that takes a search algorithm and evalua-
tor next to the base classifier. In this case, the base
classifier is the decision tree generated using the
C4.5 algorithm.
Table 4: Classification experiments with different classi-
fiers
Features group ROC area Class. error
Naive Bayes 0.968 11.20%
Bayesian Network 0.969 10.40%
Multi-Layer Perceptron 0.967 9.57%
Decision Tree C4.5 0.940 8.67%
Attribute Selected Classifier 0.943 8.45%
In these experiments (table 4), the best classifier has
been the decision tree obtained with the C4.5 algorithm.
In this case, the classification error is 8.67%. Using the
meta-classifier (Attribute Selected Classifier) the classifi-
cation error is 8.45%.
Fig. 2 shows the top nodes of the decision tree ob-
tained (using WEKA). As it is shown, decision over Av-
erage N-gram, LS perplexity and Min N-gram features
are situated at the top nodes.
Fig. 3 shows the ROC curves for all the classifiers.
False acceptation represents the percentage of abbrevia-
tions classified as acronym by mistake, and the false re-
jection is the percentage of acronyms classified as abbre-
viations. When classifying an acronym as an abbreviation
(false rejection), the system will spell the CLS instead of
pronouncing it as a standard word. On the other hand,
when the system classifies an abbreviation as an acronym
(false acceptance), the system will try to pronounce it as
Figure 2: Top nodes of the decision tree
ROC for all classifiers
0%
5%
10%
15%
20%
25%
30%
0% 5% 10% 15% 20%
False Acceptation
F a
l s
e
 
R
e
j e c
t i o
n
Naive Bayes
Bayesian Network
Multi-layer Perceptron
Decision Tree
Meta classifier
Working 
point
Figure 3: ROC curves considering the different classi-
fiers.
a standard word generating a wrong pronunciation. This
second type of error is the most dangerous one. Because
of this, in order to define a working point, the false accep-
tance must be lower than 5%. Table 5 shows the false re-
jection for several working points (false acceptation per-
centages), considering the best system in this range: the
Bayesian Network.
Table 5: Different working points
False Acceptance False Rejection
5.0% 15.1%
2.5% 18.5%
1.0% 23.5%
7. Conclusions
This paper has presented an automatic strategy to clas-
sify Capital Letter Sequences (CLSs) as abbreviation or
acronyms, when these CLSs are unknown by the TTS
(they are not in the system vocabulary). The abbrevi-
ations will be spelled while the acronyms will be pro-
nounced as standar words. In Spanish, there is a high per-
centage of acronyms that makes this work very interest-
ing. This paper has also analysed the discrimination capa-
bility of some features based on a Letter Language Model
(LLM): letter sequence perplexity, minimum and maxi-
mum probability and, average, minimum and maximum
N-gram order when computing the probability along the
letter sequence. The best features have been the letter
sequence perplexity and the average N-gram order. Fi-
nally, the paper has evaluated several classifiers for com-
bining the differents features. The lowest classification
error (8.45% in table 4) has been obtained with a meta
classifier that uses a decision tree based on the C4.5 algo-
rithm (this has been the best meta classifier in this task).
When analysing the working point (a false acceptance
lower than 5%), the best classifier, in this range, is the
Bayesian Network.
8. Acknowledgements
The work leading to these results has received fund-
ing from the European Union under grant agree-
ment n 287678. It has also been supported
by TIMPANO (TIN2011-28169-C05-03), INAPRA
(MICINN, DPI2010-21247-C02-02) and MA2VICMR
(CAM, S2009/TIC-1542) projects. Authors also thank
the other members of the Speech Technology Group and
Simple4All project for the continuous and fruitful discus-
sion on these topics.
9. References
[1] Sproat, R., et al. ”Normalization of non-standard words.” Com-
puter Speech and Language, 15(3), 287-333, 2001.
[2] Larkey, Leah, Paul Ogilvie, Andrew Price and Brenden Tamilio.
”Acrophile: An Automated Acronym Extractor and Server”, In
Proceedings of the ACM Digital Libraries conference, pp. 205-
214, 2000.
[3] Stuart Yeates. ”Automatic Extraction of Acronyms from Text”.
New Zealand Computer Science Research Students’ Conference.
1999.
[4] J.T. Chang, H Schtze, and R.B. Altman. 2002. ”Creating an On-
line Dictionary of Abbreviations from MEDLINE” JAMIA.
[5] Cannon, Garland. 1989. ”Abbreviations and acronyms in English
word-formation.” American Speech, 64:99127.
[6] Pennell and Liu. 2011. ”Toward text message normalization:
Modeling abbreviation generation.” Proceedings of the IEEE. pp.
5364-5367.
[7] El Mundo Newspaper (1996-2006). New from the ”El Mundo”
newspaper during several years.
[8] Federico, M. and Cettolo, M. (2007) Efficient Handling of N-
gram Language Models for Statistical Machine Translation Proc.
of ACL Workshop on SMT. pages 8895, Prague, Czech Republic.
[9] Witten, I.H. and Frank, E. (2005) Data Mining: Practical machine
learning tools and techniques. 2nd edition Morgan Kaufmann, San
Francisco.
[10] Quinlan, J. R. C4.5: Programs for Machine Learning. Morgan
Kaufmann Publishers, 1993.


http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.249.5803

Detecting acronyms from capital letter sequences in Spanish

Abstract

Similar works

Full text

Available Versions

Edinburgh Research Explorer

Edinburgh Research Explorer

Archivo Digital UPM

Servicio de Coordinación de Bibliotecas de la Universidad Politécnica de Madrid