Asaei, Afsaneh

Bourlard, Hervé

Dighe, Pranay

Infoscience - École polytechnique fédérale de Lausanne

Dictionary Learning for Sparse Representation ofNeural Network Exemplars in Speech RecognitionPranay Dighe∗†, Afsaneh Asaei∗, and Herve´ Bourlard∗†∗Idiap Research Institute, Martigny, Switzerland, †E´cole Polytechnique Fe´de´rale de Lausanne, SwitzerlandEmails: {pranay.dighe, afsaneh.asaei, herve.bourlard}@idiap.chAbstract—Conventional speech recognition systems relyingon exemplar-based sparse representation require huge sizeexemplars collection to represent the linguistic units. Recentwork demonstrates that despite of consistent improvement inautomatic speech recognition performance, increasing the sizeof exemplars collection after a certain (very large) dimensionleads to only minor improvements [1]. This observationsuggests the need for a better procedure to find a limitedsize collection of exemplars that can be used for sparserepresentation. In the present study, the exemplars are neuralnetwork sub-word conditional posterior probabilities. In thiscontext, we study the application of dictionary learningfor sparse modeling. We demonstrate that the posteriorexemplars live in a low-dimensional manifold that can bemodeled as a union of subspaces. Furthermore, we evaluatethe performance of dictionary learning for exemplar-basedspeech recognition to compare and contrast it with thetraditional exemplars collection approach.I. UNION OF SUBSPACES MODELDictionary learning for sparse representation relies onthe assumption that the data can be modeled as a union ofsubspaces. In this section, we provide supporting evidencethat the neural netwrok exemplars conform to this model.We perform a simple experiment of template match-ing using dynamic time warping (DTW) for 75 words-vocabulary set of Phonebook database [2]. Exemplars hereare in form of (deep) neural network based phone posteriorprobabilities [3]. Out of 11 utterances for each word, wekeep 4 utterances as training templates and use the rest fortesting. The 4 utterances in the training set were used tocreate 15 combinatorial, 1 to 4-sparse templates for DTWmatching by averaging after alignment.1-sparse templates :{TU1 , TU2 , TU3 , TU4}2-sparse templates :{TU1U2 , TU1U3 , TU1U4 , TU2U3 , TU3U4}3-sparse templates :{TU1U2U3 , TU2U3U4 , TU1U2U4 , TU1U3U4}4-sparse templates :{TU1U2U3U4}(1)We then quantify the DTW distance of the test utteranceswith the new constructed templates. The weighted sym-metric Kullback-Leibler (KL) divergence is used as thedistance measure as it was shown to be an “optimal” metricfor neural network exemplars [4]. The smaller distanceindicates better characterization of the test templates usingthe training data. This experiment is run for all test dataand the results are listed in Table I. We observe that only4.9% of test utterances have the least characterizationerror using a single closest template (DTW assumption).Moreover, only 9.7% are best characterized by the templateobtained from averaging the full training set (KL-hiddenMarkov model - HMM assumption [3]). On the otherhand, all remaining 85.4% of the utterances have theleast characterization error using the templates which areobtained as a combination of a few (2 or 3) trainingexemplars. This observation confirm the hypothesis of theeffectiveness of the union of subspace approach to modelthe neural network exemplars.Another experiment was conducted on Numbersdatabase [5], where we have huge amount of trainingdata. Instead of k-sparse template matching using DTWas in Phonebook experiment, here we learn dictionariesof size of order ∼1000 columns from training data.We perform sparse recovery of test data using thesedictionaries and analyse the support-size (number of non-zeros coefficients) of sparse representation. The resultsare illustrated in Figure 1. We observe that 31% of thetest exemplars are represented by one dictionary columnwhereas 69% are characterized by a linear combinationof very small number of columns corresponding to theirsparse representation. For dictionary learning and sparserecovery, online dictionary learning algorithm [6] and lassosolver [7] were used respectively.II. DICTIONARY LEARNINGOnce we confirm that the union of subspace model holdsfor neural network exemplars, we demonstrate experimen-tally that dictionary learning improves characterisationof the feature space as compared to a simple collectionof all exemplars of the training set while its cardinalityis still far smaller than the collection size. In isolatedword recognition experiment on Phonebook 75-vocabularydataset, a single exemplar is used as a warm start fordictionary initializing. The remaining 3 exemplars in thetraining set are then used for updating the dictionarycolumns using online dictionary learning algorithm [6].Alternatively, 4 training exemplars are concatenated toform a dictionary for sparse representation. A similarcomparison was done for connected digit recognition onNumbers database, where we can either learn word-specificdictionaries or we can directly represent each word usingall training exemplars [8]. The results are listed in Table II.We can see that the dictionary learning procedure is quiteeffective; it can benefit from the abundance of the trainingdata, while it enables us to keep the dimensionality ofthe exemplar space small and at the same time improvethe performance. This observation confirms that dictionarylearning is a more efficient way for sparse representationthan exemplar collection.0 1 2 3 4 5 6 7 8 9 1002000400060008000100001200014000Number of non−zero coefficients insparse representation →Number of Test Exemplars →(a)100 101 10210−710−610−510−410−310−210−1100Coefficient Index →|Coefficients| →(b)Fig. 1: Analysis of Sparsity: (a) gives a plot depicting the number of test exemplars versus the number of non-zerocoefficients in their sparse representation and (b) shows the fast decay of the non-zero coefficients (blue solid line) ascompared to the power-law decay (red dash line).Winning HypothesesAverage DTWMatching Cost1-sparse 23/464 95.602-sparse 177/464 84.773-sparse 219/464 84.144-sparse 45/464 86.56TABLE I: Comparison of k-sparse templates for characteri-sation of the test exemplars using the collection of trainingexemplars. The total number of test exemplars is 464, outof which 23 test exemplars has the least characterizationerror if a single training exemplar is used. Similarly, 177,219 and 45 of them have the least characterization error if 2,3 and 4 of the training exemplars is used for representationrespectively. This observation is inline with the union ofsubspace model of neural network exemplars.Task Dictionary Collection of ExemplarsPhonebook 97.2 97.0Numbers 85.4 78.6TABLE II: Comparing the speech recognition accuracy (%)on Phonebook (isolated word recognition) and (connectedword recognition using dictionary learning versus collectionof exemplars. Accuracies in case of Connected Digit are givenby (100 - WER), where WER is word error rate obtained byLevenshtein distance. The size of training data in Phonebookis small. In this case the dimension of dictionary exemplars(number of learned atoms) is 25% of the full training set. Thesize of training data in Numbers corpus is large. In this casethe dimension of dictionary exemplars (number of learnedatoms) is ∼3% of the full training set.ACKNOWLEDGMENTThe research leading to these results has receivedfunding from by SNSF project on “Parsimonious Hier-archical Automatic Speech Recognition (PHASER)” grantagreement number 200021-153507. The authors wouldlike to acknowledge Dr. David Imseng for his assistancewith speech recognition experiments.REFERENCES[1] J. Gemmeke, L. Ten Bosch, L. Boves, and B. Cranen,“Using sparse representations for exemplar basedcontinuous digit recognition,” in Proc. EUSIPCO.Citeseer, 2009, pp. 24–28.[2] J. Pitrelli, C. Fong, S. Wong, J. Spitz, and H. Le-ung, “Phonebook: a phonetically-rich isolated-wordtelephone-speech database,” in Acoustics, Speech, andSignal Processing, 1995. ICASSP-95., 1995 Interna-tional Conference on, vol. 1, May 1995, pp. 101–104vol.1.[3] G. Aradilla, H. Bourlard, and M. Magimai-Doss,“Using KL-based acoustic models in a large vocabularyrecognition task.” in INTERSPEECH, 2008, pp. 928–931.[4] S. Soldo, M. Magimai-Doss, J. Pinto, and H. Bourlard,“Posterior features for template-based ASR,” in Acous-tics, Speech and Signal Processing (ICASSP), 2011IEEE International Conference on. IEEE, 2011, pp.4864–4867.[5] R. A. Cole, M. Noel, T. Lander, and T. Durham, “Newtelephone speech corpora at cslu,” 1995.[6] J. Mairal, F. Bach, J. Ponce, and G. Sapiro, “Onlinelearning for matrix factorization and sparse cod-ing,” Journal of Machine Learning Research (JMLR),vol. 11, pp. 19–60, 2010.[7] B. Efron, T. Hastie, I. Johnstone, R. Tibshirani et al.,“Least angle regression,” The Annals of statistics,vol. 32, no. 2, pp. 407–499, 2004.[8] J. F. Gemmeke, T. Virtanen, and A. Hurmalainen,“Exemplar-based sparse representations for noise robustautomatic speech recognition,” Audio, Speech, andLanguage Processing, IEEE Transactions on, vol. 19,no. 7, pp. 2067–2080, 2011.

Dictionary Learning for Sparse Representation of Neural Network Exemplars in Speech Recognition

http://infoscience.epfl.ch/record/210015/files/Dighe_SPARS-2_2015.pdf

Dictionary Learning for Sparse Representation of Neural Network Exemplars in Speech Recognition

Abstract

Similar works

Full text

Available Versions

Infoscience - École polytechnique fédérale de Lausanne