Support vector machines with the Fisher and score-space kernels are used for text independent speaker verification to provide direct q discrimination between complete utterances. This is unlike approaches such as discriminatively trained Gaussian mixture models or other discriminative classifiers that discriminate at the frame-level only. Using the sequence-level discrimination approach we are able to achieve error-rates that are significantly better than the current state-of-the-art on the PolyVar database

Renals, Steve

Wan, Vincent

Edinburgh Research Archive

SVMSVM: SUPPORT VECTOR MACHINE SPEAKER VERIFICATION METHODOLOGYVincent Wan, Steve RenalsDepartment of Computer Science,University of Sheffield,211 Portobello Street,Sheffield S1 4DP, UK. v.wan, s.renals  @dcs.shef.ac.ukABSTRACTSupport vector machines with the Fisher and score-space ker-nels are used for text independent speaker verification to providedirect discrimination between complete utterances. This is un-like approaches such as discriminatively trained Gaussian mix-ture models or other discriminative classifiers that discriminate atthe frame-level only. Using the sequence-level discrimination ap-proach we are able to achieve error-rates that are significantly bet-ter than the current state-of-the-art on the PolyVar database.1. INTRODUCTIONCurrent state-of-the-art speaker verification systems are based ongenerative models, particularly Gaussian mixture models (GMMs)and hidden Markov models (HMMs), placed in a discriminativeframework [1]. In these systems classifiers are trained to discrim-inate between individual frames of data then the likelihood scoresof each frame are combined using an averaging step [2] or via anHMM to give an overall utterance score from which the authentic-ity of the speaker may be determined.A discriminative classifier discards information that the ob-jective function considers irrelevant. Discrimination at the frame-level is therefore not optimal for sequence classification since in-formation relevant to sequence discrimination may be discardedinadvertently.The support vector machine [3, 4] (SVM) coupled with a Fisherkernel [5] or, more generally, a score-space kernel [6, 7] enablesdirect discrimination between complete utterances. These kernelfunctions map a complete utterance onto a single point using a gen-erative model. Such a representation enables any suitable classifierto discriminate between complete sequences. However, in order torepresent a complete sequence adequately, the single data pointmust exist in a high dimensional space (determined by the numberof parameters in a generative model for the sequence). Classifica-tion of high dimensional data is a task well suited to SVMs.In this paper we extend previous work [8, 9] on applying theSVM to text independent speaker verification. Our experimentswere performed on the PolyVar database [10] and we report errorrates that are better than the current state-of-the-art by 25%.Vincent Wan was sponsored by a Motorola Studentship.2. BASELINE SYSTEMS2.1. Gaussian mixture modelsWe used diagonal covariance GMMs as a baseline classifier. Thesame GMMs are used as the underlying generative model in sec-tion 3 to achieve a mapping from a variable length sequence offeature vectors to a single fixed length vector.The probability, P  X M  , that the observed sequence, X x1  xN 	 , is generated by the model of the client, M, is usedas the utterance score, S  X  . It is estimated by the mean log likeli-hood over the sequence,S  X  logP  X M  1NN∑i  1 logP  xi M  (1)where P  xi M  is the client GMM likelihood for the ith frame. Tomake a decision, the utterance score is compared to a threshold.A better system incorporates a GMM of the impostors, Ω.Taking the ratio of the client model likelihood to the impostormodel likelihood, the classifier takes into account the probabil-ity distribution of the impostors resulting in a more discriminativeclassifier [2]. The utterance score is the difference between the loglikelihoods of the two models,S  X  log P  X M P  X Ω  logP  X M  logP  X Ω  (2)This forms the basis of many state-of-the-art speaker verificationsystems.2.2. GMM/SVM systemThe combination of a GMM likelihood ratio (GMM-LR) with anSVM to incorporate a generative model into a discriminant frame-work was done by Bengio and Mariéthoz [11]. By Bayes decisionrule, equation (2) is optimal so long as the client and impostors arewell modelled. Bengio and Maritéhoz proposed that the probabil-ity estimates are not perfect and that a better version would be,a log P  X M  b logP  X Ω  c (3)where a, b and c are adjustable parameters. Given a set of train-ing data and labels, these parameters may be estimated using anylearning algorithm. Bengio and Mariéthoz used an SVM to learnthese parameters. The input to the SVM is the two dimensionalvector made up of the log likelihoods of the client and impostormodels.Table 1. Some examples of score operatorsOperator expressionfirst derivative F̂  ∇θfirst derivative and argument F̂   ∇θ  1  Tfirst and second derivative F̂   ∇θ  vec  ∇2θ  T  T3. DISCRIMINATIVE SEQUENCE CLASSIFICATIONThe classifiers described in section 2 are trained to discriminatebetween individual frames of data. However, the classification ofthe sequence as a whole is desired. Discriminative classification ofa sequence may be achieved if a discriminant classifier is allowedto observe the entire sequence. Classifying sequences directly isdifficult since they have different lengths. A fixed-length represen-tation of the sequence may, however, be classified directly.The method for mapping from an arbitrary length sequence toa fixed length feature vector was first developed by Jaakkola andHaussler [5] and is known as the Fisher kernel. A more general ap-proach known as score-spaces, from which the Fisher kernel maybe derived, was developed by Smith et. al. [6, 7].3.1. The score-space approachThe feature space to which utterances are mapped is called thescore-space, so named because it is defined by and derived fromthe likelihood score of a generative model. Given a set of k gener-ative models,pk  X  θk  	 , the generic formulation for mapping asequence, X , to the score-space is,ψ fF̂ X  ψF̂ f   pk  X  θk  	  (4)where f   pk  X  θk  	  , a function of the scores of a set of gener-ative models, is called the score-argument and ψF̂ is the score-mapping that maps the scalar score-argument to the score-spaceusing the score-operator, F̂ . The properties of the resulting score-space depends upon the choice of operator and argument that isused. Several options for score-operators were proposed by Smithet. al. [6, 7] and have been summarised in table 1.Almost any function may be used as a score-argument. Weshall show two specific cases that lead to the likelihood score-space kernel (more commonly known as the Fisher kernel) andthe likelihood ratio score-space kernel.3.2. The likelihood score-space (Fisher) kernelBy setting the score-argument to be the log likelihood of a singlegenerative model, M, parameterised by θ,f   pk  X  θk  	   logP  X M  θ  (5)and choosing the first derivative score-operator from table 1 weobtain the mapping for the likelihood score space.ψ  X  ∇θ logP  X M  θ  (6)This mapping is also known as the Fisher mapping and was firstdeveloped and applied successfully for biological sequence analy-sis by Jaakkola and Haussler [5].Using the first derivative and argument score-operator and thesame score-argument the mapping becomesψ  X ∇θ logP  X M  θ logP  X M  θ  (7)The feature space defined by this mapping is identical to the Fishermapping with one additional feature dimension: the log likelihoodscore itself. This mapping has the benefit that no information withrespect to the original generative model, M, is lost in the transfor-mation.3.3. The likelihood ratio score-space kernelAn alternative score-argument is the log likelihood ratio of twogenerative models, a client model, M, and an impostor model, Ω,f   pk  X  θk  	   log P  X M  θ1 P  X Ω  θ2  (8)where θ    θ1θ2  . The corresponding mapping using the first deriva-tive score-operator is,ψ  X  ∇θ log P  X M  θ1 P  X Ω  θ2  (9)and using the first derivative with argument score-operator,ψ  X   log P 	 X M  θ1 P 	 X Ω  θ2 ∇θ logP 	 X M  θ1 P 	 X Ω  θ2  (10)The likelihood ratio score-space is motivated by the likelihood ra-tio GMM (GMM-LR) classifier described in section 2.1. In thesame way that the GMM-LR is a more discriminative classifierthan a single GMM alone, then so too should the correspondingscore-space kernel.4. NORMALISING SEQUENCE KERNELSClassifying the features, Ψ  X  , derived from the transformationdescribed in section 3 directly will not immediately yield a goodperformance when using SVMs. We have used two stages of nor-malisation in order to make this method work. The first stage isto whiten the data in the feature space, which involves normalis-ing the vectors, Ψ  X  to zero mean and unit variance. The secondstage may be interpreted as a Hessian preconditioning step andinvolves making a further nonlinear transformation to a higher di-mensional space.4.1. Feature space whiteningFor a given score-space, the metric of the space is defined by thegenerative models and is non-Euclidean. To correctly compute thedot product in any space requires the Riemannian metric of thespace. A kernel constructed from any of the above mappings is,K  X  Y   ψ  X  T Gψ  Y  (11)where G defines the metric of the space. In the case of the Fishermapping, G is the inverse Fisher information matrix.G   E  ψ  X  ψ  Y  T  1  (12)assuming that E  ψ  X    E  ψ  Y   0. In general, G is the in-verse of the covariance matrix of the data in score space. This canbe interpreted as a whitening step where the features in score spaceare normalised to zero mean and unit variance.Correct whitening of the data relies on the ability to computea full covariance matrix, which will not only normalise the scalingin each dimension but also make the principal component axes ofthe space orthogonal. However, the feature space dimensionality,being equal to the number of parameters in the generative model,may be large. If there were 105 parameters in the generative model(a small number by many standards) then the full covariance ma-trix would have 1010 elements. A covariance matrix of this size isimpractical to compute and invert. As an approximation the datamay be normalised by the diagonal covariance matrix so that thescale of each dimension is the same.4.2. Spherical normalisationThe second stage of normalisation is a preconditioning step thatinvolves a transformation that maps each feature vector onto thesurface of a sphere. The technique, which we call spherical nor-malisation, was first used in [9] to enable high order polynomialkernel SVMs to be trained. It involves mapping each feature vec-tor onto the surface of a unit hypersphere embedded in a space thathas one dimension more than the feature vector itself. The specificmethod adopted in this paper is to augment each vector with anextra component, d, and then normalise by the L2 norm,Ψsph  X  1 Ψ  X  2  d2Ψ  X d   (13)The feature vectors on a unit hypersphere are normalised tounit length but when d is set appropriately then no informationis lost in the transformation. The effect is to make the dot prod-uct between any two vectors better scaled (dot products betweenunit vectors lie between minus one and plus one) and to makethe Hessian diagonally dominant. Without the normalisation, themagnitudes of the dot products may vary greatly, especially whenthe dimensionality of the data is high, such as the case when thelikelihood ratio score-space kernel is used. These large variationsin magnitudes result in an ill-conditioned Hessian for which thequadratic programming (the method for optimising SVMs) solu-tion is more difficult to find. Spherical normalisation alleviatesthis problem.5. EXPERIMENTSThe PolyVar database [10] was used in our speaker verificationexperiments. The database was recorded over a telephone networkand consists of 38 client speakers (24 male and 14 female) eachwith 85 utterances recorded in 5 sessions with 17 utterances persession. There are also 952 impostor utterances from 56 speakers,each contributing 17 utterances in a single session. A strictly de-fined protocol for training an testing was adhered to. There wereapproximately 1000 test utterances (including both client and un-seen impostors) for each client speaker.The speech was processed using perceptual linear predictionwith a 32ms window and a 10ms time shift to obtain 12 cepstralcoefficients and one energy term. These features were augmentedby their first and second order derivatives yielding 39 dimensionalfeature vectors. Cepstral mean subtraction was applied to removethe effects of the communication channel. Silence frames withineach utterance were segmented out using a pretrained MLP.5.1. The GMM baselinesTwo GMM baseline systems were used in this evaluation. TheGMM and GMM likelihood ratio (GMM-LR) described in section2.1 are used. The classifier that gives state-of-the-art results is thecombined GMM/SVM system described in section 2.2. Results onthe PolyVar database for the GMM-LR and the GMM-LR/SVMhave already been published by Bengio and Mariéthoz [11]. Tomake a fair evaluation the baseline models were trained using ex-actly the same conditions used by Bengio and Mariéthoz.The impostor Gaussian mixture model contained 1000 Gaus-sians. This number was obtained by Bengio and Mariéthoz usingcross-validation. Models of various sizes were trained using 90%of the training data while the remaining 10% was used to test themodels. The model with the highest likelihood on the validationdata was chosen.The number of Gaussians in the client model was found usingten-fold cross-validation. The training data was randomly parti-tioned into ten equally sized sets. Every client model was trainedon nine of the sets and tested on the one remaining set. This was re-peated ten times, each time with a different test set. Repeating thewhole process using different numbers of Gaussians in the mod-els Bengio and Mariéthoz found the model that yielded the highestoverall likelihood had 200 components.Using the GMM-LR, Bengio and Mariéthoz reported a text in-dependent speaker verification result of 5.55% HTER on the Poly-Var test data using 19 speakers. A 4.73% HTER was reported forthe GMM-LR/SVM. These results were successfully replicated.The results obtained using 38 speakers are shown in table 2.5.2. Score-space kernelsThe number of training utterances per SVM for the Fisher and LRkernels was 85 client utterances and 952 impostor utterances mak-ing a total training size of 1037 utterances per SVM. The Fisherkernel used the derivatives of the client GMMs of the baseline sys-tem to achieve the score-space mapping. The likelihood ratio (LR)kernel used both client and impostor models of the baseline sys-tem. The feature space dimensionality of the Fisher and LR ker-nels were 15800 and 94800 respectively. Using the correspondingscore-operator with argument includes the log likelihood or loglikelihood ratio score of the GMMs making the dimensionalities15801 and 94801 respectively. The high feature space dimension-ality, particularly for the LR kernel, causes computational prob-lems for SVMs.To compute the Hessian efficiently the training data must beheld in memory. The memory needed to store the training featurevectors is 94800  1037  8 bytes or 750 MB, assuming doubleprecision floating point arithmetic is used. The SVM optimisationmatrices were therefore computed in small blocks leaving featurevectors not in use stored on hard disk. This allowed us to usethe complete set of features derived from the GMMs for training.Since linear SVMs were used, the resultant vector could alwaysbe computed from the support vectors. This saves memory andcomputation during testing.In experiments where spherical normalisation was used, themapping onto the hypersphere of equation (13) was applied ex-plicitly to the feature vectors. Since the components of the featureTable 2. Results of the PolyVar experiments. The GMM baselineconsisted of 200 diagonal covariance Gaussian components mod-elling the clients. The GMM likelihood ratio (GMM-LR) consistsof the above plus a GMM with 1000 diagonal covariance Gaus-sian components modelling the impostors. The GMM/SVM is thestate-of-the-art classifier, which uses the same client and impostorGMMs but computes a weighted log likelihood ratio. The spheri-cal normalisation constant was set to one in instances where it wasused. The half total error rates (HTER) and equal error rates (EER)are shown.Classifier % HTER % EERBaselineGMM 11.22 12.07GMM-LR 5.53 6.12GMM/SVM 5.37 5.94Fisher kernelwithout argument 6.54 6.98without argument & sph. norm 6.50 6.87with argument 6.50 6.92with argument & sph. norm 6.47 6.87LR kernelwithout argument 5.13 5.55without argument & sph. norm 3.72 4.03with argument 5.03 5.55with argument & sph. norm 3.71 4.03vectors were normalised to zero mean and unit variance we set thespherical normalisation parameter, d  1.Table 2 shows a summary of the test results and figure 1 theDET curves. It is clear from the results that sequence level discrim-ination using the LR kernel is beneficial to speaker verification. Itcan also be seen that spherical normalisation makes a significantdifference when training SVMs on the high dimensional data ofthe likelihood ratio score-space kernel.6. CONCLUSIONWe have applied SVMs to speaker verification, using sequencekernels such as the Fisher kernel and the likelihood ratio kernel.On the PolyVar database, we reduced error rates by up to 33%relative compared to the state-of-the-art. The use of spherical nor-malisation was the key to achieving this result.7. REFERENCES[1] G. Doddington, M. Przybocki, A. Martin, and D. Reynolds,“The NIST speaker recognition evaluation – overview,methodology, systems, results, perspective,” Speech Com-munication, vol. 31, no. 2-3, pp. 225–254, 2000.[2] D. A. Reynolds, “Speaker identification and verification us-ing Gaussian mixture speaker models,” Speech Communica-tion, vol. 17, pp. 91–108, 1995.[3] V. N. Vapnik, The Nature of Statistical Learning Theory,Springer, 1995.0.1 0.2 0.5 1 2 5 10 20 400.10.20.5125102040False Alarm probability (in %)Miss probability (in %)gmmlr kernelfisher kernelgmm/svmgmm-lrFig. 1. The DET curves of the classifiers are shown: the base-line Gaussian mixture model (gmm), the baseline likelihood ra-tio GMM (gmm-lr), the likelihood score-space (without argument)kernel SVM (fisher kernel), the likelihood ratio score-space (with-out argument) kernel SVM (lr kernel) and the state-of-the-art com-bined GMM/SVM classifier (gmm/svm).[4] C. J. C. Burges, “A tutorial on suport vector machines forpattern recognition,” Data Mining and Knowledge Discov-ery, vol. 2, no. 2, pp. 1–47, 1998.[5] T. S. Jaakkola and D. Haussler, “Exploiting generative mod-els in discriminative classifiers,” in Advances in Neural In-formation Processing Systems 11, M. S. Kearns, S. A. Solla,and D. A. Cohn, Eds. MIT Press, 1999.[6] N. Smith and M. J. F. Gales, “Using SVMs and discrimina-tive models for speech recognition,” in Proc. ICASSP, 2002.[7] N. Smith and M. Gales, “Speech recognition using SVMs,”in Advances in Neural Information Processing Systems 14.2002, MIT Press.[8] V. Wan and S. Renals, “Evaluation of kernel methods forspeaker verificationa and identification,” in Proc. ICASSP,2002.[9] V. Wan and W. M. Campbell, “Support vector machines forspeaker verification and identification,” in Proc. Neural Net-works for Signal Processing X, 2000, pp. 775–784.[10] A. Constantinescu and G. Chollet, “Swiss polyphone andpolyvar: Building databases for speech recognition andspeaker verification,” in Proceedings of The 3rd Slovenian-German and 2nd SDRV Workshop, Speech and Image Un-derstanding, April 24-26 1996.[11] S. Bengio and J. Mariéthoz, “Learning the decision functionfor speaker verification,” in IEEE International Conferenceon Acoustic, Speech, and Signal Processing, 2001.

SVMSVM: Support vector machine speaker verification methodology.

Crossref

SVMSVM: support vector machine speaker verification methodology

Support vector machines with the Fisher and score-space kernels are used for text independent speaker verification to provide direct discrimination between complete utterances. This is unlike approaches such as discriminatively trained Gaussian mixture models or other discriminative classifiers that discriminate at the frame-level only. Using the sequence-level discrimination approach we are able to achieve error-rates that are significantly better than the current state-of-the-art on the PolyVar database.Published in

Edinburgh Research Explorer

Speaker identiﬁcation and veriﬁcation using Gaussian mixture speaker models,”

The Nature of Statistical Learning Theory,

The NIST speaker recognition evaluation – overview, methodology, systems, results,

https://era.ed.ac.uk/bitstream/1842/1132/1/icassp03-svm.pdf

SVMSVM: Support vector machine speaker verification methodology.

Abstract

Similar works

Full text

Available Versions

Edinburgh Research Archive

Crossref

Edinburgh Research Explorer