Chng, E. S.

Li, Haizhou

Wu, Zhizheng

Voice conversion techniques present a threat to speaker verification systems. To enhance the security of speaker verification systems, We study how to automatically distinguish natural speech and synthetic/converted speech. Motivated by the research on phase spectrum in speech perception, in this study, we propose to use features derived from phase spectrum to detect converted speech. The features are tested under three different training situations of the converted speech detector: a) only Gaussian mixture model (GMM) based converted speech data are available; b) only unit-selection based converted speech data are available; c) no converted speech data areavailable for training converted speech model. Experiments conducted on the National Institute of Standards and Technology (NIST) 2006 speaker recognition evaluation (SRE) corpus show that the performance of the features derived from phase spectrum outperform the mel-frequency cepstral coefficients (MFCCs) tremendously:even without converted speech for training, the equal error rate (EER) is reduced from 20.20% of MFCCs to2.35%.<br/

Edinburgh Research Explorer

     Edinburgh Research Explorer                                      Detecting Converted Speech and Natural Speech for anti-Spoofing Attack in Speaker RecognitionCitation for published version:Wu, Z, Chng, ES & Li, H 2012, Detecting Converted Speech and Natural Speech for anti-Spoofing Attack inSpeaker Recognition. in 13th Annual Conference of the International Speech Communication Association(Interspeech 2012).Link:Link to publication record in Edinburgh Research ExplorerPublished In:13th Annual Conference of the International Speech Communication Association (Interspeech 2012)General rightsCopyright for the publications made accessible via the Edinburgh Research Explorer is retained by the author(s)and / or other copyright owners and it is a condition of accessing these publications that users recognise andabide by the legal requirements associated with these rights.Take down policyThe University of Edinburgh has made every reasonable effort to ensure that Edinburgh Research Explorercontent complies with UK legislation. If you believe that the public display of this file breaches copyright pleasecontact openaccess@ed.ac.uk providing details, and we will remove access to the work immediately andinvestigate your claim.Download date: 05. Apr. 2019See	discussions,	stats,	and	author	profiles	for	this	publication	at:	https://www.researchgate.net/publication/260343013Detecting	Converted	Speech	and	NaturalSpeech	for	anti-Spoofing	Attack	in	SpeakerRecognitionARTICLE	·	JANUARY	2012CITATIONS25READS2763	AUTHORS:Zhizheng	WuThe	University	of	Edinburgh43	PUBLICATIONS			222	CITATIONS			SEE	PROFILEEng	Siong	ChngNanyang	Technological	University91	PUBLICATIONS			608	CITATIONS			SEE	PROFILEHaizhou	LiAgency	for	Science,	Technology	and	Resear…444	PUBLICATIONS			2,927	CITATIONS			SEE	PROFILEAvailable	from:	Zhizheng	WuRetrieved	on:	11	February	2016Detecting Converted Speech and Natural Speech for anti-Spoofing Attack inSpeaker RecognitionZhizheng Wu1, Eng Siong Chng1, Haizhou Li1,2,31School of Computer Engineering, Nanyang Technological University, Singapore2Human Language Technology Department, Institute for Infocomm Research , Singapore3School of EE & Telecom, University of New South Wales, Australiawuzz@ntu.edu.sg, aseschng@ntu.edu.sg, hli@i2r.a-star.edu.sgAbstractVoice conversion techniques present a threat tospeaker verification systems. To enhance the security ofspeaker verification systems, We study how to automat-ically distinguish natural speech and synthetic/convertedspeech. Motivated by the research on phase spectrum inspeech perception, in this study, we propose to use fea-tures derived from phase spectrum to detect convertedspeech. The features are tested under three different train-ing situations of the converted speech detector: a) onlyGaussian mixture model (GMM) based converted speechdata are available; b) only unit-selection based convertedspeech data are available; c) no converted speech data areavailable for training converted speech model. Experi-ments conducted on the National Institute of Standardsand Technology (NIST) 2006 speaker recognition evalu-ation (SRE) corpus show that the performance of the fea-tures derived from phase spectrum outperform the mel-frequency cepstral coefficients (MFCCs) tremendously:even without converted speech for training, the equal er-ror rate (EER) is reduced from 20.20% of MFCCs to2.35%.Index Terms: Speaker verification, voice conversion,anti-spoofing attack, synthetic speech detection, phasespectrum1. IntroductionSpeaker verification is a process to confirm a claim ofidentity based on the user’s speech samples [1, 2]. Thereare two possible outcomes: the claim is accepted or re-jected. On the other hand, voice conversion is to modifyone speaker’s voice (source speaker) to sound like thatof another speaker (target speaker) [3, 4, 5]. Therefore,voice conversion technique can be used to modify an im-postor’s voice to sound like that of the claimed speaker toattack the speaker verification system.In response to such a potential threat, many studieshave been conducted on the vulnerability of speaker veri-fication systems against synthetic speech. The security orvulnerability of speaker verification systems have beenstudied against imposture using synthetic speech fromHMM-based speech synthesis system [6, 7, 8], adaptedstatistical speech synthesis system [9] and voice conver-sion techniques [10, 11], which are carried out on highquality speech.Spoofing attack researches on telephone speech arealso conducted. In [12], The vulnerability of GMM-based speaker verification system against voice conver-sion attacks is studied on the National Institute of Stan-dards and Technology (NIST) 2006 speaker recognitionevaluation (SRE) corpus. In our previous works [13], theperformance of five speaker verification systems includ-ing the state-of-the-art joint factor analysis (JFA) systemagainst spoofing attacks using GMM-based voice conver-sion techniques are evaluated on NIST 2006 SRE corpus.Then the vulnerability of the JFA system against unit-selection based and GMM-based converted speech is alsocompared on NIST 2006 SRE corpus [14].All the previous studies on both high quality speechand telephone speech confirmed that synthetic speechfrom statistical speech synthesis systems or voice con-version techniques present a threat to speaker verificationsystems.In [15], a technique was studied that makes use ofthe differences in the relative phase shift [16] betweenhigh quality human and synthetic speech signals to de-tect synthetic speech from a HMM-based speech synthe-sis system. In this study, it was assumed that the syntheticspeech data are available for training a detector. How-ever, in real applications, we don’t have the informationas to what kinds of speech synthesis techniques or voiceconversion techniques are used in the spoofing attacks.Hence, no synthetic speech data are actually available.In this study, we focus on detecting spoofing attacks us-ing voice conversion techniques on telephone speech, andpropose to use features derived from the phase spectrum,which has been shown to be useful in speech recognition[17, 18], to detect converted speech without the need toknow the exact voice conversion techniques.2. Phase information in voice conversionAlthough converted speech has been shown to be able toconfuse a speaker verification system in many reportedwork [10, 11, 13, 14], informal listening tests show thathuman ear can easily distinguish natural speech and con-verted speech. It has been shown that phase spectrum isuseful for speech perception [19, 20] , and the fact that theoriginal/natural phase information is missing in the con-verted speech, we would like to look into how convertedspeech can be detected by using phase features.     	   	  		 F0, Spectral parameter, et al.F0, Spectral parameter, et al.Speech signalSpeech signalFigure 1: Typical voice conversion procedure.In Figure 1, we show a typical voice transformationprocedure. A speech signal is first analyzed to extractparameters, such as fundamental frequency, spectral en-velop parameters, which are then converted by an alreadytrained transformation function. Finally the convertedparameters are passed to synthesis filter to reconstructwaveform. In the parameter extraction step, we usuallyextract fundamental frequency and speech envelop pa-rameters, ignoring the phase information. As a result,the original/natural phase information is not kept in thereconstructed waveform.The best way to build a converted speech detector isto train a converted speech model that characterizes thesignal. If converted speech data are available, convertedspeech models can be trained right away. One notes thatthe difference between voice conversion techniques liesin the transformation module in Figure 1. In the casewhen converted speech data are not available, if we canextract the parameters from natural speech and pass di-rectly to the synthesis filter, it is possible that the resultingsynthesized speech carry the characteristics of convertedspeech. We will look into how such synthesized speechcan be used to train the converted speech model.3. Phase information extraction formodelingAlthough speech signal is non-stationary, it can be as-sumed as quasi-stationary. Therefore, speech signal canbe processed by short-time Fourier analysis. Given aspeech signal x(n), the short-time Fourier transform(STFT) is given as follows:X(ω) =| X(ω) | ejψ(ω) (1)where | X(ω) | is the short-time magnitude spectrum,and ψ(ω) is the short-time phase spectrum. The squareof magnitude spectrum, | X(ω) |2, is usually called thepower spectrum, from which the mel-frequency cepstralcoefficients (MFCCs) are derived.As natural phase information is missing in the recon-structed speech/converted speech, phase spectrum can beused to derive features providing the evidence of con-verted speech. In this study, two different features willbe derived from phase spectrum as follows.3.1. Cosine normalization of phase spectrumAs the original phase spectrum is not continuous in fre-quency domain, we first unwrap the phase spectrum as acontinuous function of frequency. After unwrapping, therange of spectrum could vary which makes it difficult tomodel phase information. Cosine function is then appliedon the unwrapped phase spectrum to normalize the rangeinto [−1.0, 1.0]. Then discrete cosine transform (DCT)is applied on the Cosine normalized phase spectrum toreduce dimensionality. Finally we keep 12 cepstral co-efficients, excluding the 0th coefficient. This feature iscalled cos-phase in this study.3.2. Frequency derivative of phase spectrumThe second feature is the frequency derivative of phasespectrum, obtained by the group delay function (GDF)which is a measure of the nonlinearity of the phase spec-trum [17] and defined as the negative derivative of thephase spectrum with respect to ω:τ(ω) =XR(ω)YR(ω) + YI(ω)XI(ω)| X(ω) |2 (2)where Y (ω) is the STFT of nx(n); XR(ω), XI(ω) andYR(ω), YI(ω) are the real part and imaginary part ofX(ω) and Y (ω), respectively.To capture the fine structure of group delay phasespectrum, in practice, a modified group delay functionis adopted. Smoothed power spectrum is used instead oforiginal power spectrum and two variables are introducedto emphasis the fine structure of the phase spectrum. Themodified group delay function is described as follows.τγ(ω) =XR(ω)YR(ω) + YI(ω)XI(ω)| S(ω) |2γ (3)τα,γ(ω) =τγ(ω)| τγ(ω) | | τγ(ω) |α (4)where | S(ω) |2 is the smoothed power spectrum,τα,γ(ω) is the modified group delay phase spectrum, αand γ are two variables to make the phase spectrum bepresented in favorable form, α and γ are set to 0.4 and1.2 in this study, respectively.After modified group delay phase spectrum is ob-tained, discrete cosine transform (DCT) is applied. Wekeep 12 cepstral coefficient, excluding the 0th coefficient.The 12 dimension feature are used for model training anddetection and this feature is called MGDF-phase in thisstudy.4. Experimental setups and resultsIn this study, the performance of cos-phase and MGDF-phase features (without delta and delta-delta coefficients)are evaluated for converted speech detection under threedifferent training situations: a) only GMM-based con-verted speech data are available; b) only unit-selectionbased converted speech data are available; c) no con-verted speech data are available. We use 12 dimen-sion mel-frequency cepstral coefficients (MFCCs) andtheir delta and delta-delta coefficients, which are used inspeaker verification [13, 14], as the reference baseline.The natural/converted speech decision is made usinglog-scale likelihood ratio as follows:Λ(C) = log p(C|λconverted)− log p(C|λnatural) (5)where C is the feature vector sequence of a speechsignal, λconverted is the GMM model for convertedspeech, and λnatural is the GMM model for naturalspeech. Under the three different situations, we havethe same natural speech model λnatural, but three dif-ferent converted speech model λconverted . The numberof Gaussian components of GMM is set to 512. Equalerror rate (EER) is reported as the evaluation criterion.4.1. CorporaA subset of the NIST 2006 SRE corpus, and a subset ofthe spoofing attack corpora are adopted. The spoofingattack corpora are created using two different voice con-version methods (GMM-based voice conversion and unit-selection based voice conversion method) in our previousstudies [13, 14]. In constructing the converted speech cor-pora, mel-cepstral analysis [21] and MLSA filter [21] areadopted for feature extraction and waveform reconstruc-tion, respectively. In practice, the Speech Signal Process-ing Toolkit (SPTK) tool [22] is used.We use 100 sessions of natural speech to train thenatural speech model λnatural. The testing data con-sist of 1,500 sessions of natural speech (original wave-form from NIST 2006 SRE corpus), 1,000 sessions ofGMM-based converted speech and 1,000 sessions of unit-selection based converted speech. The duration of eachsessions of natural speech or converted speech is roughly5 minutes.4.2. GMM-based converted speech are availableAssume that we have 100 sessions of GMM-based con-verted speech for training of λconverted. The perfor-mance of the three different features are presented in Ta-ble 1. With cos-phase and MGDF-phase features, theEER is reduced from 16.8% of MFCCs to 6.60% and9.13%, respectively.Table 1: Equal error rate (EER, %) of detection perfor-mance when GMM-based converted data is available fortraining.Feature Equal Error Rate (%)MFCCs 16.80cos-phase 6.60MGDF-phase 9.134.3. Unit-selection based converted speech are avail-ableAssume that we have 100 sessions unit-selection basedconverted speech to estimate the parameters of the con-verted speech model λconverted. The detection results arepresented in Table 2. We note that the cos-phase andMGDF-phase features reduce the EER from 15.35% ofMFCCs to 3.93% and 4.60%, respectively.Table 2: Equal error rate (EER, %) of detection perfor-mance when unit-selection based converted data is avail-able for training.Feature Equal Error Rate (%)MFCCs 15.35cos-phase 3.93MGDF-phase 4.604.4. No converted speech data are availableIn this case, we assume that neither GMM-based nor unit-selection based converted speech is available for train-ing, but the speech analysis module and synthesis filterare available. Therefore, we extract parameters from the100 sessions of natural speech for training natural speechmodel λnatural, and then pass the parameters directlyto the synthesis filter to reconstruct the waveforms. Wethen use the 100 resulting sessions to train the convertedspeech model λconverted. The detection error tradeoff(DET) curves are presented in Figure 2. We can seethat even without converted speech for training the de-tector, the performance of the features derived from phasespectrum outperform the MFCCs tremendously. The cos-phase and MGDF-phase features reduce the EER from20.20% of MFCCs to 5.95% and 2.35%, respectively.  0.1  0.2  0.5    1     2     5     10    20    40    0.1   0.2  0.5    1     2     5     10    20    40  False Alarm probability (in %)Miss probability (in %)  MFCCs, EER = 20.20cos−phase, EER = 5.95MGDF−phase, EER = 2.35Figure 2: DET curve of detection performance when con-verted data is NOT available for training.5. ConclusionsIn this study, the features, cos-phase and MGDF-phase,derived from phase spectrum are adopted to detect con-verted speech for spoofing attack to enhance the securityof speaker verification system. Under three different situ-ations, the proposed features outperform the MFCCs con-sistently, especially when converted speech is not avail-able for training. To detect converted speech in theparadigm of Figure 1, we have shown that our analysis-synthesis method offers an effective alternative solutionthat simulates converted data when actual voice transfor-mation techniques are unavailable.6. References[1] J.P. Campbell Jr, “Speaker recognition: A tutorial,” Pro-ceedings of the IEEE, vol. 85, no. 9, pp. 1437–1462, 1997.[2] D.A. Reynolds, “An overview of automatic speaker recog-nition technology,” in ICASSP 2002.[3] Y. Stylianou, O. Cappe´, and E. Moulines, “Continu-ous probabilistic transform for voice conversion,” Audio,Speech, and Language Processing, IEEE Transactions on,vol. 6, no. 2, pp. 131–142, March 1998.[4] A. Kain and M.W. Macon, “Spectral voice conversion fortext-to-speech synthesis,” in ICASSP 1998.[5] Y. Stylianou, “Voice transformation: a survey,” in ICASSP2009.[6] T. Masuko, T. Hitotsumatsu, K. Tokuda, andT. Kobayashi, “On the security of HMM-based speakerverification systems against imposture using syntheticspeech,” in EUROSPEECH 1999.[7] T. Masuko, K. Tokuda, and T. Kobayashi, “Imposture us-ing synthetic speech against speaker verification based onspectrum and pitch,” in ICSLP 2000.[8] T. Satoh, T. Masuko, T. Kobayashi, and K. Tokuda, “A ro-bust speaker verification system against imposture using aHMM-based speech synthesis system,” in EUROSPEECH2001.[9] P. DeLeon, M. Pucher, and J. Yamagishi, “Evaluationof the vulnerability of speaker verification to syntheticspeech,” in Odyssey 2010.[10] Qin Jin, Arthur Toth, Alan W. Black, and Tanja Schultz,“Is voice transformation a threat to speaker identifica-tion?,” in ICASSP 2008.[11] Q. Jin, A.R. Toth, T. Schultz, and A.W. Black, “Voiceconvergin: Speaker de-identification by voice transforma-tion,” in ICASSP 2007.[12] J.F. Bonastre, D. Matrouf, and C. Fredouille, “Artificialimpostor voice transformation effects on false acceptancerates,” in Interspeech, 2007.[13] T. Kinnunen, Z.-Z. Wu, K. A. Lee, F. Sedlak, E. S. Chng,and H. Li, “Vulnerability of Speaker Verification SystemsAgainst Voice Conversion Spoofing Attacks: the Case ofTelephone Speech,” in ICASSP 2012.[14] Z. Wu, E. S. Chng, and H. Li, “Speaker verifica-tion system against two different voice conversion tech-niques in spoofing attacks,” Technical report: available athttp://www3.ntu.edu.sg/home/wuzz/.[15] P.L. De Leon, I. Hernaez, I. Saratxaga, M. Pucher, andJ. Yamagishi, “Detection of synthetic speech for the prob-lem of imposture,” in ICASSP 2011.[16] I. Saratxaga, I. Hernaez, D. Erro, E. Navas, andJ. Sanchez, “Simple representation of signal phase forharmonic speech models,” Electronics letters, vol. 45, no.7, pp. 381–383, 2009.[17] L.D. Alsteris and K.K. Paliwal, “Short-time phase spec-trum in speech processing: A review and some experimen-tal results,” Digital Signal Processing, vol. 17, no. 3, pp.578–616, 2007.[18] R.M. Hegde, H.A. Murthy, and V.R.R. Gadde, “Signifi-cance of the modified group delay feature in speech recog-nition,” Audio, Speech, and Language Processing, IEEETransactions on, vol. 15, no. 1, pp. 190–202, 2007.[19] K.K. Paliwal and L.D. Alsteris, “On the usefulness of stftphase spectrum in human listening tests,” Speech Com-munication, vol. 45, no. 2, pp. 153–170, 2005.[20] L.D. Alsteris and K.K. Paliwal, “Further intelligibility re-sults from human listening tests using the short-time phasespectrum,” Speech Communication, vol. 48, no. 6, pp.727–736, 2006.[21] T. Fukada, K. Tokuda, T. Kobayashi, and S. Imai, “Anadaptive algorithm for mel-cepstral analysis of speech,”in ICASSP 1992.[22] “Speech Signal Processing Toolkit (SPTK) version 3.4,”Software available at http://sp-tk.sourceforge.net/.

Detecting Converted Speech and Natural Speech for anti-Spoofing Attack in Speaker Recognition

https://www.pure.ed.ac.uk/ws/files/24456414/Wu_Chng_et_al_2012.pdf

Detecting Converted Speech and Natural Speech for anti-Spoofing Attack in Speaker Recognition

Abstract

Similar works

Full text

Available Versions

Edinburgh Research Explorer