Realistic Voice Gender Conversion (VGC) requires

independent scaling of the glottal (pitch) and vocal tract

(formant) related features of the input speech signal. We

present a VGC algorithm which has two novel features.

Firstly, an efficient frequency scaling algorithm is presented.

Secondly, we use this to scale all frequencies in the input

signal by the desired formant scaling factor. We then

deconvolve the glottal contribution using a standard linear

predictive analysis and frequency scale it further such that the

desired pitch scaling factor is equal to the product of the two

frequency scaling factors. Finally, we resynthesize the

converted speech. The female-to-male results were excellent

while the male-to-female results sounded synthetic

Lawlor, Bob

English

MURAL - Maynooth University Research Archive Library

A NOVEL EFFICIENT ALGORITHM FOR VOICE GENDER CONVERSIONBob Lawlor and Anthony D. FaganUniversity College Dublin, IrelandABSTRACTRealistic Voice Gender Conversion (VGC) requiresindependent scaling of the glottal (pitch) and vocal tract(formant) related features of the input speech signal. Wepresent a VGC algorithm which has two novel features.Firstly, an efficient frequency scaling algorithm is presented.Secondly, we use this to scale all frequencies in the inputsignal by the desired formant scaling factor. We thendeconvolve the glottal contribution using a standard linearpredictive analysis and frequency scale it further such that thedesired pitch scaling factor is equal to the product of the twofrequency scaling factors. Finally, we resynthesize theconverted speech. The female-to-male results were excellentwhile the male-to-female results sounded synthetic.1. INTRODUCTIONVoice Gender Conversion (VGC) consists of modifying female(male) speech such that it sounds male (female). Applicationsinclude voice gender normalisation for improved speechcompression or recognition and voice disguise. In practice,such applications require that the VGC algorithm runs inrealtime on inexpensive hardware. The human auditory systemis highly sensitive to voice perception and synthetic speech,though intelligible, often sounds unnatural. The VGCchallenge is to convert the gender related parameters of thespeech signal without affecting naturalness. For a long time itwas felt that pitch was the dominant cue in voice genderperception. However, Childers [1] showed that groupedformant information gave a higher automatic genderdistinction success rate than pitch information. Hence,realistic VGC requires independent modification of the glottal(pitch) and vocal tract (formant) related features of the sourcespeech signal. Atal [2] presented a VGC algorithm usinglinear predictive analysis to deconvolve the glottal and vocaltract contributions. He applied independent scaling factors tothe pitch frequency and the formant frequencies andbandwidths prior to resynthesizing to give gender convertedspeech. The  scaling factors were based on typical male-to-female vocal cord membrane and vocal tract length ratios. Ourapproach is similar to Atal’s, but has two novel features.Firstly, we present an efficient frequency scaling algorithmbased in the principle of time-domain overlap-add (TD-OLA).Secondly, we use this to scale all frequencies in the inputsignal by the desired formant scaling factor. We thendeconvolve the glottal contribution using Atal’s linearpredictive analysis and frequency scale it further such that thedesired pitch scaling factor is equal to the product of the twofrequency scaling factors. Finally, we resynthesize theconverted speech.2. VOICE GENDER DIFFERENCESA detailed explanation of the speech production mechanismcan be found in [3]. A typical male vocal tract is about 17.5cm in length (i.e. from vocal cords to lips) while that of afemale is about 15.2 cm. The adult male larynx is about 1.2times the size of that of the female. During puberty the malelarynx undergoes a change in shape (Adam’s apple protrusion)such that the adult male vocal cord membrane length reachesabout 1.6 times that of the female. Analysis of male andfemale voiced utterances shows that the female formantfrequencies are about 15% higher than male. This differenceis in close agreement with the male-to-female vocal tractlength ratio. Female pitch is generally about 1.7 times that ofmale. This difference is attributed mainly to the difference invocal cord membrane length although other factors such asmale/female differences in the way in which the cords openand close are believed to be relevant also [4]. The air pressurevariation produced in the region of the vocal cords is known asthe Glottal Volume Velocity Waveform (GVVW). Duringvoicing this has the form of that shown in Figure 1 [5].Figure 1. Typical male (M) and female (F) GVVWs.Monsen [5] suggests, and cites supporting references, that thehump (indicated H) in the opening phase of the male GVVWmay be due to a slightly out of phase movement of the upperand lower parts of each vocal cord, whereas the shorter femalecords come into contact with each other more as a single mass.Four parameters which are often used to characterise theGVVW are:1. The AC flow which is equal to the peak-to-peak value ofthe GVVW as shown in Figure 1.2. The Maximum Flow Declination Rate (MFDR) is definedas the minimum of d/dt(GVVW) and is generally equal to theslope of the falling edge of the glottal pulses.page 77 ICPhS99          San Franciscopage 77 ICPhS99          San Francisco3. The pitch period, T, which is the time between sucessivepeaks of the GVVW. The pitch frequency, F0 is equal to 1/T.4. The average flow (Avg. Flow) which is equal to the mean ofthe GVVW.Sulter [6] carried out extensive tests on GVVW characteristicsacross gender for both vocally trained and untrained subjectsunder soft, normal and loud voicing conditions. Table 1summarises the parameters of the GVVW which showedmajor differences across gender for the untrained subjectsunder normal voicing conditions only.Parameter Female Male Ratio (F/M)AC flow (l/s) 0.26 0.57 0.46MFDR (l/s2) 504 1026 0.491/T (Hz) 253 145 1.7Avg flow (l/s) 0.16 0.23 0.7Table 1. GVVW parameters most affected by gender2.1. Voice gender perceptionFor a long time it was believed that pitch was the dominantgender cue. However, recent studies of the speech productionmechanism have revealed more subtle differences betweenfeatures of the male and female speech waveforms. One suchstudy [1] showed that grouped formant information canprovide a slightly higher automatic gender distinction successrate (98.1%) than pitch information (96.2%). These figuressuggest that both pitch and formant information are importantcues in voice gender distinction.3. LINEAR PREDICTIVE ANALYSIS/SYNTHESISDudley [7] presented the channel vocoder (VOice CODER)for speech compression. This is based on modelling the vocaltract response with a time-varying filter. He used a 10 bandfilterbank at the encoder to sample the short time magnitudespectrum. At the decoder, these magnitude estimates wereused to control a bank of oscillators corresponding to thefilterbank centre frequencies. Flanagan [8] presented amodification to the channel vocoder called the phase vocoder.This included samples of the phase spectrum (as well assamples of the speech signal short time magnitude spectrum)which contains time-varying pitch (and its harmonics)information. These phase samples were used at the decoder tocontrol the phase of the oscillators. Flanagan found that usingthe phase vocoder in a back-to-back analysis/synthesis systemgave output ‘quality much better than the conventionalchannel vocoder’.Atal [2] presented the linear predictive approach to speechcoding based on the same speech production model as thatused by Dudley and Flanagan, i.e. a source excitation signaldriving a time-varying filter. However, Atal applied a timedomain least-mean square analysis algorithm directly to thespeech signal to estimate the parameters of the time-varyingvocal tract model filter. By passing the input speech signalthrough the inverse of the vocal tract model filter he produceda residual signal which was a good approximation to theglottal source excitation signal. By passing this residual signalback through the same vocal tract model filter, the originalspeech is perfectly reconstructed. This analysis/synthesisapproach forms the basis of many modern speech compressionalgorithms e.g. Code Excited Linear Prediction (CELP),Multi-Pulse Excited Linear Predictive Coding (MP-LPC) [9].The spectral peaks corresponding to the formant frequenciesare perceptually most important. As such peaks can beaccurately modelled using poles, Atal chose an all-pole vocaltract model filter. The all-pole model also simplified the filterparameter estimation. Atal’s main application of interest wasefficient storage and transmission of speech. However, he alsodetails a number of other applications of his linear predictiveanalysis/synthesis algorithm such as formant analysis andseparating the spectral envelope and fine structure. He notesthat ‘the synthesis procedure allows independent control ofsuch speech characteristics as spectral envelope, relativeduration’s, pitch and intensity’. One such application whichhe presents under the heading of ‘re-forming the speechsignal’, involves simulating a female voice from parametersderived from a male voice. For this application he used a 10-thorder model filter (predictor).  He scaled the pitch period by afactor of 0.58 and the formant frequencies by 1.14. He alsoscaled the formant bandwidths by a frequency-dependentfactor given by 2 – Fi/5000 where Fi is the formant centrefrequency in Hz. The 5000 in this factor is equal to half hisspeech signal sampling frequency.4. TIME & FREQUENCY SCALINGIf a signal is sampled using sample frequency, fs samples persecond, and then played back using a different conversionfrequency, fp, the duration of the signal will be scaled by thefactor, fs/fp and the frequency content of the signal will bescaled by the factor fp/fs. Time-Scale Modification (TSM) of aspeech (or other audio) signal consists of modifying itsduration without affecting its perceived frequency content.Similarly, Frequency-Scale Modification (FSM) consists ofmodifying its frequency content without affecting its duration.In 1985 Roucos [10] presented the Synchronised Overlap-and-Add (SOLA) algorithm for speech TSM. With thisapproach, overlapping segments (or frames) of the inputsignal are first extracted, a frame being typically several pitchperiods in duration. By decreasing the overlap betweensuccessive frames, time-scale expansion is realised.Similarly, by increasing the overlap, time-scale compressionis realised. In the original SOLA algorithm [10], the segmentalignment was optimised by computing a normalised cross-correlation measure, R, for a range of possible alignmentoffsets and then choosing the offset for which R is amaximum, indicating maximal similarity betweenoverlapping segments. High quality combined with moderatecomputational load has made the SOLA algorithm the choicefor many speech and audio TSM systems. We present analternative TSM algorithm which offers a significantreduction in computational load without loss of quality. If asignal is time-scaled by some factor, TS, and then played at TStimes its original sample rate, the net effect is to scale thefrequency content by the factor, TS, without affecting thepage 78 ICPhS99          San Franciscopage 78 ICPhS99          San Franciscooriginal duration, i.e. FSM. We use this approach tofrequency scale both the glottal and vocal tract relatedcomponents of speech independently to realise voice genderconversion. First we describe our efficient TSM algorithmwhich we call Adaptive Overlap-Add (AOLA).4.1 Adaptive Overlap-Add (AOLA)Figure 2. Adaptive Overlap-Add AlgorithmReferring to Figure. 2, the solid trace of plot (a) represents arectangular windowed segment of the input signal. Thewindow length, Z, is chosen such that it will accommodate atleast two pitch periods of typical male voiced speech. Forunvoiced speech, the choice is not critical and Z can be leftequal to the value chosen to satisfy the above voicedcondition. Assuming we wish to scale the duration of thissegment by some desired expansion factor, de, the stepsinvolved in the algorithm are as follows:1.  The windowed input segment (a) is duplicated and theduplicate aligned with (a) as shown in plot (b). Thealignment criterion is based on aligning the two largestpeaks or troughs.2.  A synthetic segment, (c), is produced by fading graduallyfrom (a) to (b) in the overlapping region. The naturalexpansion factor, ne, is given by the ratio of the lengths of(c) and (a) as indicated.3.  The rectangular window is stepped forward in time by st=  |CD| = Z.(1-ne)/(1-de) and the new step-size segmentof the original concatenated with (c) as indicated dottedsuch that the next segment to be expanded is the length wportion of (c) above BD, see plot (e). Repeat from step 1until the end of the signal being scaled is reached.Rationale: Plot (d) represents the desired length to which wewish to time-scale (a), i.e. Z.de. The segment of (c) aboveAB has been time-scaled by the desired factor, de, and isoutput from the time-scaling window. For each step of thewindow we repeat the peak search and update ne. Assumingne to be approximately equal to its last value, segment BD of(c) expands in the same way as (a) to length |BF| ≈ Z.ne. Stepsize portion CD of (c) expands to length |EF| ≈ |CD|.ne, butwe require it to be expanded by factor de. To achieve this wemust apply our natural expansion factor A times such that neA= de. If segment CD is to have A applications of naturalexpansion factor, ne, before leaving the expansion window,then from plot (f), our step size, st, must satisfy the followingequation    .nest.nest.nest.neAZ≈+++ 2         (1)   dene.nene.st A−−=−−≈⇒1111ZZ     (2)As ne is continuously varying, (1) (and (2)) is anapproximation. In fact, each of the ne and st terms in (1) areslightly different. By updating ne and hence st for every stepof the window, the algorithm accurately adapts to the localsignal characteristics.For time-scale compression the approach is similar. In thiscase the peaks or troughs are aligned as before but thesections of (c) to the left and right of the central overlappingsection are discarded leaving a naturally compressedsegment. If the input segment has a natural compressionfactor, nc, and the desired compression factor is dc, thenequation (2) becomesdcncst−−=11.Z (3)5. VOICE GENDER CONVERSIONFor female-to-male VCG we require to scale the formantfrequencies by some factor, SF, (~0.87 i.e. down 15%) and thepitch frequency by some factor, SP, (~0.6). The steps involvedare as follows:1. Time-scale modify the input signal by the factor SF usingthe AOLA algorithm. This shortens its duration withoutaffecting its frequency content. If it is played back now atSF times its original sample rate, the duration will berestored to its original but all frequencies (pitch andformants) lowered by the factor SF.2. Apply linear predictive inverse filtering to separate theformant information form the spectral fine structure(pitch).3. Time-scale modify the inverse filter output signal fromstep 2 (residual) by the factor SP/SF using the AOLAalgorithm.4. Using the vocal tract model filter coefficients of step 2 andthe time-scale modified residual of step 3, resynthesize thespeech signal and play it back at SP times its originalsample rate.For male-to-female VGC the procedure is identical, but thescaling factors are equal to the reciprocals of those for female-to-male.www .new .deststs t .nest .ne 2st .ne AA B C DE F(a)(b)(c)(d)(e) (f)page 79 ICPhS99          San Franciscopage 79 ICPhS99          San Francisco6. RESULTSWe applied the algorithm to a selection of signals from theDARPA TIMIT speech corpus [11]. We present here sampleresults. The full path names of these speech signals within thecorpus are: \TIMIT\TEST\DR1\FELC0\SA1.WAV: for the original femalespeech sample [SOUND 0027_01.WAV] and \TIMIT\TEST\DR1\MDAB0\SA1.WAV: for the original male[SOUND 0027_02.WAV].We tried various combinations of pitch and formant scalingfactors and filter orders. For female-to-male the pitch scalingfactors, SP were in the range 0.55 to .8 and the formant scalingfactors, SF in the range 0.82 to .92. The filter orders tried were6, 8, 10 and 12. A filter order of 8 gave better results than 6,while 10 and 12 gave no improvement over 8 so we used 8.We found that SP = 0.74 and SF = 0.87 gave the best female-to-male result [SOUND 0027_03.WAV]. For male-to-female[SOUND 0027_04.WAV] we used the same filter order andthe reciprocal scaling factors. We also generated back-to-backresults, i.e female-to-male-to-female [SOUND 0027_05.WAV]and male-to-female-to-male [SOUND 0027_06.WAV]. Theseresults are also available from the author by email(rlawlor@faraday.ucd.ie).7. DISCUSSIONThe female-to-male results were excellent, the outputsounding completely natural. The male-to-female results werenot as good with many of the output samples soundingunnatural. These findings are consistent with those of otherVGC researchers [12] [13]. Our scaling factors were very closeto those used in [12]. Childers [12] suggests that the reasonwhy male-to-female VGC is less successful when a singleformant scaling factor is used is that the factor should itself befrequency dependent (‘higher frequency formants should beshifted less than lower frequency formants’). As the singlecompromise factor is greater than unity for male-to-female andless than unity for female-to-male, the inadequacies of thecompromise will be more noticeable on male-to-female. Chan[13] cites [12] but suggests that the reason male-to-femaleVCG is more difficult is ‘unknown at present’. For theimproved speech recognition application and for most voicegender disguise applications female-to-male VGC alone isadequate. The improved speech compression applicationrequires good back-to-back performance. The back-to-backoutputs sounded like slightly noisy versions of their originalsbut suggest that the algorithm can be used to normalise voicegender to give greater compression of telephone qualityspeech.8. CONCLUSION & FURTHER WORKWe have presented an efficient VGC algorithm which giveshigh quality female-to-male VGC but poor quality male-to-female VGC. We feel that further improvement may bepossible with a suitable non-linear transformation of theGVVW. This would, however, increase the computationalburden. The algorithm presented can be implemented in real-time on a low cost digital signal processor.REFERENCES[1] Childers, D. G. and Wu, K. 1991. Gender recognition from speech.Part II: Fine analysis. Journal of the Acoustical Society of America, Vol.90, pp. 1841 – 1856.[2] Atal, B. S. and Hanauer, S. L. 1971. Speech analysis and synthesis bylinear prediction of the speech wave. Journal of the Acoustical Society ofAmerica, Vol. 50 No. 2,  pp. 637 – 655.[3] O’Shaughnessy, D. 1987. Speech Communication : Human andMachine. Addison – Wesley.[4] Titze, I. R. 1989. Physiologic and acoustic differences between maleand female voices. Journal of the Acoustical Society of America, Vol.85, No. 4, pp. 1699 - 1707.[5] Monsen R. B. and Engebretson, A. M. 1977. Study of variations in themale and female glottal wave. Journal of the Acoustical Society ofAmerica, Vol. 62, No. 4, pp. 981 - 993.[6] Sulter, A. M. and Wit, H. P. 1996. Glottal volume velocity waveformcharacteristics in subjects with and without training, related to gender,sound intensity, fundamental frequency and age. Journal of theAcoustical Society of America, Vol. 100, No. 5, pp. 3360 - 3373.[7] Dudley, H. 1939. Remaking speech. Journal of the Acoustic Societyof America, Vol. 11, No.2, pp. 169 - 175.[8] Flanagan, J.L. and Golden, R.M. 1966. Phase Vocoder. Bell systemtechnical Journal.[9] Deller, J. R. JR., Proakis, J. G. and Hansen, J. H. L. 1993. Discrete-Time Processing of Speech Signals. Macmillan Publishing Company.[10] Roucos, S. and Wilgus, A. M. 1985. High-quality time-scalemodification for speech. IEEE proceedings on acoustics, speech andsignal processing.[11] DARPA TIMIT Acoustic-Phonetic Continuous Speech Corpus,http://www.ntis.gov/fcpc/cpn4129.htm[12] Childers, D. G., Wu, K., Hicks, D.M. and Yegnanarayana, B. 1989.Voice conversion. Speech Communication 8, pp. 147-158.[13] Chan, P. A. and Damper, R. I. 1994. Voice conversion by whole-spectrum scaling. ESCA Workshop on automatic speaker recognition,identification and verification, pp. 165 - 168.page 80 ICPhS99          San Franciscopage 80 ICPhS99          San Francisco

A Novel Efficient Algorithm for Voice Gender Conversion

http://mural.maynoothuniversity.ie/9766/1/BL-Novel-1999.pdf

A Novel Efficient Algorithm for Voice Gender Conversion

Abstract

Similar works

Full text

Available Versions

MURAL - Maynooth University Research Archive Library