Folk, Laura

Schiel, Florian

English

Universität München: Elektronischen Publikationen

The Lombard Effect in Spontaneous Dialog Speech

Open Access LMU

The Lombard Effect in Spontaneous Dialog SpeechLaura Folk, Florian SchielBavarian Archive for Speech Signals, Institute for Phonetics and Speech Processing,Ludwig-Maximilians-Universita¨t, Mu¨nchen, Germanylaure|schiel@phonetik.uni-muenchen.deAbstractThe Lombard effect – environmental noise affects speech pro-duction – has already been studied extensively for read labspeech. In this study spontaneous dialog speech produced by 24German speakers has been recorded under noisy conditions andanalysed for the Lombard effect. A sophisticated experimentalsetup using behind-the-ear hearing aid equipment allows us toinsert real car noise into the perceived audio stream of speakerswhile maintaining the normal auditory feedback loop. We foundthat the main Lombard effects – rising fundamental frequencyand intensity – can be confirmed for dialog speech. Speakingrate did not slow down although reported earlier for read speech.We also found that certain rhythmicity features regarding thedynamic of the RMS energy contour change significantly underLombard conditions but only for the female speakers.Index Terms: Lombard effect, dialog speech, spontaneousspeech, fundamental frequency, energy, rhythmicity features1. IntroductionIn many real-life situations speakers have to communicate un-der impaired conditions, i.e. noisy environments of any kind. Insuch situations, most speakers raise their pitch, increase the in-tensity of their speech signal and possibly reduce their speak-ing rate. This reflexive reaction became well known as theLombard effect, first described by E´tienne Lombard in 1911([1]). Since then many investigations have confirmed his ob-servations; most have analysed laboratory speech masked withwhite noise for the interfering conditions (e.g. [2], [3], [4], [5]).All studies have shown that speakers gradually rise their funda-mental frequency when the signal-to-noise-ratio decreases, andat the same time increase the intensity of their speech signal1.The latter roughly follows a linear function with a slope of about0.5 (in decibel units), which means that speakers compensate anincrease of noise intensity by increasing their own intensity byhalf the perceived increase ([2]).Furthermore, some studies found that speaking rate is de-celerated under noisy conditions which is an obvious strategyfor remaining intelligible (e.g. [3]).These findings are consistent with Lindblom’s H-H theory([6]) which postulates a continuum between “clear”, i.e. hyper-articulated and relatively slow speech versus “reduced”, i.e.hypo-articulated speech which is relatively fast, but economic.Lindblom’s theory is founded on the notion that motor controlof speech production is a trade-off between maximal commu-nication vs. minimal effort and is therefore extremely variableaccording to different contexts and situations. The Lombardeffect is therefore simply a reflex to shift the motor control ofspeech production in the direction of hyper speech to ensure an1The rise in pitch is most likely a result of the rise in intensity.undisturbed communication. This hypothesis is supported bythe fact that under noisy conditions listeners perceive speechwhich itself was produced under noisy conditions better thanspeech which was produced in quiet ambience ([7]).Quantitative knowledge about the influence of the Lombardeffect on F0 plays an important role in forensic investigations,since fundamental frequency is a key feature for speaker recog-nition.Existing research has been based on read speech and unnat-ural white noise. To date there exist only very few investiga-tions into pseudo communication situations or real noise (e.g.“multitalker babble” ([5]) and communication inside a jet cock-pit ([8])). As already mentioned in an earlier study by Junqua etal., “studies of the Lombard reflex where data has been recordedwhile subjects are reading a list do not accurately represent thereal conditions.” ([9]).One of the most frequently experienced noisy situations inthe modern world is within the automobile. This presents a nat-ural setting for verbal communication - be it with other pas-sangers or by means of hands-free phone sets or even when talk-ing to the car’s dialog system. The goal of the present study is totest whether the Lombard reflex can be observed in an automo-tive environment and in dialog speech in the same way as in thelaboratory with read speech. More specifically our hypothesesare:1. The average and dynamic range of fundamental fre-quency (f0) increases with rising noise level.2. The average and dynamic range of intensity increaseswith rising noise level.3. Speaking rate decreases with rising noise level.Additionally - and to our knowledge for the first time - we wantto test for gender specific differences in the Lombard effect. Fi-nally, all analyses should be carried out without manual annota-tion or segmentation to allow for automated Lombard detectionbased on our findings.The remaining paper is structured as follows: The next sec-tion describes the recording of the speech material for the ex-periment. Section 3 describes the extracted features to test theabove hypotheses as well as the statistical framework of our ex-periment. Section 4 presents the results while the last sectioncontains discussion and future work.2. Recorded Speech DataWe used parts of the German speech corpus HOESI which wasrecorded in 2008 by BAS Services, Munich, in cooperation withSIEMENS company, Munich. HOESI contains multi-channeldialog recordings of the same communication partners in sev-eral automotive and lab conditions.2The part relevant for this study consists of 24 speakers intotal (12 women and 12 men), aged between 46 and 74 years.Speakers were recruited quite strictly: they had to be over 45years old, were not dialect speakers and did not have any hear-ing or speaking impairments. The speakers were assigned into12 same-gender pairs. Each pair produced four dialogs lastingabout 7min under varying noise conditions (L0-L3), where L0represented the quiet (neutral) condition and L1-L3 the noisyconditions with increasing noise-level. The noise was recordedfrom inside a standard car3 moving at different velocities. Threeconstant speed levels were selected for each of the perturbingconditions: L1 = 80km/h, L2 = 120km/h and L3 = 160km/h. Foreach recording the speakers were placed in front of each otherin a soundproof room and discussed a given topic in a free dia-log; topics varied through Lombard conditions and ranged fromdiscussions about politics to narrations about vacations or hob-bies. During all Lombard conditions L1-L3 subjects wore pro-totypes of acustically closed behind-the-ear hearing aids (BTE)especially designed for this experiment. The BTEs allowed theimpairing noise to be introduced directly into the ear, while atthe same time the listener could hear the communication partnerand his/her own voice without loss over the microphone of theBTE thus sustaining full auditory feedback. See Figure 1 fora detailed wiring diagram. The speech signal analysed for thisstudy was not taken from the BTE input but rather recordedfrom two Beyerdynamic Opus 54 headset microphones car-ried by the speakers to minimize reverberation and cross-overto the noise channel; sampling rate was 48kHz and resolution16bit. To ensure that both dialog partners uttered about the sameamount of speech a test supervisor was present during the dialogand interfered by gesture if necessary.Figure 1: Detailed wiring diagram for the recording. To achievethe correct noise levels gains (X) are adjusted by a psycho-acoustic comparison inside the running car.3. Method3.1. Analysed FeaturesThe 96 recordings (24 speakers x 4 conditions L0-L3) were seg-mented into speech and non-speech using Praat4; only the partstagged as speech were used for further analysis.2Unfortunately the total HOESI corpus is not yet freely available dueto copyright negotiations. If you are interested in working on these data,please contact the authors for possible access to parts of the corpus.3Volkswagen Passat4http://www.fon.hum.uva.nl/praat/, version 5.1.19 used.F0 contours were calculated based on the headset micro-phone signal using the Scha¨fer-Vincent algorithm ([10]) andconverted into semitones on a basis of 100Hz to make f0 dy-namics comparable across genders. The Scha¨fer-Vincent algo-rithm marks non-voiced speech parts quite reliable; non-voicedparts were excluded from further f0 analysis.We then calculated root-mean-square energy contours(RMS) using a Blackman window of 200ms size and 20ms shift.From these two contours a set of long-term features (LTF) aslisted in Table 1 were calculated for each recording.Table 1: Long-term features derived from fundamental fre-quency and enegy contours.f0-m median of f0f0-r range of f0 (quarter-quantile distance)rms-m median of RMSrms-r range of RMS (quarter-quantile distance)rms-1 median of RMS differences betweensuccessive RMS minima and maximarms-2a mean time distance of successive RMS maximarms-2b mean time distance of successive RMS minimarms-3a median of RMS differences betweenmean and minimumrms-3b median of RMS differences betweenmean and maximumrms-4a median of slope of rising flanksrms-4b median of slope of falling flanksFeatures rms-1 to rms-4b are called rhythmicity featuresbased on a method developed by Chr. Heinrich (see [11] fordetails). In a nutshell, the RMS contour is first normalized tothe average RMS of the total recording and then stylized into asuccession of maxima and minima where maxima are definedas ’above average RMS’ and minima as ’below average RMS’.Silence intervals greater than 1sec are filtered from this stylizedenergy contour and then the rhythmicity features as indicated inFigure 2 are extracted and averaged (or median) over the totalrecording. Aside from the means/medians as listed in Table 1we also calculated the standard deviation/quarter-quantile dis-tances, thus resulting in 18 measured LTF per recording andspeaker.Figure 2: Stylized RMS contour as basis for rhythmicity fea-tures.Note that the rhythmicity feature rms-1 is not the same asthe quarter-quantile range of RMS rms-r: while rms-r is cal-culated from all absolute RMS contour values, the rms-1 fea-ture is taken from the stylized (and normalized) contour max-ima and minima values. It is therefore a more robust represen-tation of the energy fluctuation typical for speech (CVCVC...)and more robust against varying recording conditions such asdistance from mouth to microphone.The rhythmicity features rms-2a/rms-2b reflect the regular-ity of minima/maxima succession and are therefore inverselycorrelated to speaking rate ([12]).Features rms-3a/b can reveal a possible asymmetry of thestylized RMS contour, while rms-4a/b combine speaking rateand intrinsic dynamics.3.2. StatisticFor all statistical analyses mixed models (MM) were applied([13]). The four increasing Lombard conditions follow a trendwhich is testable for significance within a MM, which also al-lows the speaker to be treated as a random factor5. Speakergender was treated as an additional between-speaker factor inthe MM. F-values higher than the threshold of 8.49 are consid-ered to be significant with a p-level of less than 0.01 ([14]). Inpost-hoc tests MMs were again applied, but only on split gen-der groups to test whether the factor Lombard condition has aninfluence on the features depending on the listener’s gender.4. Results and DiscussionTable 2 displays the results of the MM trend test for the individ-ual features with regard to the four Lombard conditions. Themedians of fundamental frequency f0-m and intensity rms-m in-crease significantly across the four Lombard conditions L0-L3(see Figure 3 and Figure 4); there is no significant gender dif-ference for the trend on both of the main LTF.6Table 2: Mixed Models statistics for extracted features with re-gard to Lombard conditionlogogram significance / interactionsf0-m sig. (F=323.9, p<0.01)f0-r sig. (F=15.2, p<0.01)sig. interaction on gender:female (F=33.2, p<0.01)male n.s.rms-m sig. (F=316.5, p<0.01)rms-r n.s.rms-1 sig. (F=20.4, p<0.01)rms-2a n.s.rms-2b n.s.rms-3a n.s.rms-3b sig. (F=27.9, p<0.01)rms-4a n.s.rms-4b sig. (F=26.2, p<0.01)The range of fundamental frequency f0-r shows no signif-icance on gender and Lombard condition but there is a signif-icant interaction between these two factors. Therefore gendergroups were tested separately which results in a significant trend5To eliminate speaker-dependent idiosyncrasies, for instance f0 reg-ister.6Of course there is a significant difference on absolute f0 betweengenders, but there is no significant difference between gender in thetrend.lllllllllL0.m L1.m L2.m L3.m L0.w L1.w L2.w L3.w051015f0−mfrequency (semitones)menwomenFigure 3: Boxplots LTF median f0 across 4 Lombard conditionsseparated by gender.lllllL0.m L1.m L2.m L3.m L0.w L1.w L2.w L3.w55606570rms−mRMS (dB)menwomenFigure 4: Boxplots LTF median RMS across 4 Lombard condi-tions separated by gender.across Lombard condition for female speakers only (see Figure5): while men keep their f0 range quite consistent, women ex-tend it with every increased Lombard condition.RMS range rms-r shows no significant difference acrossLombard conditions.So far, hypothesis 1 and partly hypothesis 2 can be con-firmed: while the absolute LTF of fundamental frequency andintensity rises with increasing noise, the range basically re-mains constant except for the f0 range of female speakers.On the other hand, the dynamic of the RMS increases withperturbing noise resulting in significant trends in some of therhythmicity features rms-1, rms-3b, rms-4b. Looking moreclosely we find that this significance is again mainly due tothe female speakers (rms-1: F=47.2, p<0.01, rms-3b: F=41.1,p<0.01). Men do not show an increase in their rhythmicity fea-tures, while female speakers do it to such an extent that the mainfactor for both genders is significant. The only exception isthe feature rms-4b (slope of falling flanks) which is significantL0.mL1.mL2.mL3.mL0.wL1.wL2.wL3.w23456f0−r frequency (semitones)menwomen3.63.84.04.2f0−r range plotmean of f0−rL0 L1 L2 L3menwomenFigure 5: Left: boxplots for f0 range f0-r throughout Lombardconditions. Right: range-plot for the mean of f0-r per Lombardcondition separated into genders.across Lombard conditions for both genders.Regarding the range of the rhythmicity features (not shownin Table 2) only rms-1 was found to be significant across Lom-bard conditions, again only for female speakers.Therefore, regarding the second part of hypothesis 2 – RMSrange increases with Lombard noise – we have some mixed re-sults. While the range of LTF RMS does not change, certainrhythmicity features representing intrinsic intensity dynamicsdiffer across Lombard conditions, but only for the female speak-ers.As for the timing features rms-2a, rms-2b there is no indica-tion that subjects slow down their speaking rate with increasingLombard noise.Hence, the third hypothesis (and results of earlier studiesregarding read lab speech) that speakers slow down when im-paired by noise cannot be confirmed by our data.lll l1 2 3 4051015f0−m per female speakercondition L0 till L3median (semitones)051015median (semitones)051015median (semitones)051015median (semitones)051015median (semitones)llll051015median (semitones)051015median (semitones)051015median (semitones)l ll l051015median (semitones)051015median (semitones)051015median (semitones)051015median (semitones)lll l1 2 3 4051015f0−m per male speaker condition L0 till L3051015051015051015051015051015051015051015llll051015051015051015lll l051015Figure 6: F0 medians across Lombard conditions for eachspeaker; left: female speakers, right: male speakers.Another interesting observation regarding gender-specificbehaviour was made on the LTF fundamental frequency. Fig-ure 6 shows the trend of this feature for each speaker, sepa-rated for female (left) and male (right) speakers. With increas-ing Lombard noise the fundamental frequency of the majorityof the male speakers is concentrated in a small range, whereasthe female speakers maintain their individual register through-out Lombard conditions. Whether this covergence is typical formale social behaviour or just a random effect remains to be clar-ified.5. ConclusionWe studied spontaneous dialog speech of 24 native Germanspeakers under real noise Lombard conditions to verify earlierfindings about the Lombard reflex. The recordings involved asophisticated experimental setting where natural hearing wasreplaced with modern hearing aids. This allowed natural noiserecorded from an automotive environment to be inserted intothe auditory feedback loop.Our results show that the main Lombard effects, rising ofintensity and fundamental frequency, can be confirmed for bothgenders. The range of fundamental frequency was increasedas well, but only for the female speakers. Long term range ofintensity did not change across Lombard conditions, but intrin-sic rhythmicity features representing the dynamics of root meansquare energy did - again only for female speakers. Contrary toour expectation speaking rate did not slow down with Lombardnoise.In conclusion we can say that dialog processing in noisyenvironments will have to deal with the same Lombard effectsknown for read speech, except for speaking rate. In addition,Lombard effects cannot be expected to be constant across gen-ders.6. References[1] Lombard E´ (1911) Le signe de l’e´le´vation de la voix. In: Ann.Mal. Oreil. Larynx, Vol 37, pp. 101-119.[2] Lane H, Tranel B (1971): The Lombard sign and the role of hear-ing in speech. In: Journal of Speech and Hearing Research, Vol14, pp. 677-709.[3] Charlip W, Burk K (1969): Effects of noise on selected speechparameters. In: Journal of Communication Disorders, Vol 2, pp.212-219.[4] Junqua J-C, Fincke S, Field K (1999): The Lombard Effect: areflex to better communicate with others in noise. In: Proc. of theICSLP, pp. 2083-2086.[5] Davis C, Jeesun K, Grauwinkel K, Mixdorff H (2006): LombardSpeech: Auditory (A), visual (V) and AV effects. In: Proc. ofSpeech Prosody, pp. 361-365.[6] Lindblom B (1990): Explaining phonetic variation: A sketch ofthe H and H theory. In: Hardcastle W & Marchal A (eds.): SpeechProduction and Speech Modeling. Dordrecht a.o.: Kluwer Aca-demic Publishers, pp. 403-439.[7] Uchanski R, Choi S, Braida L, Reed C, Durlach N (1996): Speak-ing clearly for the hard of hearing IV: Further studies of the roleof speaking rate. In: Journal of Speech and Hearing Research, Vol39, pp. 494-509.[8] Stanton B, Jamieson L, Allen G (1988): Acoustic-phonetic anal-ysis of loud and Lombard speech in simulated cockpit conditions.In: Proc. of the ICASSP, pp. 331-334.[9] Junqua J-C, Fincke S, Field K (1998): Influence of the speakingstyle and the noise spectral tilt on the Lombard reflex and auto-matic speech recognition. In: Proc. of the ICSLP, pp. 467-470.[10] Scha¨fer-Vincent K (1983): Pitch period detection and chaining:method and evaluation. In: Phonetica, Vol 40, pp. 177-202.[11] Schiel F, Heinrich Chr, Neumeyer V (2010): Rhythm and For-mant Features for Automatic Alcohol Detection. In: Proc. of theINTERSPEECH, Chiba, Japan, pp. 458-461.[12] Heinrich Chr, Schiel F (2011): Estimating Speaking Rate byMeans of Rhythmicity Parameters. In: Proc. of the INTER-SPEECH, Florence, Italy, submitted.[13] Baayen R H (2008): Analyzing Linguistics Data: a practical in-troduction to statistics. Cambridge University Press.[14] Reubold U, Harrington J, Kleber F (2010): Vocal aging effects onF0 and the first formant: a longitudinal analysis in adult speakers.In: Speech Communication, Vol 52, pp. 638-651.

Acoustic-phonetic analysis of loud and Lombard speech in simulated cockpit conditions. In:

Analyzing Linguistics Data: a practical introduction to statistics.

Effects of noise on selected speech parameters.

Estimating Speaking Rate by Means of Rhythmicity Parameters. In:

Explaining phonetic variation: A sketch of the H and H theory. In:

Inﬂuence of the speaking style and the noise spectral tilt on the Lombard reﬂex and automatic speech recognition. In:

Le signe de l’´ el´ evation de la voix.

Lombard Speech: Auditory (A), visual (V) and AV effects. In:

Pitch period detection and chaining: method and evaluation.

Rhythm and Formant Features for Automatic Alcohol Detection. In:

Speaking clearly for the hard of hearing IV: Further studies of the role of speaking rate.

The Lombard Effect: a reﬂex to better communicate with others in noise. In:

The Lombard sign and the role of hearing in speech.

Vocal aging effects on F0 and the ﬁrst formant: a longitudinal analysis in adult speakers.

https://epub.ub.uni-muenchen.de/13690/1/schiel_13690.pdf

The Lombard Effect in Spontaneous Dialog Speech

Abstract

Similar works

Full text

Available Versions

Universität München: Elektronischen Publikationen

Open Access LMU