This paper introduces ProsodyPro — A software tool for

facilitating large-scale analysis of speech prosody, especially

for experimental data. The program allows users to perform

systematic analysis of large amounts of data and generates a

rich set of output, including both continuous data like timenormalized

F0 contours and F0 velocity profiles suitable for

graphical analysis, and discrete measurements suitable for

statistical analysis. It maximizes efficiency by automating

tasks that do not require human judgment, and saving analysis

output in formats that are ready for further graphical and

statistical analysis

Xu, Y

UCL Discovery

TRASP'2013 Aix-en-Provence, France [O1-1]7ProsodyPro — A Tool for Large-scale Systematic Prosody AnalysisYi XuDepartment of Speech, Hearing and Phonetic Sciences, University College London, Chandler House, 2 Wakefield Street, London WC1N 1PF, UKyi.xu@ucl.ac.ukAbstractThis paper introduces ProsodyPro — A software tool forfacilitating large-scale analysis of speech prosody, especiallyfor experimental data. The program allows users to performsystematic analysis of large amounts of data and generates arich set of output, including both continuous data like time-normalized F0 contours and F0 velocity profiles suitable forgraphical analysis, and discrete measurements suitable forstatistical analysis. It maximizes efficiency by automatingtasks that do not require human judgment, and saving analysisoutput in formats that are ready for further graphical andstatistical analysis.Index Terms: ProsodyPro, time-normalization, annotation, F0velocity1. IntroductionThe need for stringent experimental control in prosody research is increasingly recognized [1, 4, 7, 8]. The analysis of experimental prosody data, however, is not always a straightforward matter. There is often a dilemma between systematic comparison of discrete measurements [8] and detailed analysis of continuous prosody [3, 5]. In most cases, details are sacrificed in favor of straightforward comparisons. In the intonation literature, for example, typically continuous F0 contours of a few utterances are shown as illustrations, and then subsequent analyses are done on only a limited set of measurements, with little or no further examination of continuous contours. This leaves many details in continuous prosody unknown. A likely reason for such compromise is the lack of tools that can simultaneously facilitate both types of analysis.2. ProsodyPro — An integrated toolThis paper introduces ProsodyPro, a Praat script that allowsusers to combine systematic comparison with detailed analysisof continuous prosody. The key design is to use time-normalization to facilitate direct comparison of continuous F0contours, while at the same time generate multiplemeasurements from non-time-normalized data suitable forstatistical analysis.2.1. Time-normalizationTime-normalization is the key method of ProsodyPro tofacilitate close scrutiny of continuous F0 contours overmultiple tokens. When an experiment has recorded manysentences, especially when each unique sentence is repeatedseveral times by each speaker and by multiple speakers, itbecomes difficult to analyze and virtually impossible to reportall the data. For example, Figure 1a shows F0 contours of theMandarin tone sequence HLFHH produced by four malespeakers. These F0 tracks are generated by the “Export visiblepitch” function in Praat [2]. Visually we can see somesimilarities across the speakers despite the differences. Buthow can we capture the similarities? One way is to take ameasurement in the middle of all syllables and average themacross the repetitions as well as the speakers, as shown inFigure 1b, from which we can see that the greatest differencesoccur on syllable 2, which carries four alternative tones.However, while statistics may show a significant differenceacross the four tones with this kind of measurement, manyfiner differences are lost. In Figures 1c and 1d, as two or threemeasurements are taken from each syllable, more details startto emerge. But it is not until Figure 1e, where eightmeasurements are taken from each syllable, does thecontinuous nature of the F0 contours clearly emerge. Normalized timeFigure 1: a. Raw F0 tracks of HLFHH by 4 male speakers, generated by Praat. b-e. Mean time-normalized F0 of a five-syllable Mandarin sentence sampled at 1, 2, 3 and 8 samples per syllable.b.c.d.e.a.TRASP'2013 Aix-en-Provence, France [O1-1]8Thus time-normalization allows averaging across repetitionsas well as speakers, a process that also smoothes out randomvariations unintended by the speaker, as well as individualdifferences, leaving only consistent variations due to tone andcontextual tonal variations. From Figure 1 we can also see thattime-normalization is only a further extension of the coarsersampling as in Figures 1b-1d, which are in fact also time-normalized. But the finer sampling allows us to see muchmore details, leaving little to guesswork. This can be seen inthe two examples in Figure 2, where similarities in focusrealization can be seen between Mandarin and English. Notethat the English F0 contours in Figure 2b are displayed in realrather than normalized time. This is achieved by taking timevalues together with F0 values, and averaging both acrossrepetitions and speakers. The averaged real time can then beused as the time axis for plots like Figure 2b.Figure 2: Mean F0 contours of Mandarin (a) and English (b) sentences in different focus conditions.  The plots are adapted from studies [6, 11] using precursor versions of ProsodyPro.Time-normalization, however, requires users to define thetemporal domain of normalization. In ProsodyPro this is doneby inserting interval boundaries in the TextGrid of anutterance. Technically ProsodyPro allows the use of any unitsas the temporal domain of normalization, e.g., syllable, word,or even phrase. But it is important that there are good reasonsto believe that the F0 contours of the unit are consistentlyproduced. Our recommendation is to use the syllable (orrhyme) whenever possible. This is based on evidence thatspeakers produce syllable-sized contours consistently [9, 10]. 2.2. Other continuous prosody outputIn addition to time-normalized F0 contours, ProsodyPro alsogenerates a number of non-time-normalized continuousprosody outputs. The following is a list of all the continuousprosody files, with a variety of time scales:X.rawf0 — Real-time F0 (Hz) converted directly fromvocal periods (F0 = 1/T, where T is vocal period insecond) marked in X.pulseX.f0 — Real-time F0 (Hz) smoothed with a trimmingalgorithm (Xu, 1999)X.smoothf0 — Real-time F0 (Hz), smoothed from X.f0with a triangular filterX.normtimef0 — Time-normalized F0 (Hz), with defaultsampling rate of 10 points/intervalX.actutimenormf0 — Time-normalized F0 (Hz) with real-time x-axis valuesX.samplef0 — F0 (Hz) sampled at at fixed time intervalsspecified by “f0 sample rate”X.semitonef0 — Semitone version of X.samplef0 X.f0velocity — Continuous F0 velocity (instantaneousrates of F0 change) in semitone/s sampled at timeintervals specified by "f0 sample rate" X.normtime_f0velocity — Time-normalized continuous F0velocityAll these files are automatically generated after an utterance isannotated. These outputs allow users to examine continuousprosody of each utterance. However, as explained above, onlythe time-normalized ones can be averaged into mean contoursacross repetitions and speakers.2.3. Discrete measurementsTime-normalization, despite its many advantages asmentioned above, is meant only for graphical comparisons.For statistical analysis, it is impractical to compare every time-normalized point. Nonetheless, time-normalization allows usto see the locations and manners of maximum differenceswhen mean continuous contours from different experimentalconditions are plotted in overlaid graphs, like those in Figure1e and Figure 2. This can help identify optimal measurementsthat best reflect the key differences between experimentalconditions, and, just as importantly, avoid pitfalls. Figure 2a,for example, shows that the F0 at the onset of the first post-focus syllable “-na” is higher than in the same syllable in theneutral focus contour. This means that maxf0 of that syllable isnot the best measurement for showing the lowered F0characteristic of all the other post-focus syllables.The following measurements are automatically generatedby ProsodyPro for each non-blank interval in the TextGrid andsaved in the file X.means, where X stands for the name of thesound file being analyzed:1. maxf0 — maximum F0 in Hz2. minf0 — minimum F0 in Hz3. excursion_size — difference between maximum F0and minimum F0 in semitones4. meanf0 — average F0 in Hz5. max_velocity — maximum F0 velocity in semitones/s6. finalf0 — F0 near the interval offset in Hz7. final_velocity — F0 velocity near the interval offsetin semitones/s8. duration — interval duration in ms9. mean intensity — mean intensity in dB2.4. Ensemble filesTo facilitate both graphical and numerical analysis,ProsodyPro has a function to pool the outputs of all individualsounds in a folder together into a large set of ensemble files.The following is a list of ensemble files generated by thecurrent version of ProsodyPro (some are optional):1) normf0.txt2) normtime_semitonef0.txt3) normtime_f0velocity.txt4) normtimeIntensity.txta.b.TRASP'2013 Aix-en-Provence, France [O1-1]95) normactutime.txt6) samplef0.txt7) f0velocity.txt8) maxf0.txt9) minf0.txt10) excursionsize.txt11) meanf0.txt12) duration.txt13) maxvelocity.txt14) finalvelocity.txt15) finalf0.txt16) meanintensity.txt17) mean_normf0.txt18) mean_normtime_semitonef0.txt19) mean_normtime_f0velocity.txt20) mean_normtimeIntensity.txt21) mean_normactutime.txt22) mean_maxf0.txt23) mean_minf0.txt24) mean_excursionsize.txt25) mean_meanf0.txt26) mean_duration.txt27) mean_maxvelocity.txt28) mean_finalvelocity.txt29) mean_finalf0.txt30) mean_meanintensity.txt31) mean_normf0_cross_speaker.txtThese are all text files that can be opened by spreadsheet,graphing or statistical programs. The first seven files containcontinuous prosody of all the annotated sounds in the folder innormalized, real or sampled time. Files 8-16 each containsvalues of a specific measurements from all the sounds. Files17-30 contain values averaged across the repetitions of uniquesentences. And file 31 contains time-normalized F0 valuesaveraged across multiple speakers. Values in files 17-21 areready for graphing mean F0, F0 velocity or intensity contoursof individual speakers. Values in file 31, which containsacross-speaker means, can generate plots like those in Figures1e and 2. The discrete mean measurements in files 22-30 areready for statistical analysis such as anova and t-test, forwhich each subject needs to contribute only a single meanvalue for each of the factors being tested.3. Workflow and time-saving featuresTo analyze large amounts of experimental data, much labor isneeded simply to process the data files, taking and recordingthe measurements, and readying the output data for graphicaland statistical analysis. ProsodyPro has incorporated manylabor-saving features specifically designed for experimentalresearch. • First, ProsodyPro is written as a Praat script, so that it isexecutable on all major computer platforms. • Second, the entire program consists of a single script fileto be run in the same folder as the sound files beinganalysized, thus reducing installation effort to aminimum. • Third, ProsodyPro automates tasks that require littlehuman judgment, including, in particular, finding andopening sound files, and saving analysis results.• Finally, ProsodyPro arranges output data in formats thatare nearly ready to be processed further by spreadsheet,graphing and statistical programs.The following is a brief sketch of the workflow ofPrododyPro, with graphic illustrations shown in Figure 3.1. Put ProsodyPro.praat in the folder containing the sound files to be analyzed.2. Launch ProsodyPro either from “Open Praat Script...” command from Praat menu, or by double-clicking the ProsodyPro icon if it is recognizable as a Praat script by the operating system.3. When the script window opens in Praat, select “Run” from the Run menu (or use the shortcut command-r or control-r).4. In the startup window (Figure 3a), check or uncheck the boxes when necessary, and set appropriate values in the text fields or simply use the default ones. Select the task “Interactive labeling” by ticking the first radio button. 5. Click Ok and three windows will appear. The PointProcess window (Figure 3b) displays waveform together with vocal cycle marks (vertical lines) generated by Praat. Here one can manually add missing marks (e.g., in the last vocalic sound in the figure) and delete redundant ones. This needs to be done only for the named intervals, as explained next. 6. The TextGrid window (Figure 3c) displays waveform and spectrogram of the current sound. (Note that the pitch track in the spectrogram panel is not used by ProsodyPro.)7. At the bottom of this window are the annotation tiers, where users can insert interval boundaries (Tier 1) and comments (Tier 2). Any blank intervals (or those labeled “sil”) in Tier 1 are ignored, which allows easy exclusion of temporal regions from analysis. The interval labels can be as simple as a, b, c... or 1, 2, 3..., since ProsodyPro ignores label content.8. The Pause window (Figure 3d) controls the workflow. To bring up the next sound to be analyzed, change the number (or leave it as is) in the current_file box and press "Continue". The number indicates the order in the String object "list" in the Object window (and in the file “FileList.txt” automatically saved to the current folder). The next sound will be 1 + current_file (So, entering 0 opens sound 1).9. To end the current analysis session, press "Finish" in the Pause window, and the order number of the last sound analyzed is shown in the Praat Info window. That number can be used as a starting point in the next analysis session. 10. After processing individual files, ensemble files can be generated by running ProsodyPro again with the “Get ensemble files” radio button checked. 11. The analysis parameters in the startup window can be modified after annotating all the sound files. To do so, start a new session with the radio button "Process all sounds without pause" checked. ProsodyPro will exhaustively run through all files nonstop.12. To generate the means files (files 17-30 listed in Section 2.4), set the value of “Nrepetitions” in the startup window to the number of repetitions in the dataset when running ProsodyPro with the "Get ensemble files" button checked. Note that the number of labeled intervals must be identical across repetitions.TRASP'2013 Aix-en-Provence, France [O1-1]10setting control and analysis parameters, b.  PointProcess window for manually rectifying vocal  pulse markings, c. TextGrid window for segmentation and annotation, d.  Pause window for controlling workflow.13. To generate mean F0 contours averaged across speakers, the following steps can be followed:a. Create a text file (speaker_folders.txt) containing the speaker folder names arranged in a single column. b. Run ProsodyPro with the 4th task (Average across speakers) checked. The script will read mean_normf0.txt from all the speaker folders, average the f0 values on a logarithmic scale, and then convert them back to Hz.c. The grand averages are saved in "mean_normf0_cross_speaker.txt". 4. LimitationsDespite its many time-saving features, the use of ProsodyProis still often labor-intensive. This is first because typicalexperimental datasets are large, having multiple speakers andmultiple repetitions. Secondly, the rectification of vocal pulsemarkings can be time-consuming, which is especially the casewhen the expectation for accuracy of F0 tracking is raised oncehuman intervention is involved. While this may not be a realdisadvantage given that the amount of labor is proportional tothe increase in accuracy of prosody analysis, it is desirable todevelop methods in future versions that can accelerate thelabeling and vocal pulse marking process.5. ConclusionsAs an integrated prosody analysis tool, ProsodyPro resolvesthe dilemma between detailed analysis of continuous prosodyand systematic comparison of discrete measurements. It alsominimizes labor by automating tasks that do not requirehuman judgment, and facilitates human intervention ofprocesses that are prone to error, thus delivering high accuracyand reliability in prosody analysis.6. References[1] Bruce, G., and Touati, P., “On the analysis of prosody inspontaneous speech with exemplification from Swedish andFrench”, Speech Communication, 11:453-458, 1992.[2] Boersma, P., “Praat, a system for doing phonetics by computer”,Glot International, 5:9/10:341-345, 2001.[3] Hawkins, S., “Roles and representations of systematic finephonetic detail in speech understanding”, Journal of Phonetics,31:373–405, 2003.[4] Nakai, S., Turk, A. E., Suomi, K. et al., “Quantity constraints onthe temporal implementation of phrasal prosody in NorthernFinnish”, Journal of Phonetics. 40(6):796-807, 2012.[5] Post, B., D’Imperio, M., and Gussenhoven, C., “Fine phoneticdetail and intonational meaning”, Proceedings of The 16thInternational Congress of Phonetic Sciences, Saarbrücken, 191-196, 2007.[6] Xu, Y., “Effects of tone and focus on the formation andalignment of F0 contours”, Journal of Phonetics, 27:55-105,1999.[7] Xu, Y., “In defense of lab speech”, Journal of Phonetics, 38:329-336, 2010.[8] Xu, Y., “Speech prosody: A methodological review”, Journal ofSpeech Sciences, 1:85-115, 2011.[9] Xu, Y., and Liu, F., “Tonal alignment, syllable structure andcoarticulation: Toward an integrated model”, Italian Journal ofLinguistics, 18:125-159, 2006.[10] Xu, Y., and Liu, F., "Intrinsic coherence of prosody andsegments", Understanding Prosody – The Role of Context,Function, and Communication, O. Niebuhr, ed., 1-26: Walter deGruyter, 2012.[11] Xu, Y., and Xu, C. X., “Phonetic realization of focus in Englishdeclarative intonation”, Journal of Phonetics, 33:159-197, 2005.

ProsodyPro — A Tool for Large-scale Systematic Prosody Analysis

http://discovery.ucl.ac.uk/1406070/1/Xu_TRASP2013.pdf

ProsodyPro — A Tool for Large-scale Systematic Prosody Analysis

Abstract

Similar works

Full text

Available Versions

UCL Discovery