Batliner, Anton

Hönig, Florian

Nöth, Elmar

Wankerl, Sebastian

English

OPUS Augsburg

Pinpointing the Difference –Visual Comparison of Non-Native Speaker Groups∗Florian Hönig1, Sebastian Wankerl1, Anton Batliner1,2, Elmar Nöth11Pattern Recognition Lab, FAU Erlangen-Nuremberg, Germany2Machine Intelligence & Signal Processing Group, TUM, Munich, Germany{florian.hoenig}@fau.deAbstractWe apply a tool originally developed for comparing patholog-ical and healthy speakers to non-native speech. The methodworks on speakers who produce a given word sequence. Usingtime-alignment, we can display prototypical loudness contours,local tempo variations, and also spectrograms, together with in-formation on variability and group effect size over time. Thesystem, which will be made publicly available, is able to exposetypical differences in a group of German and Italian speakers.Index Terms: pathological speech, non-native speech, visual-ization, interpretation, acoustic features1. IntroductionCharacterizations of non-native speech are often available asstereotypes; for a given database, one can listen through therecordings and obtain a subjective impression. However, theseare often hard to translate into acoustical correlates needed todesign systems for automatic assessment of non-native speech.We show that Visual Comparison Of Speech (VICOS), a methodand tool originally developed for comparing pathological andhealthy speakers [1], can be used to identify such correlates.VICOS characterises speaker groups by visualising prototypi-cal realizations of each group as well as noticeable differencesbetween the groups. It does so locally, so that differences can berelated to individual phonemes, which facilitates interpretabil-ity. All recordings must contain the same word sequence; thus,repetitions, insertions and deletions cannot be studied.2. Method and ResultsUsing penalised [1] dynamic time warping (DTW), we estab-lish a common time basis – relative to a ‘reference’ recording.We calculate loudness and spectrogram, and project these timeseries onto the ‘timing’ of the reference utterance. Local tempovariations are obtained by counting inserted and deleted framesin the alignments. Spectrogram and loudness are normalised,spectrogram and tempo are smoothed. The now fixed-length,directly corresponding time series are used to generate prototyp-ical realizations (average within each group) and within-groupvariability (standard dev. within each group). The effect size ofgroup affiliation is measured by Cohen’s d [2] (can always berelated to significance for constant groups, e. g. for 2x20 per-sons, |d| = 0.8 corresponds to p = 0.02, two-sided t-test).∗The research leading to these results has received funding fromthe German Federal Ministry of Education, Science, Research andTechnology (BMBF) under grant 01IS07014B (C-AuDiT), and theGerman Ministry of Economics (BMWi) under grant KF2027104ED0(AUWL).We use a sentence from the ISLE corpus [3]: We’re plan-ning to travel to egypt for a while or so. Excluding readingerrors, we obtained 19 German speakers (7f, 12m) and 22 Ital-ian speakers (4f, 18m). We omit tempo and spectrogram here;in loudness, cf. Figure 1, idiosyncrasies are identifiable, for ex-ample, German speakers seem to produce the plosive /t/ morearticulate (steeper slope of the mean, and positive effect sizein each second half); the syllable /i:/ in Egypt, bearing bothphrase and word accent, is louder in Italian (blue effect size).3. ConclusionsVICOS is a generic system for rapidly assessing systematicdifferences between speakers on the basis of possibly largedatasets, in an objective, interpretable and quantifiable way. Weshowed that it can be applied successfully for studying non-native speaker groups, too. Current work includes pitch andre-synthesis, and the usage of the projected time series as high-performance, interpretable features for automatic classification.4. References[1] S. Wankerl, F. Hönig, A. Batliner, J. R. Orozco-Arroyave, andE. Nöth, “Visual comparison of speaker groups,” in INTER-SPEECH 2015 (Show and Tell), 2015, to appear.[2] R. Coe, “It’s the effect size, stupid,” in Ann. Conf. of the BritishEducational Research Assoc., 2002, Exeter. [Online]. Available:http://www.leeds.ac.uk/educol/documents/00002182.htm[3] W. Menzel, E. Atwell, P. Bonaventura, D. Herron, P. Howarth,R. Morton, and C. Souter, “The ISLE corpus of non-native spokenEnglish,” in Proc. LREC, Athens, 2000, pp. 957–964.0.0 0.4 0.8 1.2 1.6 2.0 2.4 2.9 3.3 3.7t æ v l t u i d p ttime [sec]302010010log energy [dB]Group A LoudnessMean of Group AMean of Group B0.80.60.40.20.00.20.40.60.8effect sizeFigure 1: Loudness: typical realizations and differences (A =German, B = Italian speakers) for the phrase “travel to egypt”.Solid lines = average, semi-transparent tubes = standard de-viation. Bars at bottom = effect size (yellow/red = positive =̂higher in German; cyan/blue = negative =̂ lower in Italian).Show&Tell Contribution SLaTE 2015, Leipzig, September 4–5, 2015187 Copyright c© 2015 ISCA

Pinpointing the difference - visual comparison of non-native speaker groups

https://opus.bibliothek.uni-augsburg.de/opus4/files/71584/Hoenig15-PTD.pdf

Pinpointing the difference - visual comparison of non-native speaker groups

Abstract

Similar works

Full text

Available Versions

OPUS Augsburg