GlaToBI, a version of the ToBI prosodic transcription system which can be used to transcribe the intonation patterns of western Scottish (Glasgow) English, is currently under development. An assessment of GlaToBI, similar to the evaluation studies that were undertaken for the original ToBI system [7], and for GToBI, a version developed for German [4], has been carried out to test the new system &apos;s reliability, learnability and comprehensiveness. The results of this study show that this adaptation of the ToBI system can be applied with the expected level of reliability to the transcription of Glasgow English. 1. INTRODUCTION  Very little corpus based work has been done on the prosodic features of English dialects other than Standard American and southern British (Received Pronunciation). However, with the creation of databases such as the University of Edinburgh&apos;s HCRC Map Task corpus [1], the predominant dialect of which is western Scottish (Glasgow) English, the opportunity has arisen..

Aylett, M.

Ladd, D. R.

Mayo, C.

English

GlaToBI, a version of the ToBI prosodic transcription system which can be used to transcribe the intonation patterns of western Scottish (Glasgow) English, is cur-rently under development. An assessment of GlaToBI, similar to the evaluation studies that were undertaken for the original ToBI system [7], and for GToBI, a version developed for German [4], has been carried out to test the new system’s reliability, learnability and comprehensive-ness. The results of this study show that this adaptation of the ToBI system can be applied with the expected level of reliability to the transcription of Glasgow English. 1

Catherine Mayo

Matthew Aylett

D. Robert Ladd

CiteSeerX

Prosodic transcription of Glasgow English: an evaluation study of GlaToBI

Edinburgh Research Explorer

     Edinburgh Research Explorer                                      Prosodic transcription of Glasgow English: an evaluation studyof GlaToBICitation for published version:Mayo, C, Aylett, M & Ladd, DR 1997, 'Prosodic transcription of Glasgow English: an evaluation study ofGlaToBI'. in Intonation: Theory, Models and Applications.Link:Link to publication record in Edinburgh Research ExplorerDocument Version:Author final version (often known as postprint)Published In:Intonation: Theory, Models and ApplicationsGeneral rightsCopyright for the publications made accessible via the Edinburgh Research Explorer is retained by the author(s)and / or other copyright owners and it is a condition of accessing these publications that users recognise andabide by the legal requirements associated with these rights.Take down policyThe University of Edinburgh has made every reasonable effort to ensure that Edinburgh Research Explorercontent complies with UK legislation. If you believe that the public display of this file breaches copyright pleasecontact openaccess@ed.ac.uk providing details, and we will remove access to the work immediately andinvestigate your claim.Download date: 20. Feb. 2015PROSODIC TRANSCRIPTION OF GLASGOW ENGLISH: AN EVALUATION STUDY OFGLATOBICatherine Mayo  , Matthew Aylett, D. Robert LaddUniversity of Edinburgh,Department of Linguisticscatherin@ling.ed.ac.uk, matthewa@cogsci.ed.ac.uk, bob@ling.ed.ac.ukABSTRACTGlaToBI, a version of the ToBI prosodic transcriptionsystem which can be used to transcribe the intonationpatterns of western Scottish (Glasgow) English, is cur-rently under development. An assessment of GlaToBI,similar to the evaluation studies that were undertaken forthe original ToBI system [7], and for GToBI, a versiondeveloped for German [4], has been carried out to test thenew system’s reliability, learnability and comprehensive-ness. The results of this study show that this adaptationof the ToBI system can be applied with the expected levelof reliability to the transcription of Glasgow English.1. INTRODUCTIONVery little corpus based work has been done on theprosodic features of English dialects other than Stan-dard American and southern British (Received Pronun-ciation). However, with the creation of databases such asthe University of Edinburgh’s HCRC Map Task corpus[1], the predominant dialect of which is western Scottish(Glasgow) English, the opportunity has arisen for inves-tigation into the prosody of these minority dialects. Un-fortunately no agreed upon system exists for the prosodicanalysis of any non–standard English dialects: the orig-inal ToBI labelling system was developed specificallyfor the transcription of the “three most widely used di-alects of English—namely general American, standardAustralian and southern British” [2, p. 2], and is notnecessarily adequate for other dialectal variants withoutadaptation. This paper is a brief report on GlaToBI, anadaptation of the ToBI system for Glasgow English.1.1. Glasgow English intonationGlasgow English, along with other urban north Britishdialects, differs from Standard American and RP Englishin its more frequent use of nuclear rises [3]. In addition,the Glasgow rise is often followed by what Cruttendencalls a “plateau–slump”: after the pitch has risen at theaccented syllable, it remains high (or slightly declining)until very near the edge of the phrase, at which point itfalls again. In an analysis of the alignment of the highand low targets of Glasgow rises, Ladd [5] notes that theC. Mayo is currently afiliated with Queen Margaret College,Edinburghtypical rise in Glasgow English is a glide up on the ac-cented syllable, normally beginning before and peakingafter the accented syllable itself, a claim based on prelim-inary analyses of the HCRC Map Task corpus. These twoelements—the rise and the following plateau–slump—are the points of focus for adaptation of ToBI for Glas-gow English.1.2. GlaToBIAs in the original ToBI system, intonation is transcribedin GlaToBI as a sequence of discrete pitch prominances(pitch accents) and boundary pitch movements (edgetones). In addition, edge tones, as in the original ToBI,are divided into two further categories: phrase accents,which mark the end of intermediate phrases, and bound-ary tones, which the mark the end of full intonationalphrases. Following the ToBI system, which was basedpartly on work by Pierrehumbert [6], GlaToBI uses twomain tones, high (H) and low (L), which can be com-bined in various ways to mark all of these intonationalevents.The original ToBI system has two bitonal rising pitchaccents: L+H*, a step up from an unaccented syllableto the following accented syllable, and L*+H, a step upfrom the accented syllable to the following unaccentedone. However, neither of these accents adequately de-scribes the Glasgow English rise. In GlaToBI, therefore,both of the original ToBI rises have been removed andreplaced by just one compound pitch accent L*H. Theplacement of the “*” diacritic between the L and H tonesindicates that the stressed syllable does not align with ei-ther one or the other, but with the movement from one tothe other.The H– phrase accent in the original ToBI system trig-gers up–step, i.e. it raises the pitch of all followingboundary tones. As a result, in ToBI the phrase–finalcompound tone H–L% is realised as a level tone rela-tively high in pitch. In order to accommodate the Glas-gow English plateau–slump, up–step has been eliminatedfrom the tonal inventory: H–L% in GlaToBI thereforerepresents a phrase final high–low sequence.In addition to the above, GlaToBI has retained thedownstep “!” diacritic to mark the overall decline of thespeaker’s fundamental frequency across an utterance asin the original ToBI system. The break index tier, whichallows the transcriber to represent the strength of bound-aries or breaks between words in an utterance on a scaleL*H L*!H H−L%L*HH−L%yesi’vegot a disused monastery aye0 1 0 1 1 4 4samples0228.20 228.30 228.40 228.50 228.60 228.70 228.80 228.90 229.00 229.10 229.20 229.30 229.40 229.50   Time:228.75755sec  1894D: 1.37000   L:228.12755   R:229.49755 (F:    0.73)F0100150200250300350228.20 228.30 228.40 228.50 228.60 228.70 228.80 228.90 229.00 229.10 229.20 229.30 229.40 229.50   Time:228.75755sec 147.8D: 1.37000   L:228.12755   R:229.49755 (F:    0.73)1. Glasgow English intonation contour with GlaToBI labelsof 0–4, has also been retained unchanged from the origi-nal system. The tonal inventory of GlaToBI is as follows:Pitch accents: H* Edge tones: H–L* L–L*H H–L%!H* H–H%L*!H L–H%L–L%2. EVALUATION STUDYThis section describes an evaluation study of GlaToBIsystem based on the goals set out for the original ToBIsystem: reliability, learnabilty and comprehensiveness[8]2.1. SubjectsThe subjects for this study were three of the four design-ers of the GlaToBI system (“experts”), and four tran-scribers who had not previously been exposed to thesystem (“non–experts”). The non–experts were all atleast somewhat familiar with speech–related research,but only one had had some (rather limited) experiencewith the original ToBI system. All subjects were na-tive speakers of English, with a number of different di-alects represented, including one native speaker of Glas-gow English.2.2. Speech materialsThe materials used for the evaluation were taken from theHCRC Map Task corpus. This corpus consists of task–oriented dialogues in which the participants have to re-produce a specified route on a map. The dialogues aretherefore predominantly spontaneous speech, with somewords occasionally in citation form (e.g. “Do you have‘carpenter’s cottage’ [on your map]?”). In the evalua-tion study, the subjects transcribed 30 utterances, com-prised of 273 orthographic words. The length of eachutterance ranged from 2 words to 19 words, averagingapproximately 9 words per utterance.2.3. TrainingThe four non–expert transcribers were given an intensiveone day training session to familiarise them with ToBI–style transcription, and with GlaToBI, before participat-ing in the evaluation proper. The example utterancesused in the session were taken both from the training ma-terial provided with the original ToBI system (i.e. non–Glasgow English speech) and from the Map Task corpusitself. At the end of the training a pre–test was givento make sure that the system had been fully understood(a fifth non–expert was dropped from the study at thisstage).2.4. Evaluation properAll subjects (non–expert and expert) were tested indi-vidually within the week following the training session.During the test, the subjects were not permitted to consultwith each other or with the experimenter, but the non–experts were allowed to consult their training notes. Notime limit was put on the test; most subjects finished tran-scribing 30 utterances in one day (approximately 8 hoursin total, including breaks). The utterances and transcrip-tion tiers were presented to the subjects using Entrop-ics Waves+ speech analysis software in a UNIX environ-ment (i.e. the standard ToBI platform).2.5. AnalysisAs in the evaluation studies carried out for the originalToBI [7] and GToBI [4], a pairwise analysis was used toassess the level of agreement reached by the subjects inthis study. This method, instead of comparing the labelsof individual transcribers against the group, compares thelabels of each transcriber against the labels of every othertranscriber, for that particular aspect of the utterance. Forinstance, if four out of five transcribers label a word H*,and the fifth labels the same word L*H, the level of agree-ment is not considered to be 4 out of 5, but 6 out of 10—i.e. 6 pairs out of a possible 10 pairs of transcribers whoagree as to the exact label. Pitrelli et al. [7] state thatthis is a stringent method of analysis, because it does notreturn the deceptively high 80% agreement which wouldbe found by an analysis which simply stated that 4 out of5 transcribers agreed.3. RESULTS273 words were transcribed by all seven subjects,totalling 1911 individual transcriptions, and 5733transcriber–pair–words (273 words times 21 transcriberpairs). In the results reported here, as in the ToBI andGToBI studies, downstep was ignored for the purposesof analysis (e.g. !H* was considered to be the same asH*). Similarly, labels with the ‘p’ disfluency diacriticand the ‘–’ uncertainty diacritic were also merged withtheir unmarked labels.3.1. Pitch accentsIn the analysis of agreement for pitch accents, the follow-ing elements were examined for each word in all utter-ances: (i) the rate of agreement as to the presence or lackof an accent on the word, and (ii) the rate of agreementas to the accent marked. The possible labels availablefor marking pitch accents were: H* (and !H*), L*, L*H(and L*!H), and no accent. The level of overall agree-ment as to whether a pitch accent was present on a wordwas 78% , while the overall rate of agreement as to whichpitch accent was present was 68%.Note that in this study, the analysis of agreement as towhich accent/tone is marked on a word includes “no ac-cent/tone” as a valid category. Given a situation in whichtwo transcribers from a group of three or more agree thata word should be given a certain label, with the rest ofthe group leaving the word unlabelled, an analysis whichexamines agreement as to choice of accents/tones onlywhen there is agreement that an accent/tone is present,will find that the rate of agreement for the above situationis 100% (i.e. all transcribers who marked an accent/toneagree to its identity). However, in a group of three ormore transcribers, the pair who marked the accents areonly one pair from a possible 6 or more: an agreement of100% is therefore rather a false positive.3.2. Edge tonesEdge tones were analysed in terms of (i) the rate of agree-ment as to the presence or lack of tone on each word,(ii) the rate of agreement as to the tone marked, and (iii)the rate of agreement as to the strength of the bound-ary marked (full boundary, marked by a boundary tone,intermediate boundary, marked by a phrase accent, orno boundary). For (ii) the labels available to the tran-scribers were: H– (and !H–), L–, H–H% (and !H–H%),H–L% (and !H–L%), L–L%, L–H%, and no boundary.Overall, the level of agreement as to the presence or lackof edge tone was 92%, the highest level of agreementachieved in the study. The rate of agreement as to theedge tone marked was 82%, while transcribers agreedupon the strength of the boundary 88% of the time.3.3. Break IndicesAs it is technically not an option for no break index tobe placed on a word, break indices were only analysed interms of the level of break index marked. A small num-ber of break indices were, however, acidentally omittedby transcribers (the software provided for the originalToBI evaluation which flagged illegal transcriptions wasnot available for the current study). For purpose of anal-ysis, therefore, the categories of break index included anull category: 0, 1, 2, 3, 4, and no break index. The over-all rate of agreement as to the choice of break index usedwas 74%.3.4. Non–experts compared to expertsThe use of non–expert as well as expert GlaToBI tran-scribers was critical in order to assess the ease and ac-curacy with which the new system could be learnt. Asshown in Table 1, the level of agreement among the non–experts (N–E) was generally equal to or only slightlylower than among the experts (E), indicating that withthe appropriate training, GlaToBI can be used by non–expert transcribers to a high level of consistency.N–E E(%) (%)presence of pitch accent 75 81choice of pitch accent 62 69presence of edge tone 92 93choice of edge tone 82 82strength of edge tone 88 88choice of break index 77 72Table 1. Rate of agreement for non–experts com-pared to experts3.5. Results compared with ToBI and GToBIThe rates of agreement for GlaToBI compared to ToBIand GToBI are shown below in Table 2.GlaToBI ToBI GToBI(%) (%) (%)presence of pitch accent 78 80.6 87choice of pitch accent 65 72.4 74presence of edge tone 92 (phrase accent)(boundary tone)89.893.4 n/achoice of edge tone 82 (phrase accent)(boundary tone)85.390.0 86strength of edge tone 88 n/a 86choice of break index 74 70.4 n/aTable 2. GlaToBI compared to ToBI and GToBI3.6. Evaluation of L*HThe most fundamental change made to the original ToBIsystem was the replacement of the two original ToBIrises by L*H. Of the 273 words transcribed in this study,105 were labelled L*H by at least one transcriber. WhenL*H is used unanimously (or confused only with “no ac-cent”) it tends to be in the nuclear position in the utter-ance, and followed by an H–L% tone—i.e. the prototyp-ical Glasgow English rise–plateau–slump. However, 90of the 105 words labelled L*H in this study were alsomarked with a different pitch accent by at least one othertranscriber. 40% of these 90 were instances where L*Hwas also labelled with L* and with H*, while 43% weredisagreements between L*H and H* only. Ladd [5] notesthat for Glasgow English speech “the alignment and scal-ing of the L* and H tones appear to be quite variable. It isnot clear whether all this variation is phonologically con-ditioned (e.g. ‘compression’ conditioned by the numberof available syllables), or whether any of it is meaning-ful” (p. 145). Before GlatoBI is used for large scaletranscription, it will be important to try to resolve thesequestions on the basis of closer study of the different pat-terns of transcriber disagreement.3.7. Evaluation of studyThe levels of agreement achieved with GlaToBI tendedto be very slightly lower than those achieved in the ToBIand GToBI tests. We suggest two possible explanationsfor these differences. Unlike the evaluations of ToBIand GToBI which used read speech as well as sponta-neous speech, the evaluation of GlaToBI used only spon-taneous speech. Spontaneous speech generally suffersmore from disfluencies (hesitations, false starts, etc.) andunclear (or completely flat) pitch contours than does readspeech. As a result, the intonation of spontaneous speechis more difficult to transcribe. Another possible explana-tion for the slightly lower levels of agreement is the factthat the scale of the current evaluation is much smallerthan those carried out for both the original ToBI systemand the GToBI system: ToBI: 489 words, 26 transcribers;GToBI: 733 words, 13 transcribers; GlaToBI: 273 words,7 transcribers. The effect of one or two transcribers dis-agreeing with the others would therefore be greater inthe GlaToBI evaluation than in the two earlier studies.Taking all of this into account, the levels of agreementreached by transcribers using GlaToBI are in fact quitecomparable to those seen in both the original and the Ger-man ToBI systems.REFERENCES[1] A. H. Anderson, M. Bader, E. G. Bard, E. Boyle,G. Doherty, S. Garrod, S. Isard, J. Kowtko, J. McAl-lister, J. Miller, C. Sotillo, H. S. Thompson, andWeinert R. The HCRC Map Task Corpus. Languageand Speech, 34(4):351–366, 1991.[2] M. E. Beckman and G. M.. Ayers. Guidelines forToBI Labelling, Version 1.5. Ohio State University,1993.[3] Alan. Cruttenden. Rises in English. In J. Wind-sor Lewis, editor, Studies in General and EnglishPhonetics: Essays in Honour of Professor J. D.O’Connor, pages 155–173. Routledge, 1994.[4] M. Grice, M. Reyelt, R. Benzmu¨ller, J. Mayer, andA. Batliner. Consistency in transcription and la-belling of German intonation with GToBI. In Pro-ceedings of the 1996 International Conference onSpoken Language Processing, 1996.[5] D. R. Ladd. Intonational phonology. CambridgeUniversity Press, 1996.[6] J. Pierrehumbert. The Phonology and Phonetics ofEnglish intonation. PhD thesis, MIT, 1980.[7] J. Pitrelli, M. Beckman, and J. Hirschberg. Evalua-tion of prosodic transcription labeling reliability inthe ToBI framework. In Proceedings of the 1994International Conference on Spoken Language Pro-cessing, volume 1, pages 123–126, 1994.[8] K. Silverman, M. Beckman, J. Pitrelli, M. Osten-dorf, C. Wightman, P. Price, J. Pierrehumbert, andJ. Hirschberg. TOBI: A standard for labelling En-glish prosody. In Proceedings of the 1992 Interna-tional Conference on Spoken Language Processing,1992.

Consistency in transcription and labelling of German intonation with GToBI.

Evaluation of prosodic transcription labeling reliability in the ToBI framework.

Intonational phonology.

Rises in English. In

The HCRC Map Task Corpus. Language and Speech,

The Phonology and Phonetics of English intonation.

TOBI: A standard for labelling English prosody.

Prosodic Transcription Of Glasgow English: An Evaluation Study Of Glatobi

GlaToBI, a version of the ToBI prosodic transcription system which can be used to transcribe the intonation patterns of western Scottish (Glasgow) English, is currently under development. An assessment of GlaToBI, similar to the evaluation studies that were undertaken for the original ToBI system [7], and for GToBI, a version developed for German [4], has been carried out to test the new system’s reliability, learnability and comprehensiveness. The results of this study show that this adaptation of the ToBI system can be applied with the expected level of reliability to the transcription of Glasgow English

Mayo, Catherine

Aylett, Matthew

Ladd, D. Robert

Heriot Watt Pure

http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.55.2650

Prosodic transcription of Glasgow English: an evaluation study of GlaToBI

Abstract

Similar works

Full text

Available Versions

CiteSeerX

Edinburgh Research Explorer

CiteSeerX

Heriot Watt Pure

Heriot Watt Pure