This paper describes the latest version of the SOLE concept-to-speech system, which uses linguistic information provided by a natural language generation system to improve the prosody of synthetic speech. We discuss the types of linguistic information that prove most useful and the implications for text-to-speech systems

Black, Alan W

Hitzeman, Janet

Mellish, Chris

Oberlander, Jon

Taylor, Paul

This paper describes the latest version of the SOLE concept-tospeech system, which uses linguistic information provided by a natural language generation system to improve the prosody of synthetic speech. We discuss the types of linguistic information that prove most useful and the implications for text-to-speech systems. 1

Janet Hitzeman

Alan W. Black

Paul Taylor

Chris Mellish

Jon Oberlander

CiteSeerX

On the use of automatically generated discourse-level information in a concept-to-speech synthesis system

Black, Alan W.

Edinburgh Research Explorer

     Edinburgh Research Explorer                                      On the use of automatically generated discourse-levelinformation in a concept-to-speech synthesis systemCitation for published version:Hitzeman, J, Black, AW, Taylor, P, Mellish, C & Oberlander, J 1998, On the use of automatically generateddiscourse-level information in a concept-to-speech synthesis system. in The 5th International Conference onSpoken Language Processing, Incorporating The 7th Australian International Speech Science andTechnology Conference, Sydney Convention Centre, Sydney, Australia, 30th November - 4th December1998. ISCA, pp. 2763-2766, 5th International Conference on Spoken Language Processing, Sydney,Australia, 30/11/98.Link:Link to publication record in Edinburgh Research ExplorerDocument Version:Publisher's PDF, also known as Version of recordPublished In:The 5th International Conference on Spoken Language Processing, Incorporating The 7th AustralianInternational Speech Science and Technology Conference, Sydney Convention Centre, Sydney, Australia, 30thNovember - 4th December 1998General rightsCopyright for the publications made accessible via the Edinburgh Research Explorer is retained by the author(s)and / or other copyright owners and it is a condition of accessing these publications that users recognise andabide by the legal requirements associated with these rights.Take down policyThe University of Edinburgh has made every reasonable effort to ensure that Edinburgh Research Explorercontent complies with UK legislation. If you believe that the public display of this file breaches copyright pleasecontact openaccess@ed.ac.uk providing details, and we will remove access to the work immediately andinvestigate your claim.Download date: 05. Apr. 2019ON THE USE OF AUTOMATICALLY GENERATED DISCOURSE-LEVELINFORMATION IN A CONCEPT-TO-SPEECH SYNTHESIS SYSTEMJanet Hitzemanab Alan W. Blacka Paul Taylora Chris Mellishc Jon Oberlanderba. Centre for Speech Technology Researchb. Human Communication Research Centrec. Department of Artificial IntelligenceUniversity of EdinburghEdinburgh EH1 1HN, GBhttp://www.cstr.ed.ac.uk/projects/sole.html email: J.Hitzeman@ed.ac.ukABSTRACTThis paper describes the latest version of the SOLE concept-to-speech system, which uses linguistic information provided by a nat-ural language generation system to improve the prosody of syntheticspeech. We discuss the types of linguistic information that provemost useful and the implications for text-to-speech systems.1. INTRODUCTIONThe purpose of the SOLE project is to make use of automatically-generated, high-level linguistic information to improve the qualityof the intonation of synthetic speech. After choosing an initial set oflinguistic constructs thought to have some influence on prosody, wedeveloped an SGML-based mark-up language to serve as a generalinterface between NLG and speech synthesis systems, and trainedour synthesis system to recognise correlations between the mark-upand intonational contours so that it can make use of this mark-upwhen synthesising. As a result, many of the errors that the syn-thesiser makes with regard to knowing when to accent or deaccenta word are absent in the SOLE output. This paper reports on thecurrent results and discusses the implications for text-to-speech sys-tems in cases where it is realistic to use statistical methods for ex-ploiting certain types of high-level linguistic information.2. THE SOLE SYSTEMThe SOLE concept-to-speech system is designed to work as aportable museum guide: visitors to a museum carry a portable de-vice which detects what exhibits they are looking at and gives spo-ken explanation. SOLE generates its descriptions from a databaseof the museum exhibits’ properties. As it keeps a record of whatexhibits have already been visited, it is able to generate descriptionsof new exhibits with reference to previous ones. This gives rise to alarge number of discourse-level linguistic phenomena such as var-ious types of anaphoric reference (e.g., pronouns, definite descrip-tions, bridging references) and rhetorical relations (e.g., contrastingtwo exhibits or amplifying a particular property of an exhibit).The NLG component of SOLE was developed for the ILEXproject [5], and currently it is used for describing exhibits in theRoyal Museum of Scotland’s National Jewellery Gallery. The text-to-speech component is the Festival system. 11http://www.cstr.ed.ac.uk/projects/festival.htmlThe intonation component of Festival [4] works by using a decisiontree to analyse a set of features associated with a syllable, and todecide if a pitch accent should be assigned at that point. Typicalfeatures used include lexical stress and position in phrase etc. InSOLE, we now have access to the discourse-level information, andthis greatly enriches the feature set that the decision tree uses.3. METHODIn order to train the decision tree to use higher-level linguistic infor-mation in determining pitch accent placement, we needed a corpusconsisting of the types of descriptive texts that the ILEX systemproduces. At the time the SOLE project began, however, ILEX wasin an early stage of development, so, rather than using ILEX outputfor our corpus, we gathered a corpus of texts of the sort that ILEXwould be able to produce in its later stages. Our corpus consistsof 43 short descriptive texts, which gives us 35 minutes of speechand a total of 6331 syllables, 863 of which we set aside for test-ing. We annotated this corpus with linguistic information, whichinvolved deciding on an initial set of linguistic constructs that influ-ence prosody and that can be produced by ILEX, and developing aset of SGML tags to describe these constructs. We then recordedthree speakers reading these texts, and human labellers marked ac-cents on the speech by looking at the F0 contours.Given the tagged text, we were able to extract the linguistic infor-mation on a per-syllable basis and use it as a set of features to trainthe decision tree. The SOLE NLG component (i.e., the augmentedILEX system) automatically produces tagged text, which the traineddecision tree then uses in determining accent placement. In the sec-ond phase of the project we will annotate the corpus with Tilt pa-rameters [11] (accent duration, amplitude, peak position, etc.) andwe will also predict these values.Of the phenomena we chose to annotate in the first phase of theproject, noun phrases (with their syntactic, semantic and referencetype) and rhetorical structure gave the most significant results foraccent placement, so we will restrict our discussion here to theseconstructs.3.1. Linguistic annotationRhetorical relations. Rhetorical relations are discourse-level se-mantic relationships between segments of text. Some rhetoricalrelations, such as contrast and list, clearly have a correspondingintonational pattern; with others, such as definition and exemplifi-cation, the effect on intonation is not as obvious. Examples of a fewof the types of rhetorical relations we chose to annotate are below:(1) List: [Purple, white and green] were the colours of the suf-fragette movement.(2) Similarity: [[Like the necklace designed by Flockinger,] [thisitem is in the Organic style.]](3) Concession: [[This item is from the same period,] [but itdoesn’t have the same quality of workmanship.]]Each rhetorical structure can contain one or more rhet-emph tags,which mark the phrases within the text that express the properties orobjects being compared, contrasted, listed, etc. The following con-trastive rhetorical structure illustrates our SGML-based annotation:(4) <rhet-elem type=“contrast”><nucleus> The<rhet-emph type=“object”> god </rhet-emph>was<rhet-emph type=“property”> gilded </rhet-emph>;</nucleus><nucleus> the<rhet-emph type=“object”> demon </rhet-emph>was<rhet-emph type=“property”>stained in black ink and polished to a high sheen</rhet-emph>.</nucleus></rhet-elem>Because we are only concerned with predicting accent placement inthe first phase of the SOLE project, the rhetorical emphasis (rhet-emph) is the only relevant annotation; the rhetorical structure type,rhetorical emphasis type and the nuclei and satellites will be impor-tant when predicting tune in the next phase of the project.Noun phrases. It is well known that old information tends to bedeaccented and new information tends to be accented [1, 3]. Thefirst time an object is mentioned in a text it is part of the new infor-mation in that text, and all subsequent references to that object areconsidered references to old information, as illustrated in 5:(5) It was worn mainly by teenagers, to show that they were Bea-tles fans, or perhaps to show which of the Beatles they likedbest.The first time the NP Beatles is mentioned in the text, it is new andlikely to be accented; the subsequent reference to the Beatles refersto old information, and is unlikely to be accented.Making use of old and new information is becoming more commonin concept-to-speech systems (e.g., [9, 6, 8]). We chose a morecomplex annotation scheme for NPs, assigning them a referencetype, a syntactic type and an optional semantic type.In addition to annotating NPs as anaphors (old information) andfirst-mentions (new information), we used a third reference type,predicative, illustrated in 6:(6) This item is [a brooch].A predicative NP is one that generally occurs as the object of to be,giving a description of the subject.Among the syntactic types we assigned to NPs are the following: definite NP: Any NP using the definite determiner (the).– the brooch, the north-west portion of the coastline of theFirth of Forth bare-singular: A singular NP without a determiner.– jewellery, 1920, purple, solidarity N modifier: A noun that modifies the head noun in a noun-noun compound.– [costume] jewellery, a [dress]clip, an [Edinburgh] jew-ellerThe semantic types we chose to annotate are below: proper name: E.g., Jesse M. King, Scotland, the Middle Ages kind: An NP that describes a kind of object rather than aninstance of an object.– jewellery, people, the mass-produced variety of jewelrywhich was popular during the 1930sAn example of the annotation is in 7 (Note that the term anaphora-elem could be replaced with noun-phrase):(7) was<anaphora-elem ref-type=“predicative” syn-type=“indefinite-NP”>an<anaphora-elem ref-type=“first-mention”syn-type=“N-modifier” sem-type=“PN”>Edinburgh</anaphora-elem>jeweller</anaphora-elem>4. RESULTSTable 1 gives a comparison of the number of errors made by theTTS system using the original set of features with the number oferrors made when the SOLE linguistic features were added to theset. Overall, the addition of linguistic features reduces the error inaccent prediction by 15.5%. The features in Table 1 show the largestcontribution to the error reduction.The first two features in Table 1 are purely syntactic indicators ofwhether a syllable is in an NP or an embedded NP. This simple clas-sification isn’t very useful, as shown by the small reduction in errorSyllable feature Total occurrences TTS errors TTS + SOLE errors % error reductionany syl in an NP 601 88 85 3.4any syl in embedded NP 213 33 29 12.1any syl in an anaphor 135 22 3 86.4last stressed syl in anaphor 41 12 9 25.0any syl in a first-mention 276 35 6 82.9last stressed syl in first-mention 54 11 6 45.5any syl in a predicative NP 69 15 5 66.7any syl in a definite NP 153 18 1 94.4any syl in a bare-singular 114 24 16 33.3any syl in an N-modifier 12 3 0 100.0any syl in a deictic NP 52 5 3 40.0any syl in a proper name 114 21 0 100.0first stressed syl in a proper name 37 10 0 100.0any syl in a kind 77 7 0 100.0any syl in rhet-emph 678 104 94 9.6last stressed syl in rhet-emph 46 12 5 58.3Table 1: A comparison of the TTS system with the SOLE systemfor both features. However, the next four features, which includeinformation concerning the reference type of the NP, show large re-ductions in error. As expected, anaphors tend to be deaccented andfirst-mentions to be accented. What is unexpected is that our resultscontradict the general claim in the literature that when a phrase isaccented the accent is placed at the end of that phrase [2, 7]: in ourdata, the feature indicating that a syllable is any (stressed) syllablein a first-mention proves more useful than the feature indicating thata syllable is the last stressed syllable in a first-mention.2The next eight features also give unexpected results; PredicativeNPs, definite NPs, bare singular NPs, N-modifiers, deictic NPs,proper names and kinds are not typically spoken of as indicatorsof accenting. With N-modifiers and deictic NPs, admittedly, thenumbers are small and it is therefore difficult to make a strong ar-gument that they will be reliable indicators of accenting in anothercorpus. However, for the other features the reduction in error islarge. Predicative NPs express new information, which explains theobservation that they tend to be accented; the discovery that an NPfollowing the verb “is” or “are” is likely to be accented has strongimplications for TTS systems with shallow statistical parsing mech-anisms. In contrast, definite NPs generally express old informa-tion because they refer to objects previously mentioned in the text;3again, the implication is that a TTS system could predict deaccent-ing on an NP beginning with the. Proper names, bare singular NPsand kinds can either express old or new information, so it is surpris-ing that they serve as indicators of accenting. Also surprising is thatit is useful to know whether a syllable is the first lexically stressedsyllable in a proper name.2Our decision tree always predicts that a syllable without lexical stressis deaccented, and the linguistic features are therefore only used in decidingwhether a stressed syllable is accented.3There are exceptions, such as definites that refer to objects in the com-mon ground, as in “[The weather] is lousy today”, and bridging references,which are definites that refer to an object closely related to a previously men-tioned object, as in “There is a house on the hill. [The door] is green.” Thereis no consensus on whether these constructs describe old or new information.The last two features concern rhetorical structure. Both using a fea-ture indicating whether a syllable is inside a rhet-emph tag andusing a feature indicating whether a particular syllable is the lastlexically stressed syllable in that tag gave a reduction in error. Thereduction in error is greater with the latter feature, indicating that inrhetorical emphases the accent tends to be placed at the rightmostportion of the emphasised phrase.The type of rhetorical structure had negligible effect on accentplacement, although, unsurprisingly, the phrases with rhetorical em-phasis had a tendency to be accented. We predict that becauserhetorical structures have tunes which are dependent on their type,their type will be important in the next phase of the project, whenwe train the system to recognise other features of an accent, such asduration and amplitude.5. DISCUSSIONThere are three central conclusions to be drawn from our results:1. In the domain of descriptive texts, certain types of high-levellinguistic information are useful in determining accent place-ment, and therefore coupling a natural language generationsystem with a speech synthesis system is a good idea;2. Surprisingly, kinds, bare-singular NPs and proper names aregood predictors of accents; and3. A TTS system with a statistical parsing mechanism wouldbenefit from singling out predicative NPs, definite NPs andbare-singular NPs (both proper names and kinds) because theyare easily recognised via statistical methods and are good pre-dictors of accent placement.Another goal of the SOLE project is to formalise the provision ofdiscourse-level information as a set of SGML tags as part of thestandardisation efforts of the SABLE consortium [10]. The inten-tion here is to design a powerful interface language between lan-guage generation and speech synthesis systems, so that the synthesissystems can produce high quality speech in a variety of applicationsand domains.6. AcknowledgementsSOLE is funded by the EPSRC, grant reference GR/L50341.7. REFERENCES1. Wallace L. Chafe. Language and consciousness. Language,50:111–133, 1974.2. N. Chomsky and M. Halle. The Sound Pattern of English. Harperand Row, New York, 1968.3. David Crystal. Prosodic features and linguistic theory, pages1–46. The English tone of voice: Essays in intonation, prosodyand paralanguage. Edward Arnold, London, 1975.4. K. Dusterhoff and A. W. Black. Generating F0 contours forspeech synthesis using the Tilt intonation theory. In Proceedingsof the ESCA Workshop on Intonation, Athens, Greece, 1997.5. Janet Hitzeman, Chris Mellish, and Jon Oberlander. Dynamicgeneration of museum web pages: The intelligent labelling ex-plorer. Archives and Museum Informatics, 11:107–115, 1997.Also presented at the Museums and the Web Conference, LosAngeles, March 1997.6. Laurie Hiyakumoto, Scott Prevost, and Justine Cassell. Seman-tic and discourse information for text-to-speech intonation. InKai Alter, Hannes Pirker, and Wolfgang Finkler, editors, Con-cept to Speech Generation Systems, pages 47–56, Madrid, Spain,July 1997. Proceedings of a Workshop Sponsored by the Asso-ciation for Computational Linguistics.7. R. Jackendoff. Semantic Interpretation in Generative Grammar.MIT Press, Cambridge, MA, 1972.8. Christine H. Nakatani. Computing Prosody, Integrating Prosodicand Discourse Modelling. Springer-Verlag, 1997.9. Scott Allan Prevost. A Semantics of Contrast and InformationStructure for Specifying Intonation in Spoken Language Gener-ation. PhD thesis, University of Pennsylvania, 1995.10. R. Sproat, A. Hunt, M. Ostendorf, P. Taylor, A. W. Black,K. Lenzo, and M. Edginton. SABLE: A standard for TTSmarkup. ICSLP98, this volume.11. P. Taylor. The Tilt Intonation Model. ICSLP98, this volume.

Edinburgh Research Archive

ON THE USE OF AUTOMATICALLY GENERATED DISCOURSE-LEVELINFORMATION IN A CONCEPT-TO-SPEECH SYNTHESIS SYSTEMJanet Hitzeman Alan W. Black Paul Taylor Chris MellishJon Oberlandera. Centre for Speech Technology Researchb. Human Communication Research Centrec. Department of Artificial IntelligenceUniversity of EdinburghEdinburgh EH1 1HN, GBhttp://www.cstr.ed.ac.uk/projects/sole.html email: J.Hitzeman@ed.ac.ukABSTRACTThis paper describes the latest version of the SOLE concept-to-speech system, which uses linguistic information provided by a nat-ural language generation system to improve the prosody of syntheticspeech. We discuss the types of linguistic information that provemost useful and the implications for text-to-speech systems.1. INTRODUCTIONThe purpose of the SOLE project is to make use of automatically-generated, high-level linguistic information to improve the qualityof the intonation of synthetic speech. After choosing an initial set oflinguistic constructs thought to have some influence on prosody, wedeveloped an SGML-based mark-up language to serve as a generalinterface between NLG and speech synthesis systems, and trainedour synthesis system to recognise correlations between the mark-upand intonational contours so that it can make use of this mark-upwhen synthesising. As a result, many of the errors that the syn-thesiser makes with regard to knowing when to accent or deaccenta word are absent in the SOLE output. This paper reports on thecurrent results and discusses the implications for text-to-speech sys-tems in cases where it is realistic to use statistical methods for ex-ploiting certain types of high-level linguistic information.2. THE SOLE SYSTEMThe SOLE concept-to-speech system is designed to work as aportable museum guide: visitors to a museum carry a portable de-vice which detects what exhibits they are looking at and gives spo-ken explanation. SOLE generates its descriptions from a databaseof the museum exhibits’ properties. As it keeps a record of whatexhibits have already been visited, it is able to generate descriptionsof new exhibits with reference to previous ones. This gives rise to alarge number of discourse-level linguistic phenomena such as var-ious types of anaphoric reference (e.g., pronouns, definite descrip-tions, bridging references) and rhetorical relations (e.g., contrastingtwo exhibits or amplifying a particular property of an exhibit).The NLG component of SOLE was developed for the ILEXproject [5], and currently it is used for describing exhibits in theRoyal Museum of Scotland’s National Jewellery Gallery. The text-to-speech component is the Festival system. 11http://www.cstr.ed.ac.uk/projects/festival.htmlThe intonation component of Festival [4] works by using a decisiontree to analyse a set of features associated with a syllable, and todecide if a pitch accent should be assigned at that point. Typicalfeatures used include lexical stress and position in phrase etc. InSOLE, we now have access to the discourse-level information, andthis greatly enriches the feature set that the decision tree uses.3. METHODIn order to train the decision tree to use higher-level linguistic infor-mation in determining pitch accent placement, we needed a corpusconsisting of the types of descriptive texts that the ILEX systemproduces. At the time the SOLE project began, however, ILEX wasin an early stage of development, so, rather than using ILEX outputfor our corpus, we gathered a corpus of texts of the sort that ILEXwould be able to produce in its later stages. Our corpus consistsof 43 short descriptive texts, which gives us 35 minutes of speechand a total of 6331 syllables, 863 of which we set aside for test-ing. We annotated this corpus with linguistic information, whichinvolved deciding on an initial set of linguistic constructs that influ-ence prosody and that can be produced by ILEX, and developing aset of SGML tags to describe these constructs. We then recordedthree speakers reading these texts, and human labellers marked ac-cents on the speech by looking at the F0 contours.Given the tagged text, we were able to extract the linguistic infor-mation on a per-syllable basis and use it as a set of features to trainthe decision tree. The SOLE NLG component (i.e., the augmentedILEX system) automatically produces tagged text, which the traineddecision tree then uses in determining accent placement. In the sec-ond phase of the project we will annotate the corpus with Tilt pa-rameters [11] (accent duration, amplitude, peak position, etc.) andwe will also predict these values.Of the phenomena we chose to annotate in the first phase of theproject, noun phrases (with their syntactic, semantic and referencetype) and rhetorical structure gave the most significant results foraccent placement, so we will restrict our discussion here to theseconstructs.3.1. Linguistic annotationRhetorical relations. Rhetorical relations are discourse-level se-mantic relationships between segments of text. Some rhetoricalrelations, such as contrast and list, clearly have a correspondingintonational pattern; with others, such as definition and exemplifi-cation, the effect on intonation is not as obvious. Examples of a fewof the types of rhetorical relations we chose to annotate are below:(1) List: [Purple, white and green] were the colours of the suf-fragette movement.(2) Similarity: [[Like the necklace designed by Flockinger,] [thisitem is in the Organic style.]](3) Concession: [[This item is from the same period,] [but itdoesn’t have the same quality of workmanship.]]Each rhetorical structure can contain one or more rhet-emph tags,which mark the phrases within the text that express the properties orobjects being compared, contrasted, listed, etc. The following con-trastive rhetorical structure illustrates our SGML-based annotation:(4)   rhet-elem type=“contrast”   nucleus  The  rhet-emph type=“object”  god   /rhet-emph was  rhet-emph type=“property”  gilded  /rhet-emph  ;  /nucleus   nucleus  the  rhet-emph type=“object”  demon   /rhet-emph was  rhet-emph type=“property” stained in black ink and polished to a high sheen  /rhet-emph  .  /nucleus   /rhet-elem Because we are only concerned with predicting accent placement inthe first phase of the SOLE project, the rhetorical emphasis (rhet-emph) is the only relevant annotation; the rhetorical structure type,rhetorical emphasis type and the nuclei and satellites will be impor-tant when predicting tune in the next phase of the project.Noun phrases. It is well known that old information tends to bedeaccented and new information tends to be accented [1, 3]. Thefirst time an object is mentioned in a text it is part of the new infor-mation in that text, and all subsequent references to that object areconsidered references to old information, as illustrated in 5:(5) It was worn mainly by teenagers, to show that they were Bea-tles fans, or perhaps to show which of the Beatles they likedbest.The first time the NP Beatles is mentioned in the text, it is new andlikely to be accented; the subsequent reference to the Beatles refersto old information, and is unlikely to be accented.Making use of old and new information is becoming more commonin concept-to-speech systems (e.g., [9, 6, 8]). We chose a morecomplex annotation scheme for NPs, assigning them a referencetype, a syntactic type and an optional semantic type.In addition to annotating NPs as anaphors (old information) andfirst-mentions (new information), we used a third reference type,predicative, illustrated in 6:(6) This item is [a brooch].A predicative NP is one that generally occurs as the object of to be,giving a description of the subject.Among the syntactic types we assigned to NPs are the following: definite NP: Any NP using the definite determiner (the).– the brooch, the north-west portion of the coastline of theFirth of Forth bare-singular: A singular NP without a determiner.– jewellery, 1920, purple, solidarity N modifier: A noun that modifies the head noun in a noun-noun compound.– [costume] jewellery, a [dress]clip, an [Edinburgh] jew-ellerThe semantic types we chose to annotate are below: proper name: E.g., Jesse M. King, Scotland, the Middle Ages kind: An NP that describes a kind of object rather than aninstance of an object.– jewellery, people, the mass-produced variety of jewelrywhich was popular during the 1930sAn example of the annotation is in 7 (Note that the term anaphora-elem could be replaced with noun-phrase):(7) was  anaphora-elem ref-type=“predicative” syn-type=“indefinite-NP” an  anaphora-elem ref-type=“first-mention”syn-type=“N-modifier” sem-type=“PN” Edinburgh  /anaphora-elem jeweller  /anaphora-elem 4. RESULTSTable 1 gives a comparison of the number of errors made by theTTS system using the original set of features with the number oferrors made when the SOLE linguistic features were added to theset. Overall, the addition of linguistic features reduces the error inaccent prediction by 15.5%. The features in Table 1 show the largestcontribution to the error reduction.The first two features in Table 1 are purely syntactic indicators ofwhether a syllable is in an NP or an embedded NP. This simple clas-sification isn’t very useful, as shown by the small reduction in errorSyllable feature Total occurrences TTS errors TTS + SOLE errors % error reductionany syl in an NP 601 88 85 3.4any syl in embedded NP 213 33 29 12.1any syl in an anaphor 135 22 3 86.4last stressed syl in anaphor 41 12 9 25.0any syl in a first-mention 276 35 6 82.9last stressed syl in first-mention 54 11 6 45.5any syl in a predicative NP 69 15 5 66.7any syl in a definite NP 153 18 1 94.4any syl in a bare-singular 114 24 16 33.3any syl in an N-modifier 12 3 0 100.0any syl in a deictic NP 52 5 3 40.0any syl in a proper name 114 21 0 100.0first stressed syl in a proper name 37 10 0 100.0any syl in a kind 77 7 0 100.0any syl in rhet-emph 678 104 94 9.6last stressed syl in rhet-emph 46 12 5 58.3Table 1: A comparison of the TTS system with the SOLE systemfor both features. However, the next four features, which includeinformation concerning the reference type of the NP, show large re-ductions in error. As expected, anaphors tend to be deaccented andfirst-mentions to be accented. What is unexpected is that our resultscontradict the general claim in the literature that when a phrase isaccented the accent is placed at the end of that phrase [2, 7]: in ourdata, the feature indicating that a syllable is any (stressed) syllablein a first-mention proves more useful than the feature indicating thata syllable is the last stressed syllable in a first-mention.2The next eight features also give unexpected results; PredicativeNPs, definite NPs, bare singular NPs, N-modifiers, deictic NPs,proper names and kinds are not typically spoken of as indicatorsof accenting. With N-modifiers and deictic NPs, admittedly, thenumbers are small and it is therefore difficult to make a strong ar-gument that they will be reliable indicators of accenting in anothercorpus. However, for the other features the reduction in error islarge. Predicative NPs express new information, which explains theobservation that they tend to be accented; the discovery that an NPfollowing the verb “is” or “are” is likely to be accented has strongimplications for TTS systems with shallow statistical parsing mech-anisms. In contrast, definite NPs generally express old informa-tion because they refer to objects previously mentioned in the text;3again, the implication is that a TTS system could predict deaccent-ing on an NP beginning with the. Proper names, bare singular NPsand kinds can either express old or new information, so it is surpris-ing that they serve as indicators of accenting. Also surprising is thatit is useful to know whether a syllable is the first lexically stressedsyllable in a proper name.2Our decision tree always predicts that a syllable without lexical stressis deaccented, and the linguistic features are therefore only used in decidingwhether a stressed syllable is accented.3There are exceptions, such as definites that refer to objects in the com-mon ground, as in “[The weather] is lousy today”, and bridging references,which are definites that refer to an object closely related to a previously men-tioned object, as in “There is a house on the hill. [The door] is green.” Thereis no consensus on whether these constructs describe old or new information.The last two features concern rhetorical structure. Both using a fea-ture indicating whether a syllable is inside a rhet-emph tag andusing a feature indicating whether a particular syllable is the lastlexically stressed syllable in that tag gave a reduction in error. Thereduction in error is greater with the latter feature, indicating that inrhetorical emphases the accent tends to be placed at the rightmostportion of the emphasised phrase.The type of rhetorical structure had negligible effect on accentplacement, although, unsurprisingly, the phrases with rhetorical em-phasis had a tendency to be accented. We predict that becauserhetorical structures have tunes which are dependent on their type,their type will be important in the next phase of the project, whenwe train the system to recognise other features of an accent, such asduration and amplitude.5. DISCUSSIONThere are three central conclusions to be drawn from our results:1. In the domain of descriptive texts, certain types of high-levellinguistic information are useful in determining accent place-ment, and therefore coupling a natural language generationsystem with a speech synthesis system is a good idea;2. Surprisingly, kinds, bare-singular NPs and proper names aregood predictors of accents; and3. A TTS system with a statistical parsing mechanism wouldbenefit from singling out predicative NPs, definite NPs andbare-singular NPs (both proper names and kinds) because theyare easily recognised via statistical methods and are good pre-dictors of accent placement.Another goal of the SOLE project is to formalise the provision ofdiscourse-level information as a set of SGML tags as part of thestandardisation efforts of the SABLE consortium [10]. The inten-tion here is to design a powerful interface language between lan-guage generation and speech synthesis systems, so that the synthesissystems can produce high quality speech in a variety of applicationsand domains.6. AcknowledgementsSOLE is funded by the EPSRC, grant reference GR/L50341.7. REFERENCES1. Wallace L. Chafe. Language and consciousness. Language,50:111–133, 1974.2. N. Chomsky and M. Halle. The Sound Pattern of English. Harperand Row, New York, 1968.3. David Crystal. Prosodic features and linguistic theory, pages1–46. The English tone of voice: Essays in intonation, prosodyand paralanguage. Edward Arnold, London, 1975.4. K. Dusterhoff and A. W. Black. Generating F0 contours forspeech synthesis using the Tilt intonation theory. In Proceedingsof the ESCA Workshop on Intonation, Athens, Greece, 1997.5. Janet Hitzeman, Chris Mellish, and Jon Oberlander. Dynamicgeneration of museum web pages: The intelligent labelling ex-plorer. Archives and Museum Informatics, 11:107–115, 1997.Also presented at the Museums and the Web Conference, LosAngeles, March 1997.6. Laurie Hiyakumoto, Scott Prevost, and Justine Cassell. Seman-tic and discourse information for text-to-speech intonation. InKai Alter, Hannes Pirker, and Wolfgang Finkler, editors, Con-cept to Speech Generation Systems, pages 47–56, Madrid, Spain,July 1997. Proceedings of a Workshop Sponsored by the Asso-ciation for Computational Linguistics.7. R. Jackendoff. Semantic Interpretation in Generative Grammar.MIT Press, Cambridge, MA, 1972.8. Christine H. Nakatani. Computing Prosody, Integrating Prosodicand Discourse Modelling. Springer-Verlag, 1997.9. Scott Allan Prevost. A Semantics of Contrast and InformationStructure for Specifying Intonation in Spoken Language Gener-ation. PhD thesis, University of Pennsylvania, 1995.10. R. Sproat, A. Hunt, M. Ostendorf, P. Taylor, A. W. Black,K. Lenzo, and M. Edginton. SABLE: A standard for TTSmarkup. ICSLP98, this volume.11. P. Taylor. The Tilt Intonation Model. ICSLP98, this volume.

On the Use of Automatically Generated Discourse-Level Information in a Concept-to-Speech Synthesis System

https://www.pure.ed.ac.uk/ws/files/26502920/10.1.1.41.9503.pdf

On the Use of Automatically Generated Discourse-Level Information in a Concept-to-Speech Synthesis System

Abstract

Similar works

Full text

Available Versions

CiteSeerX

Edinburgh Research Explorer

Edinburgh Research Archive

CiteSeerX