This paper discusses the utterance generation module of a spoken dialogue system for transactions. Transactions are interesting because they involve obligations of both parties: the system should provide all relevant information; the user should feel committed to the transaction once it has been concluded. Utterance generation plays a major role in this. The utterance generation module works with prosodically annotated utterance templates. An appropriate template for a given dialogue act is selected by the following parameters: utterance type, body of the template, given information, wanted and new information. Templates respect rules of accenting and deaccenting

Hessen, Arjan van

Hulstijn, Joris

This paper discusses the utterance generation mod-ule of a spoken dialogue system for transactions. Transactions are interesting because they involve obligations of both parties: the system should pro-vide all relevant information; the user should feel committed to the transaction once it has been con-cluded. Utterance generation plays a major role in this. The utterance generation module works with prosodically annotated utterance templates. An ap-propriate template for a given dialogue act is selected by the following parameters: utterance type, body of the template, given information, wanted and new in-formation. Templates respect rules of accenting and deaccenting. 

Joris Hulstijn

Arjan Van Hessen

CiteSeerX

Utterance Generation for Transaction Dialogues

Hulstijn, J.

van Hessen, Adrianus J.

University of Twente Research Information

UTTERANCE GENERATION FOR TRANSACTIONDIALOGUESJoris Hulstijn and Arjan van HessenUniversity of TwentePO BOX   AE EnschedeThe NetherlandsfjorishessengcsutwentenlABSTRACTThis paper discusses the utterance generation module of a spoken dialogue system for transactionsTransactions are interesting because they involveobligations of both parties the system should provide all relevant information the user should feelcommitted to the transaction once it has been concluded Utterance generation plays a major role inthis The utterance generation module works withprosodically annotated utterance templates An appropriate template for a given dialogue act is selectedby the following parameters utterance type body ofthe template given information wanted and new information Templates respect rules of accenting anddeaccenting INTRODUCTIONGeneration of appropriate system utterances for spoken dialogue systems is not trivial especially whenthe dialogue involves transactions as in systems fordistant selling or ticket reservation Dialogues thatinvolve transactions display a more complex structure than mere inquiry or advisory dialogues Twotasks are executed in parallel obtaining informationabout the product and actually ordering the productJonsson 	 We have experimented with a dialogue system for theatre ticket reservation calledschisma Van der Hoeven et al 	 In ourWizardofOz corpus we also nd such complex behaviour users inquire browse and retract previouschoices for instance when tickets are too expensiveLegally transactions like these involve obligations forboth parties the system should provide all relevantinformation for the user to make a fair choice Theuser is bound by the transaction once it has beenconcludedCurrently we are implementing a spoken languageversion of schisma The quality of system promptshas a great impact on the coherence of dialoguesWrong wording or intonation often leads to misunderstanding Therefore the module must take thediscourse eects of wordorder and intonation intoaccount Dirksen 	 Van Deemter et al 	Vallduvi 	 Rats 	 One of the challengesis to build a system with a certain personality thatis reected in the nature of the prompts We hopeto create the impression that the system cares aboutthe transaction This will hopefully also make usersmore committedThe paper is structured as follows We start withan overview of the system In section  we explainthe parameters that control the utterance generationmodule and their relation to theories of focus Insection  we discuss the Fluent Dutch texttospeechpackage that is used to pronounce the templatesThe paper ends with conclusions and plans for further research SYSTEMThe system is mixed initiative There are two typesof interaction inquiry and transaction During inquiry the user has the initiative the system answersthe users questions When the user has indicatedthat he or she wants a transaction  a reservation inour case  the system takes initiative The systemchecks if all parameters needed for a transaction likethe name of the client are known If not the systemwill ask for them Our corpus shows that there areno separate phases in the dialogue and no particular order The inquiry and transaction functions ofutterances are intertwinedThe system consists of three modules in a pipeline architecture User utterances are processed by a speechrecognition module including a disambiguation module and a limited parser  When a keyboardbased In the earlier keyboardbased prototype there was acorpusbased parser preceded by a module for morphological analysis and error correctiondialogue system is adapted for speech one of themost dicult changes concerns the vericationmechanism Current speech recognition for this type of dialogue systems requires that correct recognition ofinformation is veried This can be done explicitlyby asking the user if some information was recognisedcorrectly or implicitly by incorporating the recognised information in the next prompt From userexperiments we learned that the verication strategy can be greatly improved using condence measures Bouwman and Hulstijn 	 Ruber 	When the condence measure exceeds a thresholdverication can be suppressed We also found thatit makes sense to take the cost of a domain relatedaction into account For a crucial action such as theclosure of a transaction information must be veriedexplicitly For actions such as responses to inquirieswhere recognition errors are less costly the eciencygain of suppressing verication prompts outweighsthe risk of misunderstanding Verication promptsare distinguished by a rising intonation indicatinguncertaintyThe module that decides what to do next is calledthe dialogue manager It maintains two datastructures a representation of the context and a representation of the plan the current domainrelatedaction that the system is trying to accomplishThe dialogue manager interprets the parser outputagainst the context It updates the context resolvespresuppositions and anaphora and applies inferencerules that constitute domain knowledge Based onthe parser output the context the plan and thedatabase the dialoguemanager selects a response action Actions are composed of basic database actionsand basic dialogue acts For example a reservationinvolves a plan with subactions that deal with performance selection the price and number of ticketsand a nal conrmation Planning and action selection are based on a set of principles called dialoguerulesFinally each dialogue act is put into words by a general purpose utterance generation module It determines the utterancestructure wording and prosodyof each system response General rules govern thetranslation of dialogue acts into the parameters thatcontrol the style of prompts A dierent set of rulesproduces a system with a dierent personality In anecient curt system pronouns and implicit objectsare preferred over denite NPs to express the givenitems and short answers are preferred over longeranswers that echo part of the questionwhq n verb on nd whword on st hatdecl n verb on nd no whword hatynq n verb on st subject on nd risingimp n verb on st no subject hatverif verication as decl risingtext longer text to be read punctshort short answer PPNPADV	 hatmeta eg thanks greetings yes no hatFigure  Basic Utterance Types PARAMETERSThe utterance generation module uses a list of utterance templates Templates contain gaps to belled with information items attributevalue pairslabelled with syntactic lexicaland phonetic featuresTemplates are selected on the basis of ve parameters utterance type the body of the template andpossible empty lists of information items that aremarked as given wanted and new The utterancetype and body determine the wordorder and mainintonation contour The given wanted and newslots as well as special features aect the actualwording and prosodyUtterance type The utterance type roughly corresponds to the mood of a sentence whether it isa declarative whquestion yesno question or imperative However many utterances do not consist of complete sentences Often they involve shortanswers commands or remarks consisting of singleVPs NPs or PPs Figure 	 shows a list of utterance types It is a modied version of the utterance types developed for classication of utterancesin the schisma corpus Andernach 	 The rstcolumn in gure 	 gives the main syntactic featuresoften the position of the subject and nite verb Thelast column gives the main intonation contour Theregular declarative intonation is the hat shape rising during the phrase and falling at the end Theregular interrogative intonation is characterised by asharp rising at the end When applied to an otherwise declarative sentence a rising intonation indicates uncertainty This is applied in vericationprompts For the reading of longer texts the system assumes a reading voice see below Withintext the intonation depends on punctuation marksOther utterance types may be added for applicationin multimodal interaction such as a table format forlisting performances on the screenBody This slot selects the body of the template thecontent that is not especially marked It correspondsto the tail in the linkfocustail trichotomy of Vallduvi	 Vallduvis link corresponds to the topic ofthe conversation and his focus corresponds to ournew In most templates the body is expressed bythe main verb Usually it is deaccented but important cue words like niet not or maar but get someextra stressGiven This slot contains information that is to bepresented as if it is given in the conversation or situation Linguistically this is usually reected by theuse of pronouns denite articles or demonstrativesFor most templates the given elements occupy thetopic position the rst position before the verbor the position just after the verb when some newelement is topicalised With respect to intonationgiven items are deaccented Very often the givenitems refer to the topic of the conversation Rats	 the object the conversation is currently aboutBut not always The topiccomment distinction doesnot align with the givennew or focusground distinctions Vallduvi 	 In fact in most utterancesthe global topic the performance or the reservationis not mentioned at all It is implicit In the templates the choice between pronouns denites and implicit objects is marked by features The decision ismade according to the dialogue rules that govern thetranslation of dialogue acts into the ve utteranceparametersNew This parameter contains something like the focus of the utterance The notion of focus is not welldened for questions and imperatives Therefore weprefer the older givennew distinction In most templates new elements are placed towards the end of theutterance The new eld is also used for items thatneed contrastive stress as in c that suggests thatMacbeth is sold out but no other performancesWanted This slot contains the type of informationthat is wanted from the user This is indicated bythe choice of whword eg why for reasons or by theNP in the whphrase eg which genre for genresExamples 	   show the mechanism of templateselection Only some of the features are shown Theparameters are listed as utterance type bodygiven wanted new Features and values are givenbetween brackets	 a metaattsorry noNee sorryNo sorryb verifdatetomorrowMorgenTomorrowPlease note that focus contrasts with ground TheAI notion focus of attention roughly corresponds to whatwe call topic a whq want youpolite thingWat wilt uWhat would you likeb whq want youpoliteperformanceprepto datetomorrowNaar welke voorstelling wilt u morgenTo which performance would you like to gotomorrow a declattsorry sold out itSorry maar het is uitverkochtSorry but it is sold outb decl sold out performanceimpltitleMacbeth	Macbeth is uitverkochtMacbeth is sold outc decl sold out  performancedeftitleMacbeth	De voorstelling MACBETH is uitverkochtThe performance MACBETH is sold out TEXTTOSPEECH ANDPROSODYFor pronouncing the utterance templates we use theFluent Dutch texttospeech system Dirksen andMenert 	 Fluent Dutch runs on top of theMBROLA diphone synthesiser Dutoit 	 Ituses a Dutch voice developed at the Utrecht institute of linguistics OTS Fluent Dutch operates atthree levels the grapheme level the phoneme leveland a low level representation of phones where thelength and pitch of sounds is represented For manywords the phonetic description is taken from lexical resources of Van Dale dictionaries Other phonetic information is derived by heuristic rules Wedesign prompts at the grapheme level It is possibleto manipulate prosody by adding punctuation at thegrapheme level by adding prosodic annotations atthe phoneme level or by directly manipulating thephone level The main operators for manipulationare the followingquestion Change the usual hatshaped intonationcontour into a rising contour This is used for verication prompts and yesno questionsquote Set a string of words apart from the rest ofthe utterance by small pauses and a lifted pitch levelThis is used for names and titlesaccent Calculate the intonation centre for the accented phrase The intonation centre is at the vowelthat would receive the highest pitch if it were pronounced in a normal declarative way Add 	  tothe pitch level Lengthen the vowel by   This isused for new items and important cue wordsdeaccent Calculate the intonation centre for thedeaccented phrase Level the pitch to the averageof the two neighbouring intonation centres This isused for given itemsWe can also adjust other speech parameters such asthe speaking rate and the general pitch level Thisinuences the general impression of the voice Forreading reviews and other text fragments the system assumes a reading voice characterised by a lowerspeaking rate and a reduced pitch range At the endof the transaction the system concludes with a cheerful wish Veel plezier met  title	 Have fun with title It sounds cheerful which is achieved by ahigher speaking rate a slightly lifted pitch level anda large pitch range CONCLUSIONSWe conclude that it is indeed possible to generateappropriate system utterances for transaction dialogues A coherent dialogue can be achieved amongother things by producing utterances that take thediscourse eect of intonation into account As a ruleinformation that is assumed to be given in the dialogue is deaccented expressed as a pronoun or evenleft out Given information is repeated whenever thesystem is not condent it was recognised correctlyby the speech recognition module Such vericationprompts are distinguished by a rising intonation Information that is to be presented as new is accentedQuoted expressions like artist names or titles of performances are set apart from the rest of the utterance For reading the texts and reviews that describethe content of performances the system assumes areading voiceWe expect to make the user feel committed by addingan extra handshake at the end of the transactionFor the user it is a last opportunity to change his orher mind It is also meant to show that the systemcares it will hopefully add to the personality of thesystem We hope to verify this claim by user centredevaluation studiesFurther Research User behaviour greatly dependson the quality and nature of system prompts Wewill experiment with dierent styles of prompts Inparticular we would like to compare a brief telegramstyle system with a system that uses longer politeutterancesThe schisma prototype embodies the articial agentKarin that lives in a virtual theatre Nijholt et al	 Visitors can walk around in a virtual copy ofour local music centre and meet other agents Theenvironment invites an exploratory browsing typeof interaction Karin is there to make ticket reservations and answer questions of visitors about performances and the theatre building Karin has anarticial talking face The advantage of embedding adialogue agent in a virtual environment is that newinteraction metaphors can be explored Karin willmake use of electronic leaets with photographs ofthe main actors and the theatre schedule in tableformat The virtual theatre allows users to previewthe stage from seats at dierent angles in the theatrehall Future research is therefore concerned with theinteraction of dialogue agents like Karin and a virtual environment In particular the balance betweenpresenting information by speech or by other graphical means needs to be investigatedREFERENCESAndernach T 	 A machine learning approachto the classication of dialogue utterances InProceedings of NEMLAP Bilkent TurkeyBouwman G and Hulstijn J 	 Dialogue redesign with reliabiity measures In Proceedingsof LREC Granada SpainVan Deemter K et al 	 Generation of spokenmonologues by means of templates In Proceedings of TWLT Twente The NetherlandsDirksen A 	 Accenting and deaccenting adeclarative approach In Proceedings of COLING NantesDirksen A and Menert L 	 Fluent Dutchtexttospeech Technical manual FluencySpeech TechnologyOTS UtrechtDutoit T 	 Highquality texttospeech synthesis An overview Electrical and electronicsengineering 		Jonsson A 	 Dialogue Management for Natural Language Interfaces PhD thesis LinkopingUniversityVan der Hoeven G F et al 	 Schisma anatural language accessible theatre informationand booking system In Proceedings of NLDBVersailles FranceNijholt A et al 	 Speech and language interaction in a virtual theatre In ProceedingsNLPIA Moncton CanadaRats M 	 Topic Management in InformationDialogues PhD thesis Katholieke UniversiteitBrabant The NetherlandsRuber B 	 Obtaining condence measuresfrom sentence probabilities In EUROSPEECH RhodesVallduvi E 	 The Informational ComponentPhD thesis University of Pennsylvania

English

http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.28.683

Utterance generation for transaction dialogues

Abstract

Similar works

Full text

Available Versions

CiteSeerX

University of Twente Research Information