The Prosody of Uncertainty for Spoken Dialogue Intelligent Tutoring Systems

The speech medium is more than an audio conveyance of word strings. It contains meta information about the content of the speech. The prosody of speech, pauses and intonation, adds an extra dimension of diagnostic information about the quality of a speaker\u27s answers, suggesting an important avenue of research for spoken dialogue tutoring systems. Tutoring systems that are sensitive to such cues may employ different tutoring strategies based on detected student uncertainty, and they may be able to perform more precise assessment of the area of student difficulty. However, properly identifying the cues can be challenging, typically requiring thousands of hand labeled utterances for training in machine learning. This study proposes and explores means of exploiting alternate automatically generated information, utterance correctness and the amount of practice a student has had, as indicators of student uncertainty. It finds correlations with various prosodic features and these automatic indicators and compares the result with a small set of annotated utterances, and finally demonstrates a Bayesian classifier based on correctness scores as class labels

Evaluating the effect of information presentation strategies on task success and user perceptions

Town of Cumberland Town Council Meeting March 27, 2006 is the complete packet for the Cumberland Town Council meeting for March 27, 2006, and includes the minutes of the Town Council meeting of March 13, 2006. Agenda items include: 06 – 040. To hear a presentation from the Cumberland Fire Department. 06 – 041. To hold public hearing to consider and act on a Victualer’s License for Suburban Little League for the period 4/06 through 6/06. 06 – 042. To hear a report from the Budget Committee Chairman and set public hearing dates of April 10 and April 24, 2006 to consider and act on the 2006-2007 Municipal Fiscal Year Budget. 06 – 043. To make annual appointments of Building, Plumbing, and Electrical Inspectors and Alternates and Code Enforcement Officer for Fiscal Year ‘07. 06 – 044. To consider and act on Clerk’s appointment to the Voter Registration Appeals Board. 06 – 045. To set Public Hearing date of April 10, 2006 to consider and act on a Mass Gathering Permit for the Cumberland Soccer Club Labor Day Tournament, 9/2 & 9/3, 2006. 06 – 046. To review the Contract Zone Agreement from the Chinese Gospel Church, 99 Gray Road, and refer same to Planning Board for its review and recommendation. 06 – 047. To set Public Hearing date of April 10, 2006 to consider and act on a Recycler License for Cumberland Salvage, Inc. for the period April, 2006 through April, 2011. 06 – 048. To set public hearing date to consider and act on amendments to the Traffic Ordinance to add the following stop sign locations: Oak Ridge at Greely Road Ext Bea Lane at Gross Street Hedgerow Drive at Route 9 Candlewick Lane at Country Charm Lane Country Charm Lane at Val Halla Road Hedgerow Drive at Val Halla Road Sparhawk Lane at Cottage Farms Road Prince Street at Farwell Ave (west) Prince Street at Farwell Ave (east) Spruce Lane at Route 88 06 – 049. To set May 1-5 as the dates for Bulky Waste Pick-Up on the Mainland and August 5 & 6 on Chebeague Island

Evaluating Information Presentation Strategies for Spoken Recommendations

We report the results of a Wizard-of-Oz (WoZ) study comparing two approaches to presenting information in a spoken dialogue system generating flight recommendations. We found that recommendations presented using the user-model based summarize and refine (UMSR) approach enable more efficient information retrieval than the data-driven summarize and refine (SR) approach. In addition, user ratings on four evaluation criteria showed a clear preference for recommendations based on the UMSR approach

Evaluating Information Presentation Strategies for Spoken Dialogue Systems

Institute for Communicating and Collaborative SystemsA common task for spoken dialogue systems (SDS) is to help users select a suitable option (e.g., flight, hotel, restaurant) from the set of options available. When the number of options is small, they can simply be presented sequentially. However, as the number of options increases, the system must have strategies for helping users browse the space of available options. In this thesis, I compare two approaches to information presentation in SDS: (1) the summarize and refine (SR) approach (Polifroni et al., 2003; Polifroni, 2008) in which the summaries are generated by clustering the options based on attributes that lead to the smallest number of clusters, and (2) the user-model based summarize and refine (UMSR) approach (Demberg, 2005; Demberg and Moore, 2006) which employs a user model to cluster options based on attributes that are relevant to the user and uses coherence markers (e.g., connectives, discourse cues, adverbials) to highlight the trade-offs among the presented items. Prior work has shown that users prefer approaches to information presentation that take the user’s preferences into account (e.g., Komatani et al., 2003;Walker et al., 2004; Demberg and Moore, 2006). However, due to the complexity of building a working end-to-end SDS, these studies employed an ”overhearer” evaluation methodology, in which participants read or listened to pre-prepared dialogues, thus limiting evaluation criteria to users’ perceptions (e.g., informativeness, overview of options, and so on). In order to examine whether users prefer presentations based on UMSR when they were actively interacting with a dialogue system, and to measure the effectiveness and efficiency of the two approaches, I compared them in a Wizard-of-Oz experiment. I found that in terms of both task success and dialogue efficiency the UMSR approach was superior to the SR approach. In addition, I found that users also preferred presentations based on UMSR in the interactive mode. SDS are typically developed for situations in which the user’s hands and eyes are busy. I hypothesized that the benefits of pointing out relationships among options (i.e., trade-offs) in information presentation messages outweighs the costs of processing more complex sentences. To test this hypothesis, I performed two dual task experiments comparing the two approaches to information presentation in terms of their effect on cognitive load. Again, participants performed better with presentations based on the UMSR algorithm in terms of both dialogue efficiency and task success, and I found no detrimental effect on performance of the primary task. Finally, I hypothesized that one of the main reasons why UMSR is more efficient is because it uses coherence markers to highlight relations (e.g., trade-offs) between options and attributes. To test this hypothesis, I performed an eye-tracking experiment in which participants read presentations with and without these linguistic devices, and answered evaluation and comparison questions to measure differences in item recall. In addition, I used reading times to examine comprehension differences between the two information presentation strategies. I found that the linguistic devices used in UMSR indeed facilitated item recall, with no penalty in terms of comprehension cost. Thus, in this thesis I showed that an approach to information presentation that employs a user model and uses linguistic devices such as coherence markers to highlight trade-offs among the presented items improves information browsing. User studies demonstrated that this finding also applies to situations where users are performing another demanding task simultaneously

See What I’m Saying? Comparing Intelligent Personal Assistant Use for Native and Non-Native Language Speakers

Limited linguistic coverage for Intelligent Personal Assistants (IPAs) means that many interact in a non-native language. Yet we know little about how IPAs currently support or hinder these users. Through native (L1) and non-native (L2) English speakers interacting with Google Assistant on a smartphone and smart speaker, we aim to understand this more deeply. Interviews revealed that L2 speakers prioritised utterance planning around perceived linguistic limitations, as opposed to L1 speakers prioritising succinctness because of system limitations. L2 speakers see IPAs as insensitive to linguistic needs resulting in failed interaction. L2 speakers clearly preferred using smartphones, as visual feedback supported diagnoses of communication breakdowns whilst allowing time to process query results. Conversely, L1 speakers preferred smart speakers, with audio feedback being seen as sufficient. We discuss the need to tailor the IPA experience for L2 users, emphasising visual feedback whilst reducing the burden of language production