5 research outputs found
Transparency in Language Generation: Levels of Automation
Language models and conversational systems are growing increasingly advanced,
creating outputs that may be mistaken for humans. Consumers may thus be misled
by advertising, media reports, or vagueness regarding the role of automation in
the production of language. We propose a taxonomy of language automation, based
on the SAE levels of driving automation, to establish a shared set of terms for
describing automated language. It is our hope that the proposed taxonomy can
increase transparency in this rapidly advancing field.Comment: Accepted for publication at CUI 202
What's in an accent? The impact of accented synthetic speech on lexical choice in human-machine dialogue
The assumptions we make about a dialogue partner's knowledge and
communicative ability (i.e. our partner models) can influence our language
choices. Although similar processes may operate in human-machine dialogue, the
role of design in shaping these models, and their subsequent effects on
interaction are not clearly understood. Focusing on synthesis design, we
conduct a referential communication experiment to identify the impact of
accented speech on lexical choice. In particular, we focus on whether accented
speech may encourage the use of lexical alternatives that are relevant to a
partner's accent, and how this is may vary when in dialogue with a human or
machine. We find that people are more likely to use American English terms when
speaking with a US accented partner than an Irish accented partner in both
human and machine conditions. This lends support to the proposal that synthesis
design can influence partner perception of lexical knowledge, which in turn
guide user's lexical choices. We discuss the findings with relation to the
nature and dynamics of partner models in human machine dialogue.Comment: In press, accepted at 1st International Conference on Conversational
User Interfaces (CUI 2019
See What I’m Saying? Comparing Intelligent Personal Assistant Use for Native and Non-Native Language Speakers
Limited linguistic coverage for Intelligent Personal Assistants (IPAs) means that many interact in a non-native language. Yet we know little about how IPAs currently support or hinder these users. Through native (L1) and non-native (L2) English speakers interacting with Google Assistant on a smartphone and smart speaker, we aim to understand this more deeply. Interviews revealed that L2 speakers prioritised utterance planning around perceived linguistic limitations, as opposed to L1 speakers prioritising succinctness because of system limitations. L2 speakers see IPAs as insensitive to linguistic needs resulting in failed interaction. L2 speakers clearly preferred using smartphones, as visual feedback supported diagnoses of communication breakdowns whilst allowing time to process query results. Conversely, L1 speakers preferred smart speakers, with audio feedback being seen as sufficient. We discuss the need to tailor the IPA experience for L2 users, emphasising visual feedback whilst reducing the burden of language production
The Partner Modelling Questionnaire: A validated self-report measure of perceptions toward machines as dialogue partners
Recent work has looked to understand user perceptions of speech agent
capabilities as dialogue partners (termed partner models), and how this affects
user interaction. Yet, currently partner model effects are inferred from
language production as no metrics are available to quantify these subjective
perceptions more directly. Through three studies, we develop and validate the
Partner Modelling Questionnaire (PMQ): an 18-item self-report semantic
differential scale designed to reliably measure people's partner models of
non-embodied speech interfaces. Through principal component analysis and
confirmatory factor analysis, we show that the PMQ scale consists of three
factors: communicative competence and dependability, human-likeness in
communication, and communicative flexibility. Our studies show that the measure
consistently demonstrates good internal reliability, strong test-retest
reliability over 12 and 4-week intervals, and predictable convergent/divergent
validity. Based on our findings we discuss the multidimensional nature of
partner models, whilst identifying key future research avenues that the
development of the PMQ facilitates. Notably, this includes the need to identify
the activation, sensitivity, and dynamism of partner models in speech interface
interaction.Comment: Submitted (TOCHI