A Framework of Personality Cues for Conversational Agents

Conversational agents (CAs)—software systems emulating conversations with humans through natural language—reshape our communication environment. As CAs have been widely used for applications requiring human-like interactions, a key goal in information systems (IS) research and practice is to be able to create CAs that exhibit a particular personality. However, existing research on CA personality is scattered across different fields and researchers and practitioners face difficulty in understanding the current state of the art on the design of CA personality. To address this gap, we systematically analyze existing studies and develop a framework on how to imbue CAs with personality cues and how to organize the underlying range of expressive variation regarding the Big Five personality traits. Our framework contributes to IS research by providing an overview of CA personality cues in verbal and non-verbal language and supports practitioners in designing CAs with a particular personality

Fillers in Spoken Language Understanding: Computational and Psycholinguistic Perspectives

Disfluencies (i.e. interruptions in the regular flow of speech), are ubiquitous to spoken discourse. Fillers ("uh", "um") are disfluencies that occur the most frequently compared to other kinds of disfluencies. Yet, to the best of our knowledge, there isn't a resource that brings together the research perspectives influencing Spoken Language Understanding (SLU) on these speech events. This aim of this article is to synthesise a breadth of perspectives in a holistic way; i.e. from considering underlying (psycho)linguistic theory, to their annotation and consideration in Automatic Speech Recognition (ASR) and SLU systems, to lastly, their study from a generation standpoint. This article aims to present the perspectives in an approachable way to the SLU and Conversational AI community, and discuss moving forward, what we believe are the trends and challenges in each area.Comment: To appear in TAL Journa

Alexa as an Active Listener: How Backchanneling Can Elicit Self-Disclosure and Promote User Experience

Active listening is a well-known skill applied in human communication to build intimacy and elicit self-disclosure to support a wide variety of cooperative tasks. When applied to conversational UIs, active listening from machines can also elicit greater self-disclosure by signaling to the users that they are being heard, which can have positive outcomes. However, it takes considerable engineering effort and training to embed active listening skills in machines at scale, given the need to personalize active-listening cues to individual users and their specific utterances. A more generic solution is needed given the increasing use of conversational agents, especially by the growing number of socially isolated individuals. With this in mind, we developed an Amazon Alexa skill that provides privacy-preserving and pseudo-random backchanneling to indicate active listening. User study (N = 40) data show that backchanneling improves perceived degree of active listening by smart speakers. It also results in more emotional disclosure, with participants using more positive words. Perception of smart speakers as active listeners is positively associated with perceived emotional support. Interview data corroborate the feasibility of using smart speakers to provide emotional support. These findings have important implications for smart speaker interaction design in several domains of cooperative work and social computing.Comment: To appear in Proceedings of the ACM on Human-Computer Interaction (PACM HCI). The paper will be presented in CSCW 2022 (https://cscw.acm.org/2022

LLM-Powered Conversational Voice Assistants: Interaction Patterns, Opportunities, Challenges, and Design Guidelines

Conventional Voice Assistants (VAs) rely on traditional language models to discern user intent and respond to their queries, leading to interactions that often lack a broader contextual understanding, an area in which Large Language Models (LLMs) excel. However, current LLMs are largely designed for text-based interactions, thus making it unclear how user interactions will evolve if their modality is changed to voice. In this work, we investigate whether LLMs can enrich VA interactions via an exploratory study with participants (N=20) using a ChatGPT-powered VA for three scenarios (medical self-diagnosis, creative planning, and debate) with varied constraints, stakes, and objectivity. We observe that LLM-powered VA elicits richer interaction patterns that vary across tasks, showing its versatility. Notably, LLMs absorb the majority of VA intent recognition failures. We additionally discuss the potential of harnessing LLMs for more resilient and fluid user-VA interactions and provide design guidelines for tailoring LLMs for voice assistance

Affective Expressions in Conversational Agents for Learning Environments: Effects of curiosity, humour, and expressive auditory gestures

Conversational agents -- systems that imitate natural language discourse -- are becoming an increasingly prevalent human-computer interface, being employed in various domains including healthcare, customer service, and education. In education, conversational agents, also known as pedagogical agents, can be used to encourage interaction; which is considered crucial for the learning process. Though pedagogical agents have been designed for learners of diverse age groups and subject matter, they retain the overarching goal of eliciting learning outcomes, which can be broken down into cognitive, skill-based, and affective outcomes. Motivation is a particularly important affective outcome, as it can influence what, when, and how we learn. Understanding, supporting, and designing for motivation is therefore of great importance for the advancement of learning technologies. This thesis investigates how pedagogical agents can promote motivation in learners. Prior research has explored various features of the design of pedagogical agents and what effects they have on learning outcomes, and suggests that agents using social cues can adapt the learning environment to enhance both affective and cognitive outcomes. One social cue that is suggested to be of importance for enhancing learner motivation is the expression or simulation of affect in the agent. Informed by research and theory across multiple domains, three affective expressions are investigated: curiosity, humour, and expressive auditory gestures -- each aimed at enhancing motivation by adapting the learning environment in different ways, i.e., eliciting contagion effects, creating a positive learning experience, and strengthening the learner-agent relationship, respectively. Three studies are presented in which each expression was implemented in a separate type of agent: physically-embodied, text-based, and voice-based; with all agents taking on the role of a companion or less knowledgeable peer to the learner. The overall focus is on how each expression can be displayed, what the effects are on perception of the agent, and how it influences behaviour and learning outcomes. The studies result in theoretical contributions that add to our understanding of conversational agent design for learning environments. The findings provide support for: the simulation of curiosity, the use of certain humour styles, and the addition of expressive auditory gestures, in enhancing motivation in learners interacting with conversational agents; as well as indicating a need for further exploration of these strategies in future work

Hesitations in Spoken Dialogue Systems

Betz S. Hesitations in Spoken Dialogue Systems. Bielefeld: Universität Bielefeld; 2020

STAY WITH ME - CONVERSATIONAL CHURN PREVENTION IN DIGITAL SUBSCRIPTION SERVICE

Lots of organizations use subscription business models. However, with increasing competition and technological progress switching costs for customers are decreasing. This development can translate to serious issues for subscription-based businesses, requiring action. Traditionally, businesses used mailings or calls, which are costly, time-consuming and often not effective. In this research-in-progress paper, we explore conversational churn prevention as a potential remedy. We present a conversational agent with persuasive design features (e.g., nudges) and first results from a pre-study. We conduct an in-between subject experiment and interviews for our mixed-methods evaluation of our pre-study. Our work contributes to theory, by presenting more insights into the interaction quality of conversational agents in the context of churn prevention of digital services and the role of persuasive design. We support practitioners, by guiding them towards more effective use of conversational agents to improve their services and to predict churn

Lessons from Oz: Design Guidelines for Automotive Conversational User Interfaces

This paper draws from literature and our experience of conducting Wizard-of-Oz (WoZ) studies using natural language, conversational user interfaces (CUIs) in the automotive domain. These studies have revealed positive effects of using in-vehicle CUIs on issues such as: cognitive demand/workload, passive task-related fatigue, trust, acceptance and environment engagement. A nascent set of human-centred design guidelines that have emerged is presented. These are based on the analysis of users' behaviour and the positive benefits observed, and aim to make interactions with an in-vehicle agent interlocutor safe, effective, engaging and enjoyable, while confirming with users' expectations. The guidelines can be used to inform the design of future in-vehicle CUIs or applied experimentally using WoZ methodology, and will be evaluated and refined in ongoing work.Comment: Accepted to the 11th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI '19