What Makes a Good Conversation? Challenges in Designing Truly Conversational Agents

Conversational agents promise conversational interaction but fail to deliver. Efforts often emulate functional rules from human speech, without considering key characteristics that conversation must encapsulate. Given its potential in supporting long-term human-agent relationships, it is paramount that HCI focuses efforts on delivering this promise. We aim to understand what people value in conversation and how this should manifest in agents. Findings from a series of semi-structured interviews show people make a clear dichotomy between social and functional roles of conversation, emphasising the long-term dynamics of bond and trust along with the importance of context and relationship stage in the types of conversations they have. People fundamentally questioned the need for bond and common ground in agent communication, shifting to more utilitarian definitions of conversational qualities. Drawing on these findings we discuss key challenges for conversational agent design, most notably the need to redefine the design parameters for conversational agent interaction

A Virtual Conversational Agent for Teens with Autism: Experimental Results and Design Lessons

We present the design of an online social skills development interface for teenagers with autism spectrum disorder (ASD). The interface is intended to enable private conversation practice anywhere, anytime using a web-browser. Users converse informally with a virtual agent, receiving feedback on nonverbal cues in real-time, and summary feedback. The prototype was developed in consultation with an expert UX designer, two psychologists, and a pediatrician. Using the data from 47 individuals, feedback and dialogue generation were automated using a hidden Markov model and a schema-driven dialogue manager capable of handling multi-topic conversations. We conducted a study with nine high-functioning ASD teenagers. Through a thematic analysis of post-experiment interviews, identified several key design considerations, notably: 1) Users should be fully briefed at the outset about the purpose and limitations of the system, to avoid unrealistic expectations. 2) An interface should incorporate positive acknowledgment of behavior change. 3) Realistic appearance of a virtual agent and responsiveness are important in engaging users. 4) Conversation personalization, for instance in prompting laconic users for more input and reciprocal questions, would help the teenagers engage for longer terms and increase the system's utility

THE FRIENDLY CHATBOT: REVEALING WHY PEOPLE USE CHATBOTS THROUGH A STUDY OF USER EXPERIENCE OF CONVERSATIONAL AGENTS

Chatbots are becoming increasingly popular. However, little is known about the way chatbots should be designed. Whether the users should be informed or not beforehand that they are chatting with a chatbot is an open question. Similarly, questions related to the level of ‘humanistic’ tonality in interactions with chatbots are unanswered. In this paper, we present a controlled experiment in which 40 individuals participated. Their user experience was compared depending on whether they knew that they were chatting with chatbots before or afterwards. Two different versions of chatbots were tested (one with mechanical tonality and one with humanistic tonality). Our findings illustrate that: i) it is vital that the users enter the conversation knowing that they are chatting with a chatbot; ii) tonality matters, the way chatbots are designed is pivotal for the user experience, the ‘human-like’ and friendly chatbot was preferred over the mechanical, task-oriented chatbot

Trends, challenges and processes in conversational agent design: exploring practitioners’ views through semi-structured interviews

The aim of this study is to explore the challenges and experiences of conversational agent (CA) practitioners in order to highlight their practical needs and bring them into consideration within the scholarly sphere. A range of data scientists, conversational designers, executive managers and researchers shared their opinions and experiences through semi-structured interviews. They were asked about emerging trends, the challenges they face, and the design processes they follow when creating CAs. In terms of trends, findings included mixed feelings regarding no-code solutions and a desire for a separation of roles. The challenges mentioned included a lack of socio-technical tools and conversational archetypes. Finally, practitioners followed different design processes and did not use the design processes described in the academic literature. These findings were analyzed to establish links between practitioners’ insights and discussions in related literature. The goal of this analysis is to highlight research-practice gaps by synthesising five practitioner needs that are not currently being met. By highlighting these research-practice gaps and foregrounding the challenges and experiences of CA practitioners, we can begin to understand the extent to which emerging literature is influencing industrial settings and where more research is needed to better support CA practitioners in their work

Conversational Agents for Mental Health and Well-being: Discovering Design Recommendations Using Text Mining

Conversational agents are increasingly being used by the general population due to shortages in healthcare providers and specialists, and limited access to treatments. They are also used by people to deal with loneliness and lack of companionship. As these apps are increasingly replacing real humans, there is a need to explore their design features and limitations for better design of conversational apps. Using text mining and topic modeling, this study analyzed a total of 126,610 reviews about Replika, a popular and well-established conversational agent mobile app. Our results emphasized current practices for designing conversational apps while at the same time sheds the light on limitations associated with these apps. Such limitations are related to the need for better conversations and intelligent responses, the need for advanced AI chatbots, the need to avoid questionable and inappropriate content, the need for inclusive design, and the need to address some technical limitations

Designing Women: Essentializing Femininity in AI Linguistics

Since the eighties, feminists have considered technology a force capable of subverting sexism because of technology’s ability to produce unbiased logic. Most famously, Donna Haraway’s “A Cyborg Manifesto” posits that the cyborg has the inherent capability to transcend gender because of its removal from social construct and lack of loyalty to the natural world. But while humanoids and artificial intelligence have been imagined as inherently subversive to gender, current artificial intelligence perpetuates gender divides in labor and language as their programmers imbue them with traits considered “feminine.” A majority of 21st century AI and humanoids are programmed to fit female stereotypes as they fulfill emotional labor and perform pink-collar tasks, whether through roles as therapists, query-fillers, or companions. This paper examines four specific chat-based AI --ELIZA, XiaoIce, Sophia, and Erica-- and examines how their feminine linguistic patterns are used to maintain the illusion of emotional understanding in regards to the tasks that they perform. Overall, chat-based AI fails to subvert gender roles, as feminine AI are relegated to the realm of emotional intelligence and labor

Sensemaking on the Pragmatic Web: A Hypermedia Discourse Perspective

The complexity of the dilemmas we face on an organizational, societal and global scale forces us into sensemaking activity. We need tools for expressing and contesting perspectives flexible enough for real time use in meetings, structured enough to help manage longer term memory, and powerful enough to filter the complexity of extended deliberation and debate on an organizational or global scale. This has been the motivation for a programme of basic and applied action research into Hypermedia Discourse, which draws on research in hypertext, information visualization, argumentation, modelling, and meeting facilitation. This paper proposes that this strand of work shares a key principle behind the Pragmatic Web concept, namely, the need to take seriously diverse perspectives and the processes of meaning negotiation. Moreover, it is argued that the hypermedia discourse tools described instantiate this principle in practical tools which permit end-user control over modelling approaches in the absence of consensus

LLM-Powered Conversational Voice Assistants: Interaction Patterns, Opportunities, Challenges, and Design Guidelines

Conventional Voice Assistants (VAs) rely on traditional language models to discern user intent and respond to their queries, leading to interactions that often lack a broader contextual understanding, an area in which Large Language Models (LLMs) excel. However, current LLMs are largely designed for text-based interactions, thus making it unclear how user interactions will evolve if their modality is changed to voice. In this work, we investigate whether LLMs can enrich VA interactions via an exploratory study with participants (N=20) using a ChatGPT-powered VA for three scenarios (medical self-diagnosis, creative planning, and debate) with varied constraints, stakes, and objectivity. We observe that LLM-powered VA elicits richer interaction patterns that vary across tasks, showing its versatility. Notably, LLMs absorb the majority of VA intent recognition failures. We additionally discuss the potential of harnessing LLMs for more resilient and fluid user-VA interactions and provide design guidelines for tailoring LLMs for voice assistance

How to Support Domestic Violence Survivors with Conversational Agents: Meta Requirements and Design Principles

Domestic violence is a prevalent and complicated issue that can have detrimental effects on the survivors, their families, and communities. Survivors are often reluctant to divulge their experiences to others in person for social, emotional, privacy, or cultural reasons. Consequently, many are not actively seeking support that meets their needs. Conversational agents, a form of technology support, hold great promise for facilitating counseling and support by promoting self-disclosure and enhancing user engagement. To address the knowledge gaps in design principles for conversational agents for DV survivors, we conducted in-depth interviews with 11 professionals working with domestic violence survivors. After analyzing the interview transcripts and related literature, we identified several meta-requirements and categorized them into four categories —conversation, language, support, and trust. We further grouped these meta-requirements into several design principles. Our work lays the foundation for design science research in designing and developing conversational agents to support domestic violence survivors