An End-to-End Conversational Style Matching Agent

We present an end-to-end voice-based conversational agent that is able to engage in naturalistic multi-turn dialogue and align with the interlocutor's conversational style. The system uses a series of deep neural network components for speech recognition, dialogue generation, prosodic analysis and speech synthesis to generate language and prosodic expression with qualities that match those of the user. We conducted a user study (N=30) in which participants talked with the agent for 15 to 20 minutes, resulting in over 8 hours of natural interaction data. Users with high consideration conversational styles reported the agent to be more trustworthy when it matched their conversational style. Whereas, users with high involvement conversational styles were indifferent. Finally, we provide design guidelines for multi-turn dialogue interactions using conversational style adaptation

How the agent’s gender influence users’ evaluation of a QA system

In this paper we present the results of a pilot study investigating the effects of agents’ gender-ambiguous vs. gender-marked look on the perceived interaction quality of a multimodal question answering system. Eight test subjects interacted with three system agents, each having a feminine, masculine or gender-ambiguous look. The subjects were told each agent was representing a differently configured system. In fact, they were interacting with the same system. In the end, the subjects filled in an evaluation questionnaire and participated in an in-depth qualitative interview. The results showed that the user evaluation seemed to be influenced by the agent’s gender look: the system represented by the feminine agent achieved on average the highest evaluation scores. On the other hand, the system represented by the gender-ambiguous agent was systematically lower rated. This outcome might be relevant for an appropriate agent look, especially since many designers tend to develop gender-ambiguous characters for interactive interfaces to match various users’ preferences. However, additional empirical evidence is needed in the future to confirm our findings

Design Knowledge for Virtual Learning Companions from a Value-centered Perspective

The increasing popularity of conversational agents such as ChatGPT has sparked interest in their potential use in educational contexts but undermines the role of companionship in learning with these tools. Our study targets the design of virtual learning companions (VLCs), focusing on bonding relationships for collaborative learning while facilitating students’ time management and motivation. We draw upon design science research (DSR) to derive prescriptive design knowledge for VLCs as the core of our contribution. Through three DSR cycles, we conducted interviews with working students and experts, held interdisciplinary workshops with the target group, designed and evaluated two conceptual prototypes, and fully coded a VLC instantiation, which we tested with students in class. Our approach has yielded 9 design principles, 28 meta-requirements, and 33 design features centered around the value-in-interaction. These encompass Human-likeness and Dialogue Management, Proactive and Reactive Behavior, and Relationship Building on the Relationship Layer (DP1,3,4), Adaptation (DP2) on the Matching Layer, as well as Provision of Supportive Content, Fostering Learning Competencies, Motivational Environment, and Ethical Responsibility (DP5-8) on the Service Layer

인공지능과 대화하기: 일대일 그리고 그룹 상용작용을 위한 대화형 에이전트 시스템 개발

학위논문(박사) -- 서울대학교대학원 : 사회과학대학 언론정보학과, 2022.2. 이준환."인간-컴퓨터 상호작용"과 "사용자 경험"을 넘어, "인간-인공지능 상호작용" 그리고 "알고리즘 경험"의 시대가 도래하고 있다. 기술의 발전은 우리가 의사소통하고 협업하는 방식의 패러다임을 전환했다. 기계 에이전트는 인간 커뮤니케이션에서 적극적이며 주도적인 역할을 수행한다. 하지만 효과적인 AI 기반 커뮤니케이션과 토론 시스템 디자인에 대한 이해와 논의는 부족한 것이 사실이다. 이에 본 연구는 인간-컴퓨터 상호작용의 관점에서 다양한 형태의 커뮤니케이션을 지원할 수 있는 기술적 방법을 탐색하는 것을 목표로 한다. 이를 위해 저자는 일대일 그리고 그룹 상호작용을 지원하는 대화형 에이전트를 제시한다. 구체적으로 본 연구는 1) 일대일 상호작요에서 사용자 관여를 높이는 대화형 에이전트, 2) 일상적인 소셜 그룹 토론을 지원하는 에이전트, 3) 숙의 토론을 가능하게 하는 에이전트를 디자인 및 개발하고 그 효과를 정량적 그리고 정성적으로 검증했다. 시스템을 디자인함에 있어서 인간-컴퓨터 상호작용뿐 아니라, 커뮤니케이션학, 심리학, 그리고 데이터 과학을 접목한 다학제적 접근 방식이 적용되었다. 첫 번째 연구는 일대일 상호작용 상황에서 사용자의 관여 증진을 위한 대화형 에이전트의 효과를 검증했다. 설문조사라는 맥락에서 수행된 이 연구는 웹 설문조사에서 응답자의 불성실로 인해 발생하는 응답 데이터 품질의 문제를 극복하기 위한 새로운 인터랙션 방법으로 텍스트 기반 대화형 에이전트의 가능성을 탐색하는 것을 목표로 했다. 이를 위해 2 (인터페이스: 웹 對 챗봇) X 2 (대화 스타일: 포멀 對 캐쥬얼) 실험을 진행했으며, 만족화 이론에 근거하여 응답 데이터의 품질을 평가했다. 그 결과, 챗봇 설문조사의 참여자가 웹 설문조사의 참여자보다 더 높은 수준의 관여를 보이고, 결과적으로 더 높은 품질의 데이터를 생성하는 것을 확인할 수 있었다. 하지만 이런 챗봇의 데이터 품질에 대한 효과는 챗봇이 친구 같고 캐쥬얼한 대화체를 사용할 때만 나타났다. 이 결과는 대화형 인터랙티비티가 인터페이스뿐 아니라 대화 스타일이라는 효과적인 메세지 전략을 동반할 때 발생하는 것을 의미한다. 두 번째 연구는 일상적인 소셜 채팅 그룹에서 집단의 의사결정과정과 토론을 지원하는 대화형 시스템에 대한 것이다. 이를 위해 GroupfeedBot이라는 대화형 에이전트를 제작하였으며, GroupfeedBot은 (1) 토론 시간을 관리하고, (2) 구성원들의 균등한 참여를 촉진하며, (3) 구성원들의 다양한 의견을 요약 및 조직화하는 기능을 갖고 있다. 해당 에이전트를 평가하기 위해 다양한 태스크 (추론, 의사결정, 자유 토론, 문제 해결 과제)와 그룹 규모(소규모, 중규모)에 관하여 사용자 조사를 시행했다. 그 결과 의견의 다양성 측면에서 GroupfeedBot으로 토론한 집단이 기본 에이전트와 토론한 집단보다 더 다양한 의견을 생성했지만 산출된 결과의 품질과 메시지 양에 있어서는 차이가 없는 것을 확인할 수 있었다. 균등한 참여에 대한 GroupfeedBot의 효과는 태스크의 특성에 따라 다르게 나타났는데, 특히 자유 토론 과제에서 GroupfeedBot이 참여자들의 균등한 참여를 촉진했다. 세 번째 연구는 숙의 토론을 지원하는 대화형 시스템에 대한 것이다. 세 번째 연구에서 개발된 DebateBot은 GroupfeeedBot과 달리 더 진지한 사회적 맥락에서 적용되었다. DebateBot은 (1) 생각하기-짝짓기-공유하기 (Think-Pair-Share) 전략에 따라 토론을 구조화하고, (2) 과묵한 토론자에게 의견을 요청함으로써 동등한 참여를 촉진하는 두 가지 주요 기능을 수행했다. 사용자 평가 결과 DebateBot은 그룹 상호작용을 개선함으로써 심의 토론을 가능하게 했다. 토론 구조화는 토론의 질에 긍정적인 효과를 발휘하였고, 참여자 촉진은 진정한 합의 도달에 기여하였으며, 그룹 구성원들의 주관적 만족도를 향상했다. 본 연구는 이 세 가지 연구의 결과들을 바탕으로 인간-인공지능 커뮤니케이션에 대한 다양한 시사점들을 도출하였으며, 이를 TAMED (Task-Agent-Message-Information Exchange-Relationship Dynamics) 모델로 정리하였다.The advancements in technology shift the paradigm of how individuals communicate and collaborate. Machines play an active role in human communication. However, we still lack a generalized understanding of how exactly to design effective machine-driven communication and discussion systems. How should machine agents be designed differently when interacting with a single user as opposed to when interacting with multiple users? How can machine agents be designed to drive user engagement during dyadic interaction? What roles can machine agents perform for the sake of group interaction contexts? How should technology be implemented in support of the group decision-making process and to promote group dynamics? What are the design and technical issues which should be considered for the sake of creating human-centered interactive systems? In this thesis, I present new interactive systems in the form of a conversational agent, or a chatbot, that facilitate dyadic and group interactions. Specifically, I focus on: 1) a conversational agent to engage users in dyadic communication, 2) a chatbot called GroupfeedBot that facilitates daily social group discussion, 3) a chatbot called DebateBot that enables deliberative discussion. My approach to research is multidisciplinary and informed by not only in HCI, but also communication, psychology and data science. In my work, I conduct in-depth qualitative inquiry and quantitative data analysis towards understanding issues that users have with current systems, before developing new computational techniques that meet those user needs. Finally, I design, build, and deploy systems that use these techniques to the public in order to achieve real-world impact and to study their use by different usage contexts. The findings of this thesis are as follows. For a dyadic interaction, participants interacting with a chatbot system were more engaged as compared to those with a static web system. However, the conversational agent leads to better user engagement only when the messages apply a friendly, human-like conversational style. These results imply that the chatbot interface itself is not quite sufficient for the purpose of conveying conversational interactivity. Messages should also be carefully designed to convey such. Unlike dyadic interactions, which focus on message characteristics, other elements of the interaction should be considered when designing agents for group communication. In terms of messages, it is important to synthesize and organize information given that countless messages are exchanged simultaneously. In terms of relationship dynamics, rather than developing a rapport with a single user, it is essential to understand and facilitate the dynamics of the group as a whole. In terms of task performance, technology should support the group's decision-making process by efficiently managing the task execution process. Considering the above characteristics of group interactions, I created the chatbot agents that facilitate group communication in two different contexts and verified their effectiveness. GroupfeedBot was designed and developed with the aim of enhancing group discussion in social chat groups. GroupfeedBot possesses the feature of (1) managing time, (2) encouraging members to participate evenly, and (3) organizing the members’ diverse opinions. The group which discussed with GroupfeedBot tended to produce more diverse opinions compared to the group discussed with the basic chatbot. Some effects of GroupfeedBot varied by the task's characteristics. GroupfeedBot encouraged the members to contribute evenly to the discussions, especially for the open-debating task. On the other hand, DebateBot was designed and developed to facilitate deliberative discussion. In contrast to GroupfeedBot, DebateBot was applied to more serious and less casual social contexts. Two main features were implemented in DebateBot: (1) structure discussion and (2) request opinions from reticent discussants.This work found that a chatbot agent which structures discussions and promotes even participation can improve discussions, resulting in higher quality deliberative discussion. Overall, adding structure to the discussion positively influenced the discussion quality, and the facilitation helped groups reach a genuine consensus and improved the subjective satisfaction of the group members. The findings of this thesis reflect the importance of understanding human factors in designing AI-infused systems. By understanding the characteristics of individual humans and collective groups, we are able to place humans at the heart of the system and utilize AI technology in a human-friendly way.1. Introduction 1.1 Background 1.2 Rise of Machine Agency 1.3 Theoretical Framework 1.4 Research Goal 1.5 Research Approach 1.6 Summary of Contributions 1.7 Thesis Overview 2. Related Work 2.1 A Brief History of Conversational Agents 2.2 TAMED Framework 3. Designing Conversational Agents for Dyadic Interaction 3.1 Background 3.2 Related Work 3.3 Method 3.4 Results 3.5 Discussion 3.6 Conclusion 4. Designing Conversational Agents for Social Group Discussion 4.1 Background 4.2 Related Work 4.3 Needfinding Survey for Facilitator Chatbot Agent 4.4 GroupfeedBot: A Chatbot Agent For Facilitating Discussion in Group Chats 4.5 Qualitative Study with Small-Sized Group 4.6 User Study With Medium-Sized Group 4.7 Discussion 4.8 Conclusion 5. Designing Conversational Agents for Deliberative Group Discussion 5.1 Background 5.2 Related Work 5.3 DebateBot 5.4 Method 5.5 Results 5.6 Discussion and Design Implications 5.7 Conclusion 6. Discussion 6.1 Designing Conversational Agents as a Communicator 6.2 Design Guidelines Based on TAMED Model 6.3 Technical Considerations 6.4 Human-AI Collaborative System 7. Conclusion 7.1 Research Summary 7.2 Summary of Contributions 7.3 Future Work 7.4 Conclusion박