인공지능과 대화하기: 일대일 그리고 그룹 상용작용을 위한 대화형 에이전트 시스템 개발

학위논문(박사) -- 서울대학교대학원 : 사회과학대학 언론정보학과, 2022.2. 이준환."인간-컴퓨터 상호작용"과 "사용자 경험"을 넘어, "인간-인공지능 상호작용" 그리고 "알고리즘 경험"의 시대가 도래하고 있다. 기술의 발전은 우리가 의사소통하고 협업하는 방식의 패러다임을 전환했다. 기계 에이전트는 인간 커뮤니케이션에서 적극적이며 주도적인 역할을 수행한다. 하지만 효과적인 AI 기반 커뮤니케이션과 토론 시스템 디자인에 대한 이해와 논의는 부족한 것이 사실이다. 이에 본 연구는 인간-컴퓨터 상호작용의 관점에서 다양한 형태의 커뮤니케이션을 지원할 수 있는 기술적 방법을 탐색하는 것을 목표로 한다. 이를 위해 저자는 일대일 그리고 그룹 상호작용을 지원하는 대화형 에이전트를 제시한다. 구체적으로 본 연구는 1) 일대일 상호작요에서 사용자 관여를 높이는 대화형 에이전트, 2) 일상적인 소셜 그룹 토론을 지원하는 에이전트, 3) 숙의 토론을 가능하게 하는 에이전트를 디자인 및 개발하고 그 효과를 정량적 그리고 정성적으로 검증했다. 시스템을 디자인함에 있어서 인간-컴퓨터 상호작용뿐 아니라, 커뮤니케이션학, 심리학, 그리고 데이터 과학을 접목한 다학제적 접근 방식이 적용되었다. 첫 번째 연구는 일대일 상호작용 상황에서 사용자의 관여 증진을 위한 대화형 에이전트의 효과를 검증했다. 설문조사라는 맥락에서 수행된 이 연구는 웹 설문조사에서 응답자의 불성실로 인해 발생하는 응답 데이터 품질의 문제를 극복하기 위한 새로운 인터랙션 방법으로 텍스트 기반 대화형 에이전트의 가능성을 탐색하는 것을 목표로 했다. 이를 위해 2 (인터페이스: 웹 對 챗봇) X 2 (대화 스타일: 포멀 對 캐쥬얼) 실험을 진행했으며, 만족화 이론에 근거하여 응답 데이터의 품질을 평가했다. 그 결과, 챗봇 설문조사의 참여자가 웹 설문조사의 참여자보다 더 높은 수준의 관여를 보이고, 결과적으로 더 높은 품질의 데이터를 생성하는 것을 확인할 수 있었다. 하지만 이런 챗봇의 데이터 품질에 대한 효과는 챗봇이 친구 같고 캐쥬얼한 대화체를 사용할 때만 나타났다. 이 결과는 대화형 인터랙티비티가 인터페이스뿐 아니라 대화 스타일이라는 효과적인 메세지 전략을 동반할 때 발생하는 것을 의미한다. 두 번째 연구는 일상적인 소셜 채팅 그룹에서 집단의 의사결정과정과 토론을 지원하는 대화형 시스템에 대한 것이다. 이를 위해 GroupfeedBot이라는 대화형 에이전트를 제작하였으며, GroupfeedBot은 (1) 토론 시간을 관리하고, (2) 구성원들의 균등한 참여를 촉진하며, (3) 구성원들의 다양한 의견을 요약 및 조직화하는 기능을 갖고 있다. 해당 에이전트를 평가하기 위해 다양한 태스크 (추론, 의사결정, 자유 토론, 문제 해결 과제)와 그룹 규모(소규모, 중규모)에 관하여 사용자 조사를 시행했다. 그 결과 의견의 다양성 측면에서 GroupfeedBot으로 토론한 집단이 기본 에이전트와 토론한 집단보다 더 다양한 의견을 생성했지만 산출된 결과의 품질과 메시지 양에 있어서는 차이가 없는 것을 확인할 수 있었다. 균등한 참여에 대한 GroupfeedBot의 효과는 태스크의 특성에 따라 다르게 나타났는데, 특히 자유 토론 과제에서 GroupfeedBot이 참여자들의 균등한 참여를 촉진했다. 세 번째 연구는 숙의 토론을 지원하는 대화형 시스템에 대한 것이다. 세 번째 연구에서 개발된 DebateBot은 GroupfeeedBot과 달리 더 진지한 사회적 맥락에서 적용되었다. DebateBot은 (1) 생각하기-짝짓기-공유하기 (Think-Pair-Share) 전략에 따라 토론을 구조화하고, (2) 과묵한 토론자에게 의견을 요청함으로써 동등한 참여를 촉진하는 두 가지 주요 기능을 수행했다. 사용자 평가 결과 DebateBot은 그룹 상호작용을 개선함으로써 심의 토론을 가능하게 했다. 토론 구조화는 토론의 질에 긍정적인 효과를 발휘하였고, 참여자 촉진은 진정한 합의 도달에 기여하였으며, 그룹 구성원들의 주관적 만족도를 향상했다. 본 연구는 이 세 가지 연구의 결과들을 바탕으로 인간-인공지능 커뮤니케이션에 대한 다양한 시사점들을 도출하였으며, 이를 TAMED (Task-Agent-Message-Information Exchange-Relationship Dynamics) 모델로 정리하였다.The advancements in technology shift the paradigm of how individuals communicate and collaborate. Machines play an active role in human communication. However, we still lack a generalized understanding of how exactly to design effective machine-driven communication and discussion systems. How should machine agents be designed differently when interacting with a single user as opposed to when interacting with multiple users? How can machine agents be designed to drive user engagement during dyadic interaction? What roles can machine agents perform for the sake of group interaction contexts? How should technology be implemented in support of the group decision-making process and to promote group dynamics? What are the design and technical issues which should be considered for the sake of creating human-centered interactive systems? In this thesis, I present new interactive systems in the form of a conversational agent, or a chatbot, that facilitate dyadic and group interactions. Specifically, I focus on: 1) a conversational agent to engage users in dyadic communication, 2) a chatbot called GroupfeedBot that facilitates daily social group discussion, 3) a chatbot called DebateBot that enables deliberative discussion. My approach to research is multidisciplinary and informed by not only in HCI, but also communication, psychology and data science. In my work, I conduct in-depth qualitative inquiry and quantitative data analysis towards understanding issues that users have with current systems, before developing new computational techniques that meet those user needs. Finally, I design, build, and deploy systems that use these techniques to the public in order to achieve real-world impact and to study their use by different usage contexts. The findings of this thesis are as follows. For a dyadic interaction, participants interacting with a chatbot system were more engaged as compared to those with a static web system. However, the conversational agent leads to better user engagement only when the messages apply a friendly, human-like conversational style. These results imply that the chatbot interface itself is not quite sufficient for the purpose of conveying conversational interactivity. Messages should also be carefully designed to convey such. Unlike dyadic interactions, which focus on message characteristics, other elements of the interaction should be considered when designing agents for group communication. In terms of messages, it is important to synthesize and organize information given that countless messages are exchanged simultaneously. In terms of relationship dynamics, rather than developing a rapport with a single user, it is essential to understand and facilitate the dynamics of the group as a whole. In terms of task performance, technology should support the group's decision-making process by efficiently managing the task execution process. Considering the above characteristics of group interactions, I created the chatbot agents that facilitate group communication in two different contexts and verified their effectiveness. GroupfeedBot was designed and developed with the aim of enhancing group discussion in social chat groups. GroupfeedBot possesses the feature of (1) managing time, (2) encouraging members to participate evenly, and (3) organizing the members’ diverse opinions. The group which discussed with GroupfeedBot tended to produce more diverse opinions compared to the group discussed with the basic chatbot. Some effects of GroupfeedBot varied by the task's characteristics. GroupfeedBot encouraged the members to contribute evenly to the discussions, especially for the open-debating task. On the other hand, DebateBot was designed and developed to facilitate deliberative discussion. In contrast to GroupfeedBot, DebateBot was applied to more serious and less casual social contexts. Two main features were implemented in DebateBot: (1) structure discussion and (2) request opinions from reticent discussants.This work found that a chatbot agent which structures discussions and promotes even participation can improve discussions, resulting in higher quality deliberative discussion. Overall, adding structure to the discussion positively influenced the discussion quality, and the facilitation helped groups reach a genuine consensus and improved the subjective satisfaction of the group members. The findings of this thesis reflect the importance of understanding human factors in designing AI-infused systems. By understanding the characteristics of individual humans and collective groups, we are able to place humans at the heart of the system and utilize AI technology in a human-friendly way.1. Introduction 1.1 Background 1.2 Rise of Machine Agency 1.3 Theoretical Framework 1.4 Research Goal 1.5 Research Approach 1.6 Summary of Contributions 1.7 Thesis Overview 2. Related Work 2.1 A Brief History of Conversational Agents 2.2 TAMED Framework 3. Designing Conversational Agents for Dyadic Interaction 3.1 Background 3.2 Related Work 3.3 Method 3.4 Results 3.5 Discussion 3.6 Conclusion 4. Designing Conversational Agents for Social Group Discussion 4.1 Background 4.2 Related Work 4.3 Needfinding Survey for Facilitator Chatbot Agent 4.4 GroupfeedBot: A Chatbot Agent For Facilitating Discussion in Group Chats 4.5 Qualitative Study with Small-Sized Group 4.6 User Study With Medium-Sized Group 4.7 Discussion 4.8 Conclusion 5. Designing Conversational Agents for Deliberative Group Discussion 5.1 Background 5.2 Related Work 5.3 DebateBot 5.4 Method 5.5 Results 5.6 Discussion and Design Implications 5.7 Conclusion 6. Discussion 6.1 Designing Conversational Agents as a Communicator 6.2 Design Guidelines Based on TAMED Model 6.3 Technical Considerations 6.4 Human-AI Collaborative System 7. Conclusion 7.1 Research Summary 7.2 Summary of Contributions 7.3 Future Work 7.4 Conclusion박

Look me in the eyes: A survey of eye and gaze animation for virtual agents and artificial systems

International audienceA person's emotions and state of mind are apparent in their face and eyes. As a Latin proverb states: "The face is the portrait of the mind; the eyes, its informers.". This presents a huge challenge for computer graphics researchers in the generation of artificial entities that aim to replicate the movement and appearance of the human eye, which is so important in human-human interactions. This State of the Art Report provides an overview of the efforts made on tackling this challenging task. As with many topics in Computer Graphics, a cross-disciplinary approach is required to fully understand the workings of the eye in the transmission of information to the user. We discuss the movement of the eyeballs, eyelids, and the head from a physiological perspective and how these movements can be modelled, rendered and animated in computer graphics applications. Further, we present recent research from psychology and sociology that seeks to understand higher level behaviours, such as attention and eye-gaze, during the expression of emotion or during conversation, and how they are synthesised in Computer Graphics and Robotics

Social behavior modeling based on Incremental Discrete Hidden Markov Models

12 pagesInternational audienceModeling multimodal face-to-face interaction is a crucial step in the process of building social robots or users-aware Embodied Conversational Agents (ECA). In this context, we present a novel approach for human behavior analysis and generation based on what we called "Incremental Discrete Hidden Markov Model" (IDHMM). Joint multimodal activities of interlocutors are first modeled by a set of DHMMs that are specific to supposed joint cognitive states of the interlocutors. Respecting a task-specific syntax, the IDHMM is then built from these DHMMs and split into i) a recognition model that will determine the most likely sequence of cognitive states given the multimodal activity of the in- terlocutor, and ii) a generative model that will compute the most likely activity of the speaker given this estimated sequence of cognitive states. Short-Term Viterbi (STV) decoding is used to incrementally recognize and generate behav- ior. The proposed model is applied to parallel speech and gaze data of interact- ing dyads

Automatic Context-Driven Inference of Engagement in HMI: A Survey

An integral part of seamless human-human communication is engagement, the process by which two or more participants establish, maintain, and end their perceived connection. Therefore, to develop successful human-centered human-machine interaction applications, automatic engagement inference is one of the tasks required to achieve engaging interactions between humans and machines, and to make machines attuned to their users, hence enhancing user satisfaction and technology acceptance. Several factors contribute to engagement state inference, which include the interaction context and interactants' behaviours and identity. Indeed, engagement is a multi-faceted and multi-modal construct that requires high accuracy in the analysis and interpretation of contextual, verbal and non-verbal cues. Thus, the development of an automated and intelligent system that accomplishes this task has been proven to be challenging so far. This paper presents a comprehensive survey on previous work in engagement inference for human-machine interaction, entailing interdisciplinary definition, engagement components and factors, publicly available datasets, ground truth assessment, and most commonly used features and methods, serving as a guide for the development of future human-machine interaction interfaces with reliable context-aware engagement inference capability. An in-depth review across embodied and disembodied interaction modes, and an emphasis on the interaction context of which engagement perception modules are integrated sets apart the presented survey from existing surveys

Acoustic Features of Different Types of Laughter in North Sami Conversational Speech

Peer reviewe

A museum guide robot: Dealing with multiple participants in the real-world

Pitsch K, Gehle R, Wrede S. A museum guide robot: Dealing with multiple participants in the real-world. Presented at the Workshop "Robots in public spaces. Towards multi-party, short-term, dynamic human-robot interaction" at ICSR 2013.Using video-recordings from a real-world eld trial of a mu-seum guide robot, we show how a robot's gaze influences the visitors' state of participation in group constellations. Then, we compare the robot's conduct to a human tour guide's gaze strategies. We argue that a robot system, to deal with real-world everyday situations, needs to be equipped with knowledge about interactional coordination, incremental processing and strategies for pro-actively shaping the users' conduct