Designing Embodied Interactive Software Agents for E-Learning: Principles, Components, and Roles

Embodied interactive software agents are complex autonomous, adaptive, and social software systems with a digital embodiment that enables them to act on and react to other entities (users, objects, and other agents) in their environment through bodily actions, which include the use of verbal and non-verbal communicative behaviors in face-to-face interactions with the user. These agents have been developed for various roles in different application domains, in which they perform tasks that have been assigned to them by their developers or delegated to them by their users or by other agents. In computer-assisted learning, embodied interactive pedagogical software agents have the general task to promote human learning by working with students (and other agents) in computer-based learning environments, among them e-learning platforms based on Internet technologies, such as the Virtual Linguistics Campus (www.linguistics-online.com). In these environments, pedagogical agents provide contextualized, qualified, personalized, and timely assistance, cooperation, instruction, motivation, and services for both individual learners and groups of learners. This thesis develops a comprehensive, multidisciplinary, and user-oriented view of the design of embodied interactive pedagogical software agents, which integrates theoretical and practical insights from various academic and other fields. The research intends to contribute to the scientific understanding of issues, methods, theories, and technologies that are involved in the design, implementation, and evaluation of embodied interactive software agents for different roles in e-learning and other areas. For developers, the thesis provides sixteen basic principles (Added Value, Perceptible Qualities, Balanced Design, Coherence, Consistency, Completeness, Comprehensibility, Individuality, Variability, Communicative Ability, Modularity, Teamwork, Participatory Design, Role Awareness, Cultural Awareness, and Relationship Building) plus a large number of specific guidelines for the design of embodied interactive software agents and their components. Furthermore, it offers critical reviews of theories, concepts, approaches, and technologies from different areas and disciplines that are relevant to agent design. Finally, it discusses three pedagogical agent roles (virtual native speaker, coach, and peer) in the scenario of the linguistic fieldwork classes on the Virtual Linguistics Campus and presents detailed considerations for the design of an agent for one of these roles (the virtual native speaker)

Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data

Large language models (LLMs) provide a new way to build chatbots by accepting natural language prompts. Yet, it is unclear how to design prompts to power chatbots to carry on naturalistic conversations while pursuing a given goal, such as collecting self-report data from users. We explore what design factors of prompts can help steer chatbots to talk naturally and collect data reliably. To this aim, we formulated four prompt designs with different structures and personas. Through an online study (N = 48) where participants conversed with chatbots driven by different designs of prompts, we assessed how prompt designs and conversation topics affected the conversation flows and users' perceptions of chatbots. Our chatbots covered 79% of the desired information slots during conversations, and the designs of prompts and topics significantly influenced the conversation flows and the data collection performance. We discuss the opportunities and challenges of building chatbots with LLMs.Comment: 22 pages including Appendix, 7 figures, 7 tables. Accepted to PACM HCI (CSCW 2024

인공지능과 대화하기: 일대일 그리고 그룹 상용작용을 위한 대화형 에이전트 시스템 개발

학위논문(박사) -- 서울대학교대학원 : 사회과학대학 언론정보학과, 2022.2. 이준환."인간-컴퓨터 상호작용"과 "사용자 경험"을 넘어, "인간-인공지능 상호작용" 그리고 "알고리즘 경험"의 시대가 도래하고 있다. 기술의 발전은 우리가 의사소통하고 협업하는 방식의 패러다임을 전환했다. 기계 에이전트는 인간 커뮤니케이션에서 적극적이며 주도적인 역할을 수행한다. 하지만 효과적인 AI 기반 커뮤니케이션과 토론 시스템 디자인에 대한 이해와 논의는 부족한 것이 사실이다. 이에 본 연구는 인간-컴퓨터 상호작용의 관점에서 다양한 형태의 커뮤니케이션을 지원할 수 있는 기술적 방법을 탐색하는 것을 목표로 한다. 이를 위해 저자는 일대일 그리고 그룹 상호작용을 지원하는 대화형 에이전트를 제시한다. 구체적으로 본 연구는 1) 일대일 상호작요에서 사용자 관여를 높이는 대화형 에이전트, 2) 일상적인 소셜 그룹 토론을 지원하는 에이전트, 3) 숙의 토론을 가능하게 하는 에이전트를 디자인 및 개발하고 그 효과를 정량적 그리고 정성적으로 검증했다. 시스템을 디자인함에 있어서 인간-컴퓨터 상호작용뿐 아니라, 커뮤니케이션학, 심리학, 그리고 데이터 과학을 접목한 다학제적 접근 방식이 적용되었다. 첫 번째 연구는 일대일 상호작용 상황에서 사용자의 관여 증진을 위한 대화형 에이전트의 효과를 검증했다. 설문조사라는 맥락에서 수행된 이 연구는 웹 설문조사에서 응답자의 불성실로 인해 발생하는 응답 데이터 품질의 문제를 극복하기 위한 새로운 인터랙션 방법으로 텍스트 기반 대화형 에이전트의 가능성을 탐색하는 것을 목표로 했다. 이를 위해 2 (인터페이스: 웹 對 챗봇) X 2 (대화 스타일: 포멀 對 캐쥬얼) 실험을 진행했으며, 만족화 이론에 근거하여 응답 데이터의 품질을 평가했다. 그 결과, 챗봇 설문조사의 참여자가 웹 설문조사의 참여자보다 더 높은 수준의 관여를 보이고, 결과적으로 더 높은 품질의 데이터를 생성하는 것을 확인할 수 있었다. 하지만 이런 챗봇의 데이터 품질에 대한 효과는 챗봇이 친구 같고 캐쥬얼한 대화체를 사용할 때만 나타났다. 이 결과는 대화형 인터랙티비티가 인터페이스뿐 아니라 대화 스타일이라는 효과적인 메세지 전략을 동반할 때 발생하는 것을 의미한다. 두 번째 연구는 일상적인 소셜 채팅 그룹에서 집단의 의사결정과정과 토론을 지원하는 대화형 시스템에 대한 것이다. 이를 위해 GroupfeedBot이라는 대화형 에이전트를 제작하였으며, GroupfeedBot은 (1) 토론 시간을 관리하고, (2) 구성원들의 균등한 참여를 촉진하며, (3) 구성원들의 다양한 의견을 요약 및 조직화하는 기능을 갖고 있다. 해당 에이전트를 평가하기 위해 다양한 태스크 (추론, 의사결정, 자유 토론, 문제 해결 과제)와 그룹 규모(소규모, 중규모)에 관하여 사용자 조사를 시행했다. 그 결과 의견의 다양성 측면에서 GroupfeedBot으로 토론한 집단이 기본 에이전트와 토론한 집단보다 더 다양한 의견을 생성했지만 산출된 결과의 품질과 메시지 양에 있어서는 차이가 없는 것을 확인할 수 있었다. 균등한 참여에 대한 GroupfeedBot의 효과는 태스크의 특성에 따라 다르게 나타났는데, 특히 자유 토론 과제에서 GroupfeedBot이 참여자들의 균등한 참여를 촉진했다. 세 번째 연구는 숙의 토론을 지원하는 대화형 시스템에 대한 것이다. 세 번째 연구에서 개발된 DebateBot은 GroupfeeedBot과 달리 더 진지한 사회적 맥락에서 적용되었다. DebateBot은 (1) 생각하기-짝짓기-공유하기 (Think-Pair-Share) 전략에 따라 토론을 구조화하고, (2) 과묵한 토론자에게 의견을 요청함으로써 동등한 참여를 촉진하는 두 가지 주요 기능을 수행했다. 사용자 평가 결과 DebateBot은 그룹 상호작용을 개선함으로써 심의 토론을 가능하게 했다. 토론 구조화는 토론의 질에 긍정적인 효과를 발휘하였고, 참여자 촉진은 진정한 합의 도달에 기여하였으며, 그룹 구성원들의 주관적 만족도를 향상했다. 본 연구는 이 세 가지 연구의 결과들을 바탕으로 인간-인공지능 커뮤니케이션에 대한 다양한 시사점들을 도출하였으며, 이를 TAMED (Task-Agent-Message-Information Exchange-Relationship Dynamics) 모델로 정리하였다.The advancements in technology shift the paradigm of how individuals communicate and collaborate. Machines play an active role in human communication. However, we still lack a generalized understanding of how exactly to design effective machine-driven communication and discussion systems. How should machine agents be designed differently when interacting with a single user as opposed to when interacting with multiple users? How can machine agents be designed to drive user engagement during dyadic interaction? What roles can machine agents perform for the sake of group interaction contexts? How should technology be implemented in support of the group decision-making process and to promote group dynamics? What are the design and technical issues which should be considered for the sake of creating human-centered interactive systems? In this thesis, I present new interactive systems in the form of a conversational agent, or a chatbot, that facilitate dyadic and group interactions. Specifically, I focus on: 1) a conversational agent to engage users in dyadic communication, 2) a chatbot called GroupfeedBot that facilitates daily social group discussion, 3) a chatbot called DebateBot that enables deliberative discussion. My approach to research is multidisciplinary and informed by not only in HCI, but also communication, psychology and data science. In my work, I conduct in-depth qualitative inquiry and quantitative data analysis towards understanding issues that users have with current systems, before developing new computational techniques that meet those user needs. Finally, I design, build, and deploy systems that use these techniques to the public in order to achieve real-world impact and to study their use by different usage contexts. The findings of this thesis are as follows. For a dyadic interaction, participants interacting with a chatbot system were more engaged as compared to those with a static web system. However, the conversational agent leads to better user engagement only when the messages apply a friendly, human-like conversational style. These results imply that the chatbot interface itself is not quite sufficient for the purpose of conveying conversational interactivity. Messages should also be carefully designed to convey such. Unlike dyadic interactions, which focus on message characteristics, other elements of the interaction should be considered when designing agents for group communication. In terms of messages, it is important to synthesize and organize information given that countless messages are exchanged simultaneously. In terms of relationship dynamics, rather than developing a rapport with a single user, it is essential to understand and facilitate the dynamics of the group as a whole. In terms of task performance, technology should support the group's decision-making process by efficiently managing the task execution process. Considering the above characteristics of group interactions, I created the chatbot agents that facilitate group communication in two different contexts and verified their effectiveness. GroupfeedBot was designed and developed with the aim of enhancing group discussion in social chat groups. GroupfeedBot possesses the feature of (1) managing time, (2) encouraging members to participate evenly, and (3) organizing the members’ diverse opinions. The group which discussed with GroupfeedBot tended to produce more diverse opinions compared to the group discussed with the basic chatbot. Some effects of GroupfeedBot varied by the task's characteristics. GroupfeedBot encouraged the members to contribute evenly to the discussions, especially for the open-debating task. On the other hand, DebateBot was designed and developed to facilitate deliberative discussion. In contrast to GroupfeedBot, DebateBot was applied to more serious and less casual social contexts. Two main features were implemented in DebateBot: (1) structure discussion and (2) request opinions from reticent discussants.This work found that a chatbot agent which structures discussions and promotes even participation can improve discussions, resulting in higher quality deliberative discussion. Overall, adding structure to the discussion positively influenced the discussion quality, and the facilitation helped groups reach a genuine consensus and improved the subjective satisfaction of the group members. The findings of this thesis reflect the importance of understanding human factors in designing AI-infused systems. By understanding the characteristics of individual humans and collective groups, we are able to place humans at the heart of the system and utilize AI technology in a human-friendly way.1. Introduction 1.1 Background 1.2 Rise of Machine Agency 1.3 Theoretical Framework 1.4 Research Goal 1.5 Research Approach 1.6 Summary of Contributions 1.7 Thesis Overview 2. Related Work 2.1 A Brief History of Conversational Agents 2.2 TAMED Framework 3. Designing Conversational Agents for Dyadic Interaction 3.1 Background 3.2 Related Work 3.3 Method 3.4 Results 3.5 Discussion 3.6 Conclusion 4. Designing Conversational Agents for Social Group Discussion 4.1 Background 4.2 Related Work 4.3 Needfinding Survey for Facilitator Chatbot Agent 4.4 GroupfeedBot: A Chatbot Agent For Facilitating Discussion in Group Chats 4.5 Qualitative Study with Small-Sized Group 4.6 User Study With Medium-Sized Group 4.7 Discussion 4.8 Conclusion 5. Designing Conversational Agents for Deliberative Group Discussion 5.1 Background 5.2 Related Work 5.3 DebateBot 5.4 Method 5.5 Results 5.6 Discussion and Design Implications 5.7 Conclusion 6. Discussion 6.1 Designing Conversational Agents as a Communicator 6.2 Design Guidelines Based on TAMED Model 6.3 Technical Considerations 6.4 Human-AI Collaborative System 7. Conclusion 7.1 Research Summary 7.2 Summary of Contributions 7.3 Future Work 7.4 Conclusion박

One Explanation Does Not Fit All The Promise of Interactive Explanations for Machine Learning Transparency

The need for transparency of predictive systems based on Machine Learning algorithms arises as a consequence of their ever-increasing proliferation in the industry. Whenever black-box algorithmic predictions influence human affairs, the inner workings of these algorithms should be scrutinised and their decisions explained to the relevant stakeholders, including the system engineers, the system's operators and the individuals whose case is being decided. While a variety of interpretability and explainability methods is available, none of them is a panacea that can satisfy all diverse expectations and competing objectives that might be required by the parties involved. We address this challenge in this paper by discussing the promises of Interactive Machine Learning for improved transparency of black-box systems using the example of contrastive explanations -- a state-of-the-art approach to Interpretable Machine Learning. Specifically, we show how to personalise counterfactual explanations by interactively adjusting their conditional statements and extract additional explanations by asking follow-up "What if?" questions. Our experience in building, deploying and presenting this type of system allowed us to list desired properties as well as potential limitations, which can be used to guide the development of interactive explainers. While customising the medium of interaction, i.e., the user interface comprising of various communication channels, may give an impression of personalisation, we argue that adjusting the explanation itself and its content is more important. To this end, properties such as breadth, scope, context, purpose and target of the explanation have to be considered, in addition to explicitly informing the explainee about its limitations and caveats...Comment: Published in the Kunstliche Intelligenz journal, special issue on Challenges in Interactive Machine Learnin

‘IMPLICIT CREATION’ – NON-PROGRAMMER CONCEPTUAL MODELS FOR AUTHORING IN INTERACTIVE DIGITAL STORYTELLING

Interactive Digital Storytelling (IDS) constitutes a research field that emerged from several areas of art, creation and computer science. It inquires technologies and possible artefacts that allow ‘highly-interactive’ experiences of digital worlds with compelling stories. However, the situation for story creators approaching ‘highly-interactive’ storytelling is complex. There is a gap between the available technology, which requires programming and prior knowledge in Artificial Intelligence, and established models of storytelling, which are too linear to have the potential to be highly interactive. This thesis reports on research that lays the ground for bridging this gap, leading to novel creation philosophies in future work. A design research process has been pursued, which centred on the suggestion of conceptual models, explaining a) process structures of interdisciplinary development, b) interactive story structures including the user of the interactive story system, and c) the positioning of human authors within semi-automated creative processes. By means of ‘implicit creation’, storytelling and modelling of simulated worlds are reconciled. The conceptual models are informed by exhaustive literature review in established neighbouring disciplines. These are a) creative principles in different storytelling domains, such as screenwriting, video game writing, role playing and improvisational theatre, b) narratological studies of story grammars and structures, and c) principles of designing interactive systems, in the areas of basic HCI design and models, discourse analysis in conversational systems, as well as game- and simulation design. In a case study of artefact building, the initial models have been put into practice, evaluated and extended. These artefacts are a) a conceived authoring tool (‘Scenejo’) for the creation of digital conversational stories, and b) the development of a serious game (‘The Killer Phrase Game’) as an application development. The study demonstrates how starting out from linear storytelling, iterative steps of ‘implicit creation’ can lead to more variability and interactivity in the designed interactive story. In the concrete case, the steps included abstraction of dialogues into conditional actions, and creating a dynamic world model of the conversation. This process and artefact can be used as a model illustrating non-programmer approaches to ‘implicit creation’ in a learning process. Research demonstrates that the field of Interactive Digital Storytelling still has to be further advanced until general creative principles can be fully established, which is a long-term endeavour, dependent upon environmental factors. It also requires further technological developments. The gap is not yet closed, but it can be better explained. The research results build groundwork for education of prospective authors. Concluding the thesis, IDS-specific creative principles have been proposed for evaluation in future work

Proceedings ICPW'07: 2nd International Conference on the Pragmatic Web, 22-23 Oct. 2007, Tilburg: NL

Proceedings ICPW'07: 2nd International Conference on the Pragmatic Web, 22-23 Oct. 2007, Tilburg: N