Interacting with educational chatbots: A systematic review

Chatbots hold the promise of revolutionizing education by engaging learners, personalizing learning activities, supporting educators, and developing deep insight into learners’ behavior. However, there is a lack of studies that analyze the recent evidence-based chatbot-learner interaction design techniques applied in education. This study presents a systematic review of 36 papers to understand, compare, and reflect on recent attempts to utilize chatbots in education using seven dimensions: educational field, platform, design principles, the role of chatbots, interaction styles, evidence, and limitations. The results show that the chatbots were mainly designed on a web platform to teach computer science, language, general education, and a few other fields such as engineering and mathematics. Further, more than half of the chatbots were used as teaching agents, while more than a third were peer agents. Most of the chatbots used a predetermined conversational path, and more than a quarter utilized a personalized learning approach that catered to students’ learning needs, while other chatbots used experiential and collaborative learning besides other design principles. Moreover, more than a third of the chatbots were evaluated with experiments, and the results primarily point to improved learning and subjective satisfaction. Challenges and limitations include inadequate or insufficient dataset training and a lack of reliance on usability heuristics. Future studies should explore the effect of chatbot personality and localization on subjective satisfaction and learning effectiveness

Does Role-Playing Chatbots Capture the Character Personalities? Assessing Personality Traits for Role-Playing Chatbots

The emergence of large-scale pretrained language models has revolutionized the capabilities of new AI application, especially in the realm of crafting chatbots with distinct personas. Given the "stimulus-response" nature of chatbots, this paper unveils an innovative open-ended interview-style approach for personality assessment on role-playing chatbots, which offers a richer comprehension of their intrinsic personalities. We conduct personality assessments on 32 role-playing chatbots created by the ChatHaruhi library, across both the Big Five and MBTI dimensions, and measure their alignment with human perception. Evaluation results underscore that modern role-playing chatbots based on LLMs can effectively portray personality traits of corresponding characters, with an alignment rate of 82.8% compared with human-perceived personalities. Besides, we also suggest potential strategies for shaping chatbots' personalities. Hence, this paper serves as a cornerstone study for role-playing chatbots that intersects computational linguistics and psychology. Our resources are available at https://github.com/LC1332/Chat-Haruhi-SuzumiyaComment: A Personality Traits Test Over ChatHaruh

Co-designing a chatbot for and with refugees and migrants

An information portal, HandbookGermany.de, is developed to support the integration of refugees and migrants into society in Germany. However, the information-seeking process is exhausting, cumbersome, and even confusing if refugees and migrants are not proficient at using web services. In light of this, a chatbot-based conversational service is considered as an alternative to enhance the information-seeking experience. For the purpose of designing products and services for refugees and migrants, a great deal of research proposes employing co-design methods as an effective means. The overall aim of this thesis is to explore, understand, and define possibilities of improving refugees and migrants’ experiences of social integration by proposing an engaging and efficient chatbot solution. Furthermore, this thesis aims to explore the necessity of co-design approach as a critical methodology to develop solutions. Therefore, the main research question in this thesis is how can a co-design approach contribute to designing a chatbot supporting social integration within the context of refugees and migrants. User experience, problems, and needs are unveiled in depth by listening to migrants and refugees’ problems, behaviors, and expectations (i.e., document studies, questionnaires, cultural probes, and expert interviews), and observing how migrants interact with the chatbot (i.e., participant observations and empathy probes). The research findings are then transformed into design questions. The designer, developers, and migrants jointly generate concepts leveraging generative toolkits in co-design workshops. By using surveys, the Method for the Assessment of eXperience (MAX), and property checklists, the resulting concepts are later validated with refugees and migrants. As research through design, this thesis draws three conclusions. Firstly, the co-design approach benefits defining problems in the complex context of refugees and migrants by supporting them in expressing ideas and thoughts. The defined problems can then be converted into design questions that promote the proceeding of the design process. Secondly, the co-design approach helps to develop mature concepts, which lays a foundation for the final design. Thirdly, the utilization of co-design tools plays an essential role in validating and refining the solution efficiently, as they make ideas concrete and visible so that refugees and migrants can easily reflect on them throughout the whole design process

Designing Personality-Adaptive Conversational Agents for Mental Health Care

Millions of people experience mental health issues each year, increasing the necessity for health-related services. One emerging technology with the potential to help address the resulting shortage in health care providers and other barriers to treatment access are conversational agents (CAs). CAs are software-based systems designed to interact with humans through natural language. However, CAs do not live up to their full potential yet because they are unable to capture dynamic human behavior to an adequate extent to provide responses tailored to users’ personalities. To address this problem, we conducted a design science research (DSR) project to design personality-adaptive conversational agents (PACAs). Following an iterative and multi-step approach, we derive and formulate six design principles for PACAs for the domain of mental health care. The results of our evaluation with psychologists and psychiatrists suggest that PACAs can be a promising source of mental health support. With our design principles, we contribute to the body of design knowledge for CAs and provide guidance for practitioners who intend to design PACAs. Instantiating the principles may improve interaction with users who seek support for mental health issues