Evorus: A Crowd-powered Conversational Assistant Built to Automate Itself Over Time

Crowd-powered conversational assistants have been shown to be more robust than automated systems, but do so at the cost of higher response latency and monetary costs. A promising direction is to combine the two approaches for high quality, low latency, and low cost solutions. In this paper, we introduce Evorus, a crowd-powered conversational assistant built to automate itself over time by (i) allowing new chatbots to be easily integrated to automate more scenarios, (ii) reusing prior crowd answers, and (iii) learning to automatically approve response candidates. Our 5-month-long deployment with 80 participants and 281 conversations shows that Evorus can automate itself without compromising conversation quality. Crowd-AI architectures have long been proposed as a way to reduce cost and latency for crowd-powered systems; Evorus demonstrates how automation can be introduced successfully in a deployed system. Its architecture allows future researchers to make further innovation on the underlying automated components in the context of a deployed open domain dialog system.Comment: 10 pages. To appear in the Proceedings of the Conference on Human Factors in Computing Systems 2018 (CHI'18

Chatbots as Unwitting Actors

Chatbots are popular for both task-oriented conversations and unstructured conversations with web users. Several different approaches to creating comedy and art exist across the field of computational creativity. Despite the popularity and ease of use of chatbots, there have not been any attempts by artists or comedians to use these systems for comedy performances. We present two initial attempts to do so from our comedy podcast and call for future work toward both designing chatbots for performance and for performing alongside chatbots

Improving fairness in machine learning systems: What do industry practitioners need?

The potential for machine learning (ML) systems to amplify social inequities and unfairness is receiving increasing popular and academic attention. A surge of recent work has focused on the development of algorithmic tools to assess and mitigate such unfairness. If these tools are to have a positive impact on industry practice, however, it is crucial that their design be informed by an understanding of real-world needs. Through 35 semi-structured interviews and an anonymous survey of 267 ML practitioners, we conduct the first systematic investigation of commercial product teams' challenges and needs for support in developing fairer ML systems. We identify areas of alignment and disconnect between the challenges faced by industry practitioners and solutions proposed in the fair ML research literature. Based on these findings, we highlight directions for future ML and HCI research that will better address industry practitioners' needs.Comment: To appear in the 2019 ACM CHI Conference on Human Factors in Computing Systems (CHI 2019

Designing Process-based Chatbots in Enterprises: The Case of Business Travel Organization Considering the Users’ Perspective and Business Value

Chatbots have attracted much research attention in recent years, and organizations have increasingly begun applying them in everyday working life. However, researchers have rarely investigated how chatbots can support everyday tasks in enterprises. As such, we lack design knowledge for chatbots that support internal business processes since research has mostly examined customer-facing use cases. Notably, researchers have rarely considered chatbots’ economic and user-related effects, which, thus, remain unknown. To address this gap, we conducted a design science research study to survey a process-based chatbot application for business processes. From examining the scenario, we deduced design principles and implemented a software artifact. We evaluated the concept with 69 participants and surveyed the users’ perspective in terms of design and acceptance and the organizational perspective in terms of process efficiency and quality. In doing so, 1) we derived six design principles for process-based chatbots and implemented a respective chatbot, which enabled a user-adapted process and provided situational-dependent input options and support; 2) we found that users had a positive attitude towards using chatbots for business processes in terms of user experience and acceptance; and 3) the process performed at an economically efficient level that compared well with existing solutions and that IT affinity and prior experience had no influence on performance. Furthermore, our solution improved the process quality compared to the existing solution

Game-inspired Pedagogical Conversational Agents: A Systematic Literature Review

Pedagogical conversational agents (PCAs) are an innovative way to help learners improve their academic performance via intelligent dialog systems. However, PCAs have not yet reached their full potential. They often fail because users perceive conversations with them as not engaging. Enriching them with game-based approaches could contribute to mitigating this issue. One could enrich a PCA with game-based approaches by gamifying it to foster positive effects, such as fun and motivation, or by integrating it into a game-based learning (GBL) environment to promote effects such as social presence and enable individual learning support. We summarize PCAs that are combined with game-based approaches under the novel term “game-inspired PCAs”. We conducted a systematic literature review on this topic, as previous literature reviews on PCAs either have not combined the topics of PCAs and GBL or have done so to a limited extent only. We analyzed the literature regarding the existing design knowledge base, the game elements used, the thematic areas and target groups, the PCA roles and types, the extent of artificial intelligence (AI) usage, and opportunities for adaptation. We reduced the initial 3,034 records to 50 fully coded papers, from which we derived a morphological box and revealed current research streams and future research recommendations. Overall, our results show that the topic offers promising application potential but that scholars and practitioners have not yet considered it holistically. For instance, we found that researchers have rarely provided prescriptive design knowledge, have not sufficiently combined game elements, and have seldom used AI algorithms as well as intelligent possibilities of user adaptation in PCA development. Furthermore, researchers have scarcely considered certain target groups, thematic areas, and PCA roles. Consequently, our paper contributes to research and practice by addressing research gaps and structuring the existing knowledge base

Co-Design Disaster Management Chatbot with Indigenous Communities

Indigenous communities are disproportionately impacted by rising disaster risk, climate change, and environmental degradation due to their close relationship with the environment and its resources. Unfortunately, gathering the necessary information or evidence to request or co-share sufficient funds can be challenging for indigenous people and their lands. This paper aims to co-design an AI-based chatbot with two tribes and investigate their perception and experience of using it in disaster reporting practices. The study was conducted in two stages. Firstly, we interviewed experienced first-line emergency managers and invited tribal members to an in-person design workshop. Secondly, based on qualitative analysis, we identified three themes of emergency communication, documentation, and user experience. Our findings support that indigenous communities favored the proposed Emergency Reporter chatbot solution. We further discussed how the proposed chatbot could empower the tribes in disaster management, preserve sovereignty, and seek support from other agencies

A Maturity Assessment Framework for Conversational AI Development Platforms

Conversational Artificial Intelligence (AI) systems have recently sky-rocketed in popularity and are now used in many applications, from car assistants to customer support. The development of conversational AI systems is supported by a large variety of software platforms, all with similar goals, but different focus points and functionalities. A systematic foundation for classifying conversational AI platforms is currently lacking. We propose a framework for assessing the maturity level of conversational AI development platforms. Our framework is based on a systematic literature review, in which we extracted common and distinguishing features of various open-source and commercial (or in-house) platforms. Inspired by language reference frameworks, we identify different maturity levels that a conversational AI development platform may exhibit in understanding and responding to user inputs. Our framework can guide organizations in selecting a conversational AI development platform according to their needs, as well as helping researchers and platform developers improving the maturity of their platforms.Comment: 10 pages, 10 figures. Accepted for publication at SAC 2021: ACM/SIGAPP Symposium On Applied Computin

Design Knowledge for Virtual Learning Companions from a Value-centered Perspective

The increasing popularity of conversational agents such as ChatGPT has sparked interest in their potential use in educational contexts but undermines the role of companionship in learning with these tools. Our study targets the design of virtual learning companions (VLCs), focusing on bonding relationships for collaborative learning while facilitating students’ time management and motivation. We draw upon design science research (DSR) to derive prescriptive design knowledge for VLCs as the core of our contribution. Through three DSR cycles, we conducted interviews with working students and experts, held interdisciplinary workshops with the target group, designed and evaluated two conceptual prototypes, and fully coded a VLC instantiation, which we tested with students in class. Our approach has yielded 9 design principles, 28 meta-requirements, and 33 design features centered around the value-in-interaction. These encompass Human-likeness and Dialogue Management, Proactive and Reactive Behavior, and Relationship Building on the Relationship Layer (DP1,3,4), Adaptation (DP2) on the Matching Layer, as well as Provision of Supportive Content, Fostering Learning Competencies, Motivational Environment, and Ethical Responsibility (DP5-8) on the Service Layer

AI and extremism in social networks

Studien utforsker hvordan midler som kunstig intelligens, AI- drevne chatbots, kan være kilder man kan regne med som moralske aktører på digitale plattformer og som kan være identifiserbare opprørsmodeller til bekjempelse av ekstremistiske og voldsforherligende ytringer på sosiale medieplattformer. Fremveksten av digital nettverkskommunikasjon har lettet prosessen med sosiale bevegelser, noe fenomenet «Den arabiske våren» tydelig demonstrerer. Sosiale medier har vært et verdifullt verktøy når det gjelder å utvikle kollektive identiteter med en felles ideologi for å fremme et bestemt mål eller en sak og gi alternative plattformer for undertrykte samfunn. Imidlertid forblir virkningen og konsekvensene av sosiale medier i samfunn der maktbalansen forrykkes gjennom fundamentale endringer et bekymringsfullt fenomen. Radikaliserte individer og grupper har også hevdet sin tilstedeværelse på sosiale medieplattformer gjennom å fremme fordommer, hat og vold. Ekstremistiske grupper bruker ulike taktikker for å utøve makten sin på disse plattformene. Bekjempelsen av voldelig ekstremisme på sosiale medieplattformer blir som regel ikke koordinert av aktuelle aktører som regjeringer, sosiale medieselskaper, FN eller andre private organisasjoner. I tillegg har fremdeles ikke forsøk på å konstituere AI til bekjempelse av voldelig ekstremisme blitt gjennomført, men lovende resultater har blitt oppnådd gjennom noen initiativer. Prosjektet som en ‘case study’ ser på den nylige reformen i Etiopia som ble gjennomført av Nobels fredsprisvinner 2019 Abiy Ahmed etter at han tiltrådte som statsminister i Etiopia i april 2018. Etter flere tiår med undertrykkelse har den nye maktovertakelsen der det politiske rommet ble åpnet opp og ytringsfrihet ble tillatt, uventet ført til et skred av etniske gruppers polarisering. Nye etno-ekstremister har dukket frem fra alle kriker og kroker av landet og også fra sin tilværelse i diaspora. Studien ser videre på hvilken rolle sosiale medier til tider spiller ved direkte å presse på for å påvirke til og dermed forårsake voldelige handlinger på grasrota.Ved å bruke en kvalitativ forskningsmetode for ustrukturerte intervjuer med etiopiske brukere av sosiale medier, journalister og aktivister, identifiserer studien kjerneaspektene ved konfliktene og foreslår initiativer som kan brukes til å motvirke voldelig etnisk ekstremisme. Ved å bruke relevant litteratur ser prosjektet videre på innarbeidelsen av kunstig intelligens (AI) i «moralske handlinger» på sosiale medier og hvordan den kan utformes slik at den av seg selv kan ta i bruk moralske beslutningsevner i nettverket. I tillegg ser studien på mulighetene videre for bekjempelse av voldelig ekstremisme og skisserer den spesifikke rollen ikke menneskelige aktører som profesjonelle troll og bots på sosiale medier bør spille for å slåss mot radikalisering som kan føre til voldelige handlinger.Mastergradsoppgave i digital kulturMAHF-DIKULDIKULT35