Game-inspired Pedagogical Conversational Agents: A Systematic Literature Review

Pedagogical conversational agents (PCAs) are an innovative way to help learners improve their academic performance via intelligent dialog systems. However, PCAs have not yet reached their full potential. They often fail because users perceive conversations with them as not engaging. Enriching them with game-based approaches could contribute to mitigating this issue. One could enrich a PCA with game-based approaches by gamifying it to foster positive effects, such as fun and motivation, or by integrating it into a game-based learning (GBL) environment to promote effects such as social presence and enable individual learning support. We summarize PCAs that are combined with game-based approaches under the novel term “game-inspired PCAs”. We conducted a systematic literature review on this topic, as previous literature reviews on PCAs either have not combined the topics of PCAs and GBL or have done so to a limited extent only. We analyzed the literature regarding the existing design knowledge base, the game elements used, the thematic areas and target groups, the PCA roles and types, the extent of artificial intelligence (AI) usage, and opportunities for adaptation. We reduced the initial 3,034 records to 50 fully coded papers, from which we derived a morphological box and revealed current research streams and future research recommendations. Overall, our results show that the topic offers promising application potential but that scholars and practitioners have not yet considered it holistically. For instance, we found that researchers have rarely provided prescriptive design knowledge, have not sufficiently combined game elements, and have seldom used AI algorithms as well as intelligent possibilities of user adaptation in PCA development. Furthermore, researchers have scarcely considered certain target groups, thematic areas, and PCA roles. Consequently, our paper contributes to research and practice by addressing research gaps and structuring the existing knowledge base

The Gamification of Crowdsourcing Systems: Empirical Investigations and Design

Recent developments in modern information and communication technologies have spawned two rising phenomena, gamification and crowdsourcing, which are increasingly being combined into gamified crowdsourcing systems. While a growing number of organizations employ crowdsourcing as a way to outsource tasks related to the inventing, producing, funding, or distributing of their products and services to the crowd – a large group of people reachable via the internet – crowdsourcing initiatives become enriched with design features from games to motivate the crowd to participate in these efforts. From a practical perspective, this combination seems intuitively appealing, since using gamification in crowdsourcing systems promises to increase motivations, participation and output quality, as well as to replace traditionally used financial incentives. However, people in large groups all have individual interests and motivations, which makes it complex to design gamification approaches for crowds. Further, crowdsourcing systems exist in various forms and are used for various tasks and problems, thus requiring different incentive mechanisms for different crowdsourcing types. The lack of a coherent understanding of the different facets of gamified crowdsourcing systems and the lack of knowledge about the motivational and behavioral effects of applying various types of gamification features in different crowdsourcing systems inhibit us from designing solutions that harness gamification’s full potential. Further, previous research canonically uses competitive gamification, although crowdsourcing systems often strive to produce cooperative outcomes. However, the potentially relevant field of cooperative gamification has to date barely been explored. With a specific focus on these shortcomings, this dissertation presents several studies to advance the understanding of using gamification in crowdsourcing systems

Scalable and Quality-Aware Training Data Acquisition for Conversational Cognitive Services

Dialog Systems (or simply bots) have recently become a popular human-computer interface for performing user's tasks, by invoking the appropriate back-end APIs (Application Programming Interfaces) based on the user's request in natural language. Building task-oriented bots, which aim at performing real-world tasks (e.g., booking flights), has become feasible with the continuous advances in Natural Language Processing (NLP), Artificial Intelligence (AI), and the countless number of devices which allow third-party software systems to invoke their back-end APIs. Nonetheless, bot development technologies are still in their preliminary stages, with several unsolved theoretical and technical challenges stemming from the ambiguous nature of human languages. Given the richness of natural language, supervised models require a large number of user utterances paired with their corresponding tasks -- called intents. To build a bot, developers need to manually translate APIs to utterances (called canonical utterances) and paraphrase them to obtain a diverse set of utterances. Crowdsourcing has been widely used to obtain such datasets, by paraphrasing the initial utterances generated by the bot developers for each task. However, there are several unsolved issues. First, generating canonical utterances requires manual efforts, making bot development both expensive and hard to scale. Second, since crowd workers may be anonymous and are asked to provide open-ended text (paraphrases), crowdsourced paraphrases may be noisy and incorrect (not conveying the same intent as the given task). This thesis first surveys the state-of-the-art approaches for collecting large training utterances for task-oriented bots. Next, we conduct an empirical study to identify quality issues of crowdsourced utterances (e.g., grammatical errors, semantic completeness). Moreover, we propose novel approaches for identifying unqualified crowd workers and eliminating malicious workers from crowdsourcing tasks. Particularly, we propose a novel technique to promote the diversity of crowdsourced paraphrases by dynamically generating word suggestions while crowd workers are paraphrasing a particular utterance. Moreover, we propose a novel technique to automatically translate APIs to canonical utterances. Finally, we present our platform to automatically generate bots out of API specifications. We also conduct thorough experiments to validate the proposed techniques and models

Gamifying Language Resource Acquisition

PhD ThesisNatural Language Processing, is an important collection of methods for processing the vast amounts of available natural language text we continually produce. These methods make use of supervised learning, an approach that learns from large amounts of annotated data. As humans, we’re able to provide information about text that such systems can learn from. Historically, this was carried out by small groups of experts. However, this did not scale. This led to various crowdsourcing approaches being taken that used large pools of non-experts. The traditional form of crowdsourcing was to pay users small amounts of money to complete tasks. As time progressed, gamification approaches such as GWAPs, showed various benefits over the micro-payment methods used before. These included a cost saving, worker training opportunities, increased worker engagement and potential to far exceed the scale of crowdsourcing. While these were successful in domains such as image labelling, they struggled in the domain of text annotation, which wasn’t such a natural fit. Despite many challenges, there were also clearly many opportunities and benefits to applying this approach to text annotation. Many of these are demonstrated by Phrase Detectives. Based on lessons learned from Phrase Detectives and investigations into other GWAPs, in this work, we attempt to create full GWAPs for NLP, extracting the benefits of the methodology. This includes training, high quality output from non-experts and a truly game-like GWAP design that players are happy to play voluntarily

Conversational Agents in Education – A Systematic Literature Review

Conversational Agents (CAs) are widely spread in a variety of domains, such as health and customer service. There is a recent trend of increasing publications and implementations of CAs in education. We conduct a systematic literature review to identify common methodologies, pedagogical CA roles, addressed target groups, the technologies and theories behind, as well as human-like design aspects. The initially found 3329 records were systematically reduced to 252 fully coded articles. Based on the analysis of the codings, we derive further research streams. Our results reveal a research gap for long-term studies on the use of CAs in education, and there is insufficient holistic design knowledge for pedagogical CAs. Moreover, target groups other than academic students are rarely considered. We condense our findings in a morphological box and conclude that pedagogical CAs have not yet reached their full potential of long-term practical application in education

Human Behavior Experimentation and Participation in Scientific Activities in the Wild

La cooperació és un dels trets del comportament que defineixen els éssers humans, però, encara estem tractant de comprendre per què els humans cooperem. Els experiments conductuals estan dissenyats per donar llum sobre els mecanismes de cooperació i altres trets del comportament. Aquesta dissertació es centra en avançar en el camp de l'experimentació conductual utilitzant les pràctiques de ciència ciutadana, i es divideix en dos blocs. En el primer, presentem dues plataformes, una que permet estudiar com promoure el pensament científic i la participació científica, i l'altra que permet estudiar els trets del comportament humà amb un conjunt de jocs de comportament. Les dues plataformes estàn dissenyades per ajudar a crear experiments en el camp i per fomentar la participació en el marc de la ciència ciutadana. En el segon bloc avaluem les plataformes a través d'un conjunt d'experiments, i analitzem l'existència de patrons de comportament. Primer, vam estudiar la solidesa de la plataforma observant si sorgeixen estratègies iguals en rèpliques del mateix experiment realitzat amb diferents mostres de població. En el segon experiment, analitzem els patrons de comportament que emergeixen quan els participants s'enfronten a un conjunt de dilemes socials. Els dos últims experiments són "collective-risk dilemmas" sobre el canvi climàtic. En un, estudiem com les desigualtats de recursos generen comportaments injustos. L'altre es porta a terme dins d'un ecosistema concret per estudiar les tensions que hi ha entre els diferents actors del col·lectiu. Tenint en compte els resultats dels experiments, podem entendre com ens comportem quan enfrontem dilemes socials i, en conseqüència, avaluar els trets de comportament i l'aparició de patrons de comportament. Els dissenys, els resultats i la metodologia d'anàlisi presentats en aquest treball ajudaran a establir les bases per a futurs experiments de comportament al camp.La cooperación es uno de los rasgos de comportamiento que definen a los seres humanos, sin embargo, todavía estamos tratando de comprender por qué los humanos cooperamos. Los experimentos conductuales están diseñados para arrojar luz sobre los mecanismos de cooperación y otros rasgos de comportamiento. Esta disertación se centra en avanzar en el campo de la experimentación conductual utilizando las prácticas de ciencia ciudadana, y se divide en dos bloques. En el primero, presentamos dos plataformas, una que permite estudiar cómo promover el pensamiento científico y la participación científica, y otra para estudiar los rasgos del comportamiento humano con un conjunto de juegos de comportamiento. Ambas plataformas están diseñadas para ayudar a crear experimentos en el campo y para fomentar la participación en el marco de la ciencia ciudadana. En el segundo bloque evaluamos las plataformas a través de un conjunto de experimentos, y analizamos la existencia de patrones de comportamiento. Primero, estudiamos la solidez de la plataforma al observar si surgen estrategias iguales en réplicas del mismo experimento realizado con diferentes muestras de población. En el segundo experimento, analizamos los patrones de comportamiento que emergen cuando los participantes enfrentan un conjunto de dilemas sociales. Los dos últimos experimentos son "collective-risk dilemmas" sobre el cambio climático. En uno, estudiamos cómo las desigualdades de recursos generan comportamientos injustos. El otro se lleva a cabo dentro de un ecosistema concreto para estudiar las tensiones que existen entre los diferentes actores del colectivo. Teniendo en cuenta los resultados de los experimentos, podemos entender cómo nos comportamos cuando enfrentamos dilemas sociales y, en consecuencia, evaluar los rasgos de comportamiento y la aparición de patrones de comportamiento. Los diseños, los resultados y la metodología de análisis presentados en este trabajo ayudarán a establecer las bases para futuros experimentos de comportamiento en el campo.Cooperation is one of the behavioral traits that define human beings, however we are still trying to understand why humans cooperate. Behavioral experiments are designed to shed light into the mechanisms behind cooperation -- and other behavioral traits. This dissertation is focused on advancing the field of behavioral experimentation using experiments based on citizen science, and it is divided in two blocks. In the first, we present two platforms, one to understand how it can serve as a catalyst to promote of scientific thinking and engage in science, and another to study traits of human behavior with a suite of behavioral games. Both platforms are designed to help creating experiments in the wild and to encourage the participation within the framework of citizen science. In the second block we evaluate the platforms through a set of experiments, and we analyze the existence of behavioral patterns. First, we study the robustness of the platform by looking whether equal strategies emerge in replicas of the same experiment performed with different population samples. In the second experiment we analyze the behavioral patterns that emerge when participants face a set of social dilemmas. The last two experiments are collective-risk dilemmas framed in climate change. In one, we study how the resource inequalities generate unfair behaviors. The other is carried out within a given ecosystem to study the tensions that exist between actors of the collective. Considering the experiments' results, we can comprehend how we behave when we face social dilemmas, and consequently evaluate behavioral traits and the emergence of behavioral patterns. The designs, the results and the methodology of analysis presented in this work will help set the basis for future behavioral experiments in the field

Innovation through Cross-Fertilization: Serious games and gamification in the EU-funded research projects

[eng] In recent years, ecosystems of innovation have gained substantial momentum in academic research. As a response to recent calls in open innovation literature for novel research, this doctoral thesis extends the study of the process and ecosystems of innovation in projects that include serious games and gamification by considering the cross-fertilization of knowledge and technologies. Organizations’ orchestration of activities within their activity systems and transformation of their business models through innovation to realize opportunities with the objective to increase value creation are part of the topic of this dissertation. It has endeavoured to improve the understanding of how cross-fertilized alliances are formed, what their outcomes are, what causes them to generate value (or not) and what capabilities organizations need in order to successfully manage and reap value from the innovation process. For this purpose, two approaches that support innovation have been complementarily taken into account: the knowledge-technological perspective and the management perspective. These perspectives are analysed with the information retrieved from a database of 87 H2020 projects including serious games and/or gamification, 519 organizations and 597 observations. Later, in order to get more insights into the Innovation Management Strategies, a project coordinators survey was conducted. The Knowledge-Technology perspective presents how creating adequate multidisciplinary knowledge and technology is fundamental to ensuring the long-term success of an emerging technology including serious games and/or gamification, and how important is the research and innovation that takes place in the practitioners’ communities. The Management perspective presents the analysis of the innovation management strategies that boost the cross-fertilization of technologies that include serious games and/or gamification. These strategies were analysed by considering literature on innovation and network theories, absorptive capacity and dynamic capabilities. Some personal interviews were conducted with independent experts to understand and have elements for the analysis and discussion of the previous results. Findings suggest that the multidisciplinarity of a project is highly influenced by the creation of knowledge and technology. Furthermore, the management strategies boosting high levels of cross-fertilization of knowledge and technologies -including serious games and/or gamification- are principally market and customer-oriented strategies. Practical and methodological contributions from this study could enrich innovation literature from the point of view of technological and management approaches. The thesis concludes with fruitful avenues for future research.[nor] I de senere årene har økosystemer for innovasjon fått vesentlig fart i akademisk forskning. Nylig har det oppstått et større behov for mer forskning i området åpen innovasjon. Som et respons til dette, utvider denne avhandlingen seg på studier om prosesser og økosystemer av innovasjon i prosjekter. Prosjektene inkluderer seriøse spill og det som kalles for gamification. Målet er å vurdere kryss-befruktning av kunnskap og teknologi. En del av tema for denne avhandling er hvordan organisasjoner, gjennom innovasjon, virkeliggjør mulighetene for å øke verdiskapning. Dette vurderes ut ifra organisasjoners orkestrering av aktiviteter innenfor deres aktivitetssystemer og transformasjon av forretningsmodellene. Det har lenge vært forsøkt på å forbedre forståelsen av hvordan kryss-befruktet allianser dannes, hva er resultatene, hva skal til for å generere verdi (eller ikke), og hvilke evner organisasjoner trenger for å kunne forvalte og innhente verdier. På bakgrunn av dette, har to tilnærminger som støtter innovasjon, blitt komplementært tatt med i betraktningen. Disse er, den kunnskaps-teknologiske perspektiv og ledelses perspektivet. Perspektivene blir analysert med informasjon hentet fra en database med 87 H2020 prosjekter, inkludert seriøse spill og eller gamification. Det er totalt 519 organisasjoner og 597 observasjoner. I senere tid, for å få et større innblikk i strategier for innovasjonsledelse, ble det gjennomført en prosjekt koordinator undersøkelse. Det kunnskaps-teknologiske perspektivet innebærer hvordan en kan skape tilstrekkelig tverrfaglig kunnskap. Her er teknologi grunnleggende for å sikre langsiktig suksess til en fremtredende teknologi, som inkluderer seriøse spill og eller gamification, og viktigheten av forskningen og innovasjonen som fremkommer i utøvernes samfunn. På den andre siden, ledelses perspektivet inkluderer analysen av innovasjonsstrategier som har som mål å øke kryss-befruktning av teknologier som inkluderer seriøse spill og eller gamification. Strategiene ble analysert ved å vurdere innovasjon- og nettverks teorier, evnen til å absorbere, og dynamisk kapasitet litteratur. For å forstå og samle elementer for analysen og diskusjonen av tidligere resultater, bestemte personlige intervjuer ble gjennomført med uavhengige eksperter. Funnene viser at flerfaglighet av et prosjekt er sterkt påvirket av etablering av kunnskap og teknologi. Videre, ledelses strategier er med på å øke nivåer av kryss-befruktning av kunnskap og teknologi. Dette inkluderer seriøse spill og eller gamification, men hovedsakelig markeds- og kundeorienterte strategier. De praktiske og metodologiske bidrag fra denne studien er med på å berike innovasjons litteratur fra det teknologiske og det ledelsesmessig synspunkt. Avhandlingen avsluttes med fruktbare veier for fremtidig forskning.[cat] Al llarg dels darrers anys, els ecosistemes d’innovació han pres un impuls substancial en la recerca acadèmica. Com a resposta a les darreres crides a noves recerques en la literatura sobre innovació oberta, aquesta tesi doctoral amplia l’estudi del procés i els ecosistemes d’innovació en projectes que inclouen jocs seriosos i gamificació al considerar la fertilització creuada de coneixement i tecnologies. L’orquestració d’activitats per part de les organitzacions dins dels seus sistemes d’activitats i la transformació dels seus models comercials a través de la innovació per generar oportunitats amb l’objectiu d’augmentar la creació de valor són part dels temes d’aquesta tesi. Hi ha hagut un esforç per millorar la comprensió de com es formen aliances amb la fertilització creuada, quins són els seus resultats, què fa que generin valor (o no) i quines capacitats necessiten les organitzacions per gestionar i assolir valor a través del procés d’innovació. Amb aquest propòsit, dos enfocaments que recolzen la innovació s’han tingut en compte de manera complementària: la perspectiva coneixement-tecnologia i la perspectiva de gestió. Aquestes perspectives s’analitzen amb la informació obtinguda d’una base de dades de 87 projectes H2020 que inclouen jocs seriosos i/o gamificació, 519 organitzacions i 597 observacions. Posteriorment, amb l’objectiu d’obtenir informació addicional sobre les Estratègies de Gestió de la Innovació, es va realitzar una enquesta adreçada als coordinadors dels projectes. La perspectiva de Coneixement i Tecnologia mostra com la creació adequada de coneixement i tecnologia multidisciplinaris és fonamental per garantir l’èxit a llarg termini d’una tecnologia emergent, que inclogui els jocs seriosos i la gamificació, i com d’important és la recerca i la innovació que té lloc en les comunitats de professionals. La perspectiva de Gestió mostra l’anàlisi de les estratègies de gestió de la innovació que fomenten la fertilització creuada de tecnologies que inclouen jocs seriosos i/o gamificació. Aquestes estratègies es van analitzar a partir de la literatura en innovació i teories de xarxes, capacitat d’absorció i capacitats dinàmiques. També, es van dur a terme entrevistes personals amb experts independents per comprendre i tenir elements per a l’anàlisi i la discussió dels resultats anteriors. Els resultats suggereixen que la multidisciplinarietat d’un projecte està molt influenciada per la creació de coneixement i tecnologia. A més, les estratègies de gestió que impulsen els alts nivells de fertilització creuada de coneixement i tecnologies –inclosos els jocs seriosos i/o la gamificació- són principalment estratègies orientades al mercat i al client. Les contribucions pràctiques i metodològiques d’aquest estudi podrien enriquir la literatura sobre innovació des del punt de vista dels enfocaments tecnològics i de gestió. La tesi conclou amb suggeriments de línies de recerca futures.[spa] Durante los últimos años, los ecosistemas de innovación han tomado un impulso sustancial en la investigación académica. Como respuesta a las recientes llamadas a nuevas investigaciones en la literatura sobre innovación abierta, esta tesis doctoral amplía el estudio del proceso y los ecosistemas de innovación en proyectos que incluyen juegos serios y gamificación al considerar la fertilización cruzada de conocimiento y tecnologías. La orquestación de actividades por parte de las organizaciones dentro de sus sistemas de actividades y la transformación de sus modelos comerciales a través de la innovación para generar oportunidades con el objetivo de aumentar la creación de valor son parte de los temas de esta tesis. Ha habido un esfuerzo por mejorar la comprensión de cómo se forman alianzas con fertilización cruzada, cuáles son sus resultados, qué hace que generen valor (o no) y qué capacidades necesitan las organizaciones para gestionar y cosechar valor a través del proceso de innovación. Con este propósito, dos enfoques que apoyan la innovación se han tenido en cuenta de manera complementaria: la perspectiva conocimiento-tecnología y la perspectiva de gestión. Estas perspectivas se analizan con la información obtenida de una base de datos de 87 proyectos H2020 que incluyen juegos serios y/o gamificación, 519 organizaciones y 597 observaciones. Posteriormente, con el objetivo de obtener información adicional sobre las Estrategias de Gestión de la Innovación, se realizó una encuesta a los coordinadores de los proyectos. La perspectiva de Conocimiento y Tecnología muestra cómo la creación adecuada de conocimiento y tecnología multidisciplinarios es fundamental para garantizar el éxito a largo plazo de una tecnología emergente, que incluya los juegos serios y la gamificación, y cómo de importante es la investigación y la innovación en las comunidades de profesionales. La perspectiva de Gestión muestra el análisis de las estrategias de gestión de la innovación que fomentan la fertilización cruzada de tecnologías que incluyen juegos serios y/o gamificación. Estas estrategias se analizaron a partir de la literatura en innovación y teorías de redes, la capacidad de absorción y las capacidades dinámicas. También, se realizaron entrevistas personales con expertos independientes para comprender y tener elementos para el análisis y la discusión de los resultados anteriores. Los hallazgos sugieren que la multidisciplinariedad de un proyecto está muy influenciada por la creación de conocimiento y tecnología. Además, las estrategias de gestión que impulsan los altos niveles de fertilización cruzada de conocimientos y tecnologías -incluidos los juegos serios y/o la gamificación- son principalmente estrategias orientadas al mercado y al cliente. Las contribuciones prácticas y metodológicas de este estudio podrían enriquecer la literatura sobre innovación desde el punto de vista de los enfoques tecnológicos y de gestión. La tesis concluye con sugerencias de líneas de investigación futuras