Conversational Browsing

How can we better understand the mechanisms behind multi-turn information seeking dialogues? How can we use these insights to design a dialogue system that does not require explicit query formulation upfront as in question answering? To answer these questions, we collected observations of human participants performing a similar task to obtain inspiration for the system design. Then, we studied the structure of conversations that occurred in these settings and used the resulting insights to develop a grounded theory, design and evaluate a first system prototype. Evaluation results show that our approach is effective and can complement query-based information retrieval approaches. We contribute new insights about information-seeking behavior by analyzing and providing automated support for a type of information-seeking strategy that is effective when the clarity of the information need and familiarity with the collection content are low

Planning Technologies for Interactive Storytelling

Since AI planning was first proposed for the task of narrative generation in interactive storytelling (IS), it has emerged as the dominant approach in this field. This chapter traces the use of planning technologies in this area, considers the core issues involved in the application of planning technologies in IS, and identifies some of the remaining challenges

Cultural Heritage Storytelling, Engagement and Management in the Era of Big Data and the Semantic Web

The current Special Issue launched with the aim of further enlightening important CH areas, inviting researchers to submit original/featured multidisciplinary research works related to heritage crowdsourcing, documentation, management, authoring, storytelling, and dissemination. Audience engagement is considered very important at both sites of the CH production–consumption chain (i.e., push and pull ends). At the same time, sustainability factors are placed at the center of the envisioned analysis. A total of eleven (11) contributions were finally published within this Special Issue, enlightening various aspects of contemporary heritage strategies placed in today’s ubiquitous society. The finally published papers are related but not limited to the following multidisciplinary topics:Digital storytelling for cultural heritage;Audience engagement in cultural heritage;Sustainability impact indicators of cultural heritage;Cultural heritage digitization, organization, and management;Collaborative cultural heritage archiving, dissemination, and management;Cultural heritage communication and education for sustainable development;Semantic services of cultural heritage;Big data of cultural heritage;Smart systems for Historical cities – smart cities;Smart systems for cultural heritage sustainability

DATA-DRIVEN STORYTELLING FOR CASUAL USERS

Today’s overwhelming volume of data has made effective analysis virtually inaccessible for the general public. The emerging practice of data-driven storytelling is addressing this by framing data using familiar mechanisms such as slideshows, videos, and comics to make even highly complex phenomena understandable. However, current data stories still do not utilize the full potential of the storytelling domain. One reason for this is that current data-driven storytelling practice does not leverage the full repertoire of media that can be used for storytelling, such as speech, e-learning, and video games. In this dissertation, we propose a taxonomy focused specifically on media types for the purpose of widening the purview of data-driven storytelling by putting more tools in the hands of designers. We expand the idea of data-driven storytelling into the group of casual users, who are the consumers of information and non-professionals with limited time, skills, and motivation , to bridge the data gap between the advanced data analytics tools and everyday internet users. To prove the effectiveness and the wide acceptance of our taxonomy and data-driven storytelling among the casual users, we have collected examples for data-driven storytelling by finding, reviewing, and classifying ninety-one examples. Using our taxonomy as a generative tool, we also explored two novel storytelling mechanisms, including live-streaming analytics videos—DataTV—and sequential art (comics) that dynamically incorporates visual representations—Data Comics. Meanwhile, we widened the genres we explored to fill the gaps in the literature. We also evaluated Data Comics and DataTV with user studies and expert reviews. The results show that Data Comics facilitates data-driven storytelling in terms of inviting reading, aiding memory, and viewing as a story. The results also show that an integrated system as DataTV encourages authors to create and present data stories

Abstract visualization of large-scale time-varying data

The explosion of large-scale time-varying datasets has created critical challenges for scientists to study and digest. One core problem for visualization is to develop effective approaches that can be used to study various data features and temporal relationships among large-scale time-varying datasets. In this dissertation, we first present two abstract visualization approaches to visualizing and analyzing time-varying datasets. The first approach visualizes time-varying datasets with succinct lines to represent temporal relationships of the datasets. A time line visualizes time steps as points and temporal sequence as a line. They are generated by sampling the distributions of virtual words across time to study temporal features. The key idea of time line is to encode various data properties with virtual words. We apply virtual words to characterize feature points and use their distribution statistics to measure temporal relationships. The second approach is ensemble visualization, which provides a highly abstract platform for visualizing an ensemble of datasets. Both approaches can be used for exploration, analysis, and demonstration purposes. The second component of this dissertation is an animated visualization approach to study dramatic temporal changes. Animation has been widely used to show trends, dynamic features and transitions in scientific simulations, while animated visualization is new. We present an automatic animation generation approach that simulates the composition and transition of storytelling techniques and synthesizes animations to describe various event features. We also extend the concept of animated visualization to non-traditional time-varying datasets--network protocols--for visualizing key information in abstract sequences. We have evaluated the effectiveness of our animated visualization with a formal user study and demonstrated the advantages of animated visualization for studying time-varying datasets

Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation

This paper surveys the current state of the art in Natural Language Generation (NLG), defined as the task of generating text or speech from non-linguistic input. A survey of NLG is timely in view of the changes that the field has undergone over the past decade or so, especially in relation to new (usually data-driven) methods, as well as new applications of NLG technology. This survey therefore aims to (a) give an up-to-date synthesis of research on the core tasks in NLG and the architectures adopted in which such tasks are organised; (b) highlight a number of relatively recent research topics that have arisen partly as a result of growing synergies between NLG and other areas of artificial intelligence; (c) draw attention to the challenges in NLG evaluation, relating them to similar challenges faced in other areas of Natural Language Processing, with an emphasis on different evaluation methods and the relationships between them.Comment: Published in Journal of AI Research (JAIR), volume 61, pp 75-170. 118 pages, 8 figures, 1 tabl

Auditory and haptic feedback to train basic mathematical skills of children with visual impairments

Physical manipulatives, such as rods or tiles, are widely used for mathematics learning, as they support embodied cognition, enable the execution of epistemic actions, and foster conceptual metaphors. Counting them, children explore, rearrange, and reinterpret the environment through the haptic channel. Vision generally complements physical actions, which makes using traditional manipulatives limited for children with visual impairments (VIs). Digitally augmenting manipulatives with feedback through alternative modalities might improve them. We specifically discuss conveying number representations to children with VIs using haptic and auditory channels within an environment encouraging exploration and supporting active touch counting strategies while promoting reflection. This paper presents LETSMath, a tangible system for training basic mathematical skills of children with VIs, developed through Design-Based Research with three iterations in which we involved 19 children with VIs and their educators. We discuss how the system may support training skills in the composition of numbers and the impact that the different system features have on slowing down the interaction pace to trigger reflection, in understanding, and in incorporation.Universitat Pompeu Fabra (Spain) through MIREGAMIS: 2018 LLAV 00009Agencia Nacional de Investigación e Innovación - ANIIFundación CeibalCentro Interdisciplinario en Cognición para la Enseñanza y el Aprendizaje - CICEA, Universidad de la RepúblicaUniversitat Oberta de Catalunya (Spain) through Ministry of Science, Innovation, and Universities IJCI-2017-32162LASIGE Research Unit (Portugal) through FCT project mIDR (AAC02/SAICT/-2017, project 30347, cofunded by COMPETE/FEDER/FNR), the LASIGE Research Unit, ref. UIDB/00408/2020 and ref. UIDP/00408/2020