Interactive Digital Storytelling: Towards a Hybrid Conceptual Approach

1 Introduction In this contribution, Interactive Digital Storytelling is viewed as a hybrid form of game design and cinematic storytelling for the understanding and making of future learning and entertainment applications. The paper shall present formal design models that provide a conceptual bridge between both traditional linear narrative techniques as well as agent-based emergent conversations with virtual characters. In summary, a theoretical classification of thinking models for authors and interactive experiences for users will be presented. The conceptual work is based upon practical experiments within several research projects on edutainment, which employ conversations with virtual characters to convey information and to entertain. By building several prototypes, two different approaches where explored to combine plot-based interactive storytelling with character-based emergent conversations. Visual impressions of the examples are shown in Fig. 1 and will be explained in more detail in the full paper. In both examples, several virtual animated characters converse digitally with each other and with a user who mainly types text with the keyboard, optionally complemented by choice functionality and special hardware interfaces. The resulting conversations differ in their direction of approaching a middle ground between predefined narrative presentations and emergent conversations with a user, by combining emerging chatterbot dialogues with a story structure. The user experiences a semi-autonomous behaviour of interacting agents. This paper is not about the difference between stories and games. The motivation is on the potential of both to offer structures for learning and entertainment. Instead of trying to draw a distinct line between them, conceptual models for authors have to be defined, who are responsible to flesh out a suitable design within a variability of forms. Design elements include aspects of drama and filmmaking, dialogue design, as well as game design and game tuning. The actual challenge for the design of learning applications with conversational agents is the necessity that authors have to take on responsibility concerning the intended outcome and effect. In fact, they have to balance the bias between a pre-structured storyline (and possibly a timeline) which they may have strictly defined, and the agency that users shall experience through the design of the author. However, there is no one-dimensional borderline between both. In the following, the paper presents a model with several levels which shall help to form a more differentiated picture. 2 Conceptual Models for Storytelling and Agency In Fig. 2, a traditional modus operandi is sketched at four abstract levels. The distinction between levels may vary from project to project. The four levels were found to be suitable for the addition of interaction at each of the levels to form a classification of genres. On the top level of highest story abstraction, the overall dramatic outline is sketched. For example, there may be a hero’s journey in 3 acts, or a Propp model. Further, authors break down the story into scenes which are handled at the next level. Each scene will be defined by a scene script. Within a scene, dialogues and interactions between actors are defined, and lead to stage directions. If producing for an animated film, these directions are strictly mapped onto virtual actors by a skilled animator, who defines the way the virtual actors move and behave. When storytelling gets interactive, the user can influence the storytelling. In fact, in games as well as in constructivist scenarios for learning, users need to experience agency within a story. However, there are different levels at which to affect the outcome. In Fig. 3, the first author model (compare Fig. 2) has been extrapolated according to the need of introducing agency at each level. Opposed to the author, a participant is modelled who now may contribute to each level. The first implication for the author is that it’s not enough to just model a database of descriptions, but to add rules and models, which control an autonomous behaviour at each level in reaction to the participants. Then, it is possible to think of gradations of granted agency versus authored determination. Within Fig. 3 this is indicated by the sliders between control and autonomy at each level. The levels rather represent conceptual stages for authoring than elements of software architecture, though there are parallels to architectures of a number of existing systems of game and story engines. Semi-autonomy occurs on the edge between predefined factual information and rules for each level. The more rules on one level, the more agency can be experienced by potentially affecting the respective level. For example: It is imaginable that participants only experience agency on the lowest level, as a feeling of presence in a scenario. In this case, everything is predefined, but avatars would still react with nonverbal cues to the visitor and recognize her, comparable to a virtual cursor that shows a live status. At the conversation level, participants can for example have agency in an entertaining and informative chatbot dialogue with the characters. They may even not be able to affect anything in the story logic, but participate at dialogue level with speech acts. Agency at scene level would mean to have real choices about the outcome of a scene, for example, the story of the game would have to change according to user’s actions. On the top level, players would influence the whole story of the application, if the "agency slider" would be at a 100% to the right. For example, a simulation such as "The Sims" (Electronic Arts) can be put into the classification here. For factual knowledge transfer in a didactic lesson situation, the highest level could stay predefined, while the lower levels allow for conversational interaction, however constrained. If authors only provide a rule base with little pre-scripted structuring, they achieve a conceptual model more like an exploration or gaming experience depending completely on the action of the player. While arranging the bias at each level to various slider positions, several abstract genres of Interactive Digital Storytelling can be rebuilt in the model, which helps to specify exactly what kind of user experience an application shall provide. It is a conceptual model that can be used to classify story-related games, and it particularly supports authors coming from linear media, stepping into interactive storytelling. 3 Further Work In the full paper, I will also tackle related work while comparing with other theoretical models between games and stories, including references of the taxonomies of C. Lindley, M. Leblanc, J. Klabbers, B. Laurel, C. Pearce, and traditional classifications such as of R. Caillois. I will give more examples how existing products of Interactive Storytelling fit into the classification, and raise the question if new genres have to be defined particularly for Interactive Digital Storytelling. Literature Caillois, Roger: Man, Play and Games. (orig.: Les Jeux Et Les Hommes 1958) University of Illinois Press, Reprint (2001) Hunicke, Robin; LeBlanc, Marc; Zubek, Robert: MDA: A Formal Approach to Game Design and Game Research. In: Workshop Proceedings: Challenges in Game AI. 19th National Conference on Artificial Intelligence AAAI (2004) Klabbers, Jan H.G.: The gaming landscape: A taxonomy for classifying games and simulations. In Copier & Raessens (Eds.) Level up: Digital Games Research Conference. Utrecht University (2003) Lindley, Craig: Narrative, Game Play and Alternative Time Structures for Virtual Environments. In: Proc. TIDSE 2004, Technologies for Interactive Digital Storytelling and Entertainment, Darmstadt, Springer LNCS vol. 3105, 2004 Pearce, Celia: Emergent authorship: The next interactive revolution. In: Computers & Graphics 26, p. 21-29 (2002) Spierling, U.: Conceptual Models for Interactive Digital Storytelling in Knowledge Media Applications. In: Proc. TIDSE 2004, Technologies for Interactive Digital Storytelling and Entertainment, Darmstadt, Springer LNCS vol. 3105, 200

From Personalization to Adaptivity: Creating Immersive Visits through Interactive Digital Storytelling at the Acropolis Museum

Storytelling has recently become a popular way to guide museum visitors, replacing traditional exhibit-centric descriptions by story-centric cohesive narrations with references to the exhibits and multimedia content. This work presents the fundamental elements of the CHESS project approach, the goal of which is to provide adaptive, personalized, interactive storytelling for museum visits. We shortly present the CHESS project and its background, we detail the proposed storytelling and user models, we describe the provided functionality and we outline the main tools and mechanisms employed. Finally, we present the preliminary results of a recent evaluation study that are informing several directions for future work

Virtual reality in theatre education and design practice - new developments and applications

The global use of Information and Communication Technologies (ICTs) has already established new approaches to theatre education and research, shifting traditional methods of knowledge delivery towards a more visually enhanced experience, which is especially important for teaching scenography. In this paper, I examine the role of multimedia within the field of theatre studies, with particular focus on the theory and practice of theatre design and education. I discuss various IT applications that have transformed the way we experience, learn and co-create our cultural heritage. I explore a suite of rapidly developing communication and computer-visualization techniques that enable reciprocal exchange between students, theatre performances and artefacts. Eventually, I analyse novel technology-mediated teaching techniques that attempt to provide a new media platform for visually enhanced information transfer. My findings indicate that the recent developments in the personalization of knowledge delivery, and also in student-centred study and e-learning, necessitate the transformation of the learners from passive consumers of digital products to active and creative participants in the learning experience

Smart Conversational Agents for Reminiscence

In this paper we describe the requirements and early system design for a smart conversational agent that can assist older adults in the reminiscence process. The practice of reminiscence has well documented benefits for the mental, social and emotional well-being of older adults. However, the technology support, valuable in many different ways, is still limited in terms of need of co-located human presence, data collection capabilities, and ability to support sustained engagement, thus missing key opportunities to improve care practices, facilitate social interactions, and bring the reminiscence practice closer to those with less opportunities to engage in co-located sessions with a (trained) companion. We discuss conversational agents and cognitive services as the platform for building the next generation of reminiscence applications, and introduce the concept application of a smart reminiscence agent

Machinima interventions: innovative approaches to immersive virtual world curriculum integration

The educational value of Immersive Virtual Worlds (IVWs) seems to be in their social immersive qualities and as an accessible simulation technology. In contrast to these synchronous applications this paper discusses the use of educational machinima developed in IVW virtual film sets. It also introduces the concept of media intervention, proposing that digital media works best when simply developed for deployment within a blended curriculum to inform learning activity, and where the media are specifically designed to set challenges, seed ideas, or illustrate problems. Machinima, digital films created in IVWs, or digital games offer a rich mechanism for delivering such interventions. Scenes are storyboarded, constructed, shot and edited using techniques similar to professional film production, drawing upon a cast of virtual world avatars controlled through a human–computer interface, rather than showing real‐life actors. The approach enables academics or students to make films using screen capture software and desktop editing tools. In student‐generated production models the learning value may be found in the production process itself. This paper discusses six case studies and several themes from research on ideas for educational machinima including: access to production; creativity in teaching and learning; media intervention methodology; production models; reusability; visualisation and simulation

‘IMPLICIT CREATION’ – NON-PROGRAMMER CONCEPTUAL MODELS FOR AUTHORING IN INTERACTIVE DIGITAL STORYTELLING

Interactive Digital Storytelling (IDS) constitutes a research field that emerged from several areas of art, creation and computer science. It inquires technologies and possible artefacts that allow ‘highly-interactive’ experiences of digital worlds with compelling stories. However, the situation for story creators approaching ‘highly-interactive’ storytelling is complex. There is a gap between the available technology, which requires programming and prior knowledge in Artificial Intelligence, and established models of storytelling, which are too linear to have the potential to be highly interactive. This thesis reports on research that lays the ground for bridging this gap, leading to novel creation philosophies in future work. A design research process has been pursued, which centred on the suggestion of conceptual models, explaining a) process structures of interdisciplinary development, b) interactive story structures including the user of the interactive story system, and c) the positioning of human authors within semi-automated creative processes. By means of ‘implicit creation’, storytelling and modelling of simulated worlds are reconciled. The conceptual models are informed by exhaustive literature review in established neighbouring disciplines. These are a) creative principles in different storytelling domains, such as screenwriting, video game writing, role playing and improvisational theatre, b) narratological studies of story grammars and structures, and c) principles of designing interactive systems, in the areas of basic HCI design and models, discourse analysis in conversational systems, as well as game- and simulation design. In a case study of artefact building, the initial models have been put into practice, evaluated and extended. These artefacts are a) a conceived authoring tool (‘Scenejo’) for the creation of digital conversational stories, and b) the development of a serious game (‘The Killer Phrase Game’) as an application development. The study demonstrates how starting out from linear storytelling, iterative steps of ‘implicit creation’ can lead to more variability and interactivity in the designed interactive story. In the concrete case, the steps included abstraction of dialogues into conditional actions, and creating a dynamic world model of the conversation. This process and artefact can be used as a model illustrating non-programmer approaches to ‘implicit creation’ in a learning process. Research demonstrates that the field of Interactive Digital Storytelling still has to be further advanced until general creative principles can be fully established, which is a long-term endeavour, dependent upon environmental factors. It also requires further technological developments. The gap is not yet closed, but it can be better explained. The research results build groundwork for education of prospective authors. Concluding the thesis, IDS-specific creative principles have been proposed for evaluation in future work

Investigating situated cultural practices through cross-sectoral digital collaborations: policies, processes, insights

The (Belfast) Good Friday Agreement represents a major milestone in Northern Ireland's recent political history, with complex conditions allowing for formation of a ‘cross-community’ system of government enabling power sharing between parties representing Protestant/loyalist and Catholic/nationalist constituencies. This article examines the apparent flourishing of community-focused digital practices over the subsequent ‘post-conflict’ decade, galvanised by Northern Irish and EU policy initiatives armed with consolidating the peace process. Numerous digital heritage and storytelling projects have been catalysed within programmes aiming to foster social processes, community cohesion and cross-community exchange. The article outlines two projects—‘digital memory boxes’ and ‘interactive galleon’—developed during 2007–2008 within practice-led PhD enquiry conducted in collaboration with the Nerve Centre, a third-sector media education organisation. The article goes on to critically examine the processes involved in practically realising, and creatively and theoretically reconciling, community-engaged digital production in a particular socio-political context of academic-community collaboration

A pedagogical model for science education through blended learning

Comunicação publicada nas atas da conferência EADTU 2015This paper proposes a framework to support science education through blended learning, based on a participatory and interactive approach supported by ICT-based tools, called Science Learning Activities Model (SLAM). The study constitutes a work in progress and started as a response to complex societal developments such as a changing labour market, high turnover rate of knowledge, and use of technology as a natural part of daily life activities. Another concern was the emergence of new challenges in education, like learning in various authentic contexts and in collaboration with others, in ways that influence the circumstances learners live in. Many of these challenges are related to science and it would be expected that students were interested in science, however the contrary is the case. So, after reviewing the relevant literature and the current trend towards a learner-centred approach, we contend that there is a need to provide a model with which teachers can design science courses with high motivational impact on students. By using today’s flexible, interactive and immersive technologies (mobile, AR, VR) combined with the appropriate pedagogies, we believe it is possible to have students more motivated in science areas (STEM), and expect a more creative response to the world problems that surround them