Wayang Authoring: A Web-based Authoring Tool for Visual Storytelling for Children

This research focuses on the development of the Wayang Authoring tool as it aims to assist children in creating and performing stories, developing an appreciation for cultural artifacts, and enhancing intercultural empathy while building a young storyteller community in a virtual world. This study seeks a framework of interaction design of an authoring media which is appropriate to support children s narrative development. The concept of the tool is based on the narrative element of the ancient Indonesian art form wayang, a traditional two dimensional shadow puppet theater. To understand the user s requirements and the cultural dimension, children and professional story performers who use wayang have been involved in the design process. In order to evaluate the tool, several workshops have been conducted with children from different cultural backgrounds as well as with their teachers. Wayang Authoring is composed of three elements: the imagination-building element, the creative acting element and the social interaction element. Children take existing materials as an inspiration tool, imagine what they themselves want to tell, create a story based on their own ideas, play with their creations, share their stories and creations with others, and reflect on their experiences at the end. This virtual creative production tool is expected to provide a space for young people to change their role from a simple user to a (co-)creator in the virtual and narrative worlds. The core contributions are in the field of web technology for storytelling. The uses of web-based authoring media enable children to put themselves into the process of developing stories. When they are connecting stories, they are connected and immersed with other children as well. They have to act and play by themselves or with others within the stories in order to experience the narratives. They train to have the skills to interact, to share their ideas and to collaborate constructively. This makes it possible for them to participate in today s media-driven culture. This research found that a better understanding of how stories are crafted and brought to life in a performance tradition offers a better design of interaction of an authoring media. The handling of cultural artifacts supports the ability to understand different cultural codes and to pursue the learning process surrounding the original culture behind these artifacts

Transformative interventions. An ecological-enactive approach to art practices

Starting from an ecological-enactive approach to human cognition (Rietveld, Kiverstein 2014) I have articulated a series of transformative interventions whose purpose is to explore how art practices can reorganize our form of life (Noë, 2015; Rietveld, 2019). To do this, I discuss how a plethora of heterogeneous tools traceable in the performing arts, such as masks, puppets, and hybrid costumes, can help us, through what I call monstrous practices, to explore imaginative dimensions that our own bodies "cannot afford." This is the core of the transformative chain that I will define monster-monstrous-Monster: we feed imaginative “monsters” to become “monstrous”– that is, to pool and cross-fertilize our abilities – to confront the "Monsters" in our lives. My main interest is in analyzing how it is possible to create or collect new affordances so as to transfigure one's repertoire of possibilities and transform a shared practice. Each transformative intervention is not only defined through written words but is also developed through unorthodox sociomaterial invitations, usually not used in philosophical practice: storyboards, visual ethnographies, performance projects, and installations, which I will define more properly through an enriched notion of real-life thinking model (Rietveld; RAAAF)

Abstracts: HASTAC 2017: The Possible Worlds of Digital Humanities

The document contains abstracts for HASTAC 2017

Designing interactive technology for cross-cultural appreciation of intangible cultural heritage: Chinese traditional painting and puppetry

Ph. D. ThesisDigital heritage is becoming a significant component of cultural heritage, and cultural organisations are increasingly using interactive technologies to showcase and safeguard heritage assets. However, few studies focus on using interactive technology to enhance the appreciation of Intangible Cultural Heritage (ICH) amongst cross-cultural audiences. This dissertation explores the design of interactive technologies to support the cultural appreciation, learning, and experience of Chinese ICH. In addition, the research seeks to explore the value of Human-Computer Interaction (HCI) design strategies in supporting the appreciation of ICH. The research uses HCI design strategies to specifically explore how interactive technology might be effectively utilised in two case-study contexts, supporting traditional Chinese painting and traditional Chinese puppetry. To this end, in stage one of the research, a qualitative study involving interviews, workshops, and fieldwork for design was undertaken with potential cross-cultural audiences and both Chinese and international painting and puppetry practitioners. Based on the results of these studies, several suggestions were developed for safeguarding ICH across cultural boundaries. In the next stage of the research, two interactive applications were designed and deployed that supported cross-cultural audiences’ appreciation of traditional of ICH. One application explored Chinese painting, the other Chinese puppetry. Using both qualitative and quantitative methods, studies were conducted that examined the efficacy of both applications and offered suggestions for a holistic approach to cross-cultural appreciation through the use of interactive applications. The analysis focuses on the use of element-based archiving to increase aesthetic appreciation, gestural/tangible interfaces for cultural engagement, and the use of interactive access to inspire self-expression and collaborative appreciation. Finally, this research relies on practical methods to deconstruct cultural elements from the HCI perspective and enhance the cross-cultural appreciation of Chinese ICH. It thus provides a framework for assisting non-Chinese people to better understand the cultural significance of Chinese ICH. The findings have design implications for both HCI researchers and digital heritage researchers

Shadows, touch and digital puppeteering: a media archaeological approach

Aims The practical aim of this research project is to create a multi-touch digital puppetry system that simulates shadow theatre environments and translates gestural acts of touch into live and expressive control of virtual shadow figures. The research is focussed on the qualities of movement achievable through the haptics of single and multi-touch control of the digital puppets in the simulation. An associated aim is to create a collaborative environment where multiple performers can control dynamic animation and scenography, and create novel visualisations and narratives. The conceptual aim is to link traditional and new forms of puppetry seeking cultural significance in the ‘remediation’ of old forms that avail themselves of new haptic resources and collaborative interfaces. The thesis evaluates related prior art where traditional worlds of shadow performance meet new media, digital projection and 3D simulation, in order to investigate how changing technical contexts transform the potential of shadows as an expressive medium. Methodology The thesis uses cultural analysis of relevant documentary material to contextualise the practical work by relating the media archaeology of 2D puppetry—shadows, shadowgraphs and silhouettes—to landmark work in real-time computer graphics and performance animation. The survey considers the work of puppeteers, animators, computer graphics specialists and media artists. Through practice and an experimental approach to critical digital creativity, the study provides practical evidence of multiple iterations of controllable physics-based animation delivering expressive puppet motion through touch and multiuser interaction. Video sequences of puppet movement and written observational analysis document the intangible aspects of animation in performance. Through re-animation of archival shadow puppets, the study presents an emerging artistic media archaeological method. The major element of this method has been the restoration of a collection of Turkish Karagöz Shadow puppets from the Institut International de la Marionnette (Charleville, France) into a playable digital form. Results The thesis presents a developing creative and analytical framework for digital shadow puppetry. It proposes a media archaeological method for working creatively with puppet archives that unlock the kinetic and expressive potential of restored figures. The interaction design introduces novel approaches to puppetry control systems—using spring networks—with objects under physics-simulation that demonstrate emergent expressive qualities. The system facilitates a dance of agency¹ between puppeteer and digital instrument. The practical elements have produced several software iterations and a tool-kit for generating elegant, nuanced multi-touch shadow puppetry. The study presents accidental discoveries—serendipitous benefits of open-ended practical exploration. For instance: the extensible nature of the control system means novel input—other than touch—can provide exciting potential for accessible user interaction, e.g. with gaze duration and eye direction. The study also identifies limitations including the rate of software change and obsolescence, the scope of physics-based animation and failures of simulation. Originality/value The work has historical value in that it documents and begins a media archaeology of digital puppetry, an animated phenomenon of increasing academic and commercial interest. The work is of artistic value providing an interactive approach to making digital performance from archival material in the domain of shadow theatre. The work contributes to the electronic heritage of existing puppetry collections. The study establishes a survey of digital puppetry, setting a research agenda for future studies. Work may proceed to digitise, rig and create collaborative and web-mediated touch-based motion control systems for 2D and 3D puppets. The present study thus provides a solid platform to restore past performances and create new work from old, near forgotten-forms. ¹ Following Andrew Pickering, puppetry is ‘a temporally extended back-and-forth dance of human and non-human agency in which activity and passivity on both sides are reciprocally intertwined’ PICKERING, A. 2010. Material Culture and the Dance of Agency. In: BEAUDRY, M. C. & HICKS, D. (eds.) Oxford Handbook of Material Culture Studies. Oxford University Press.

The effectiveness of computer based interactive oral health education

The Western Isles of Scotland have historically high levels of dental disease in the five year old age group amongst the worst in the UK. The “Action Plan for Scotland” has implemented a multidisciplinary approach to deal with this problem. This includes a major role for schools in supporting and improving oral health, by reducing the availability of cariogenic produce in schools and actively promoting healthier diets. In light of this the researcher created an interactive computer programme, designed to educate children about healthy eating and improve their ability to identify cariogenic foods. The interactive computer programme was designed to integrate into the school curriculum providing a combined teaching tool and learning resource; for elements of both the health curriculum and IT attainment targets. To assess the efficacy of the interactive computer programme a blind randomised controlled trial was designed to measure: • Its ability to teach children the difference between healthy and unhealthy food. • If it could positively influence the children’s selection of playtime snack. The computer programme was initially assessed by a peer group consisting of Primary School Teachers, Dental staff (Glasgow University Dental School) and Dieticians (Western Isles Health Board). This was to ensure the content contained the correct nutritional and oral health message and that the interactive computer programme was educationally appropriate, for the age group within the study. The computer programme was then assessed by a user group, consisting of pupils from Sandwick Hill Primary School, aged from four and a half to seven. Changes were then made in relation to the format and content of the programme to improve and refine it. An initial pilot study was undertaken within Sandwick Hill Primary School to assess the methodology of the controlled trial and the randomisation and blinding of the participants. This also allowed refinement of the assessment tool to be used within the study. The assessment tool was designed to determine the children’s ability to identify healthy and unhealthy foods and to record their playtime snack. Two schools were involved in the controlled trial, Stornoway Primary School and Laxdale Primary School. Positive consent was received for Eighty-six pupils in total. There were forty five boys (52.3%) and forty one girls (47.7%). The mean age was 5.7, (range 4 to 7 years). The teaching staff involved within the study were given a tutorial to explain the use of the programme and the protocols relating to randomisation and blinding. The participants were then randomly allocated to one of two groups, the intervention or control group. Both groups were then assessed to provide a comparative baseline. The intervention group were provided with the interactive computer programme. They were to use the programme for fifteen minutes a time over three weeks. The teachers were encouraged to allow the children to access the programme at least five to six times during this period. The control group were provided with traditional paper based educational material which was completed during class time. After three weeks the children were reassessed and the educational materials removed. The children were then assessed again after three months to assess longevity and retention of the acquired knowledge. The researcher remained blind to group allocation until the key was broken after analysis of the results. Regarding identification of healthy food, regression analysis showed significant improvement in both groups, but t-tests revealed no significant difference between them. The groups matched well at baseline [Two- Sample T-test for means, p=0.979 95% CI -4.88, 4.76]; the intervention group showed greater improvement at 3 weeks but this was not significant [Two- Sample T-test for means, p= 0.135 95% CI -7.56, 1.04]. There was no difference seen at 3 months [Two- Sample T-test for means. P= 0.547, 95% CI -5.12, 2.74]. There was neither an improvement nor a difference between the two groups in snack selection. This study provides evidence as to the effectiveness of interactive technology in relation to oral health education. It shows that interactive computer technology can provide an alternative to paper based educational materials. This study does not however show it to be significantly more effective. The study also shows that the use of the interactive computer programme was ineffective in modifying behaviour, in relation to diet, in this age group

Using Tangible Interaction and Virtual Reality to Support Spatial Perspective Taking Ability

According to several large-scale and longitudinal studies, spatial ability, one of the primary mental abilities, has been shown as a significant predictor for STEM learning (Science, Technology, Engineering, and Mathematics) and career success. Frameworks in HCI (Human-Computer Interaction) and TEI (Tangible and Embodied Interaction) also indicated how the spatial-related aspects of interaction are a common design theme for interfaces using emerging technologies. However, currently only very few interactive systems (using TEI) are designed around a target spatial ability. TEI’s direct effects on spatial ability are also not well-investigated. Meanwhile, a growing body of research from cognitive sciences, such as embodied cognition and Common Coding Theory, shows that physical movements can enhance cognition in aspects that involve spatial thinking. Also, virtual reality (VR) affords better 3D perception for digital environments, and provides design opportunities to engage users with spatial tasks that may not be otherwise imagined or achieved in the real world. This research describes how we designed and built the system TASC (Tangibles for Augmenting Spatial Cognition), which combines body movement tracking and tangible objects with VR. We recap our design process and design rationales, along with how the finalized system was designed to enhance embodiment as a means to activate, support, engage, and hopefully augment spatial perspective taking ability. We conducted a user study with qualitative and quantitative evaluation methods. Respectively, the qualitative evaluation aimed to understand how the participants used the system; the quantitative evaluation was a multi-condition experiment with pre-tests and post-tests used to investigate if and how the system could improve spatial perspective taking ability. We built the digital pre/post-tests based on PTSOT (Perspective Taking/Spatial Orientation Test) (Hegarty, Kozhevnikov, & Waller, 2008). The study in total involved 52 participants: 6 participants (3M/3F) in the pilot study, 46 in the main study (3 conditions, around 15 per condition, each condition was overall gender-balanced). The qualitative analysis focused on the VR-TEI condition (the “main system”). Using thematic analysis with the video clips and written notes (both taken during the participants’ interaction), and audio clips (recorded during the post-interaction interview), we synthesized the qualitative results into 4 themes: (1) Spatial strategies: akin but unique; (2) The use of gestures & verbalization; (3) Positive experience with the system; (4) The potentials of the system. The quantitative statistical analysis, using ANOVA and t-test for the 3-condition experiment, showed that every condition yielded perspective taking improvement from taking the test twice. However, only the VR-TEI condition led to statistically significant improvement. We conclude the research with discussion and future possibilities in these themes of: (a) The role of embodiment; (2) Further explorations of intermediate conditions; (3) A deeper look at sample size and validity; (4) Designing & evaluating TEIs for other spatial abilities; (5) Integration with STEM curriculum. The main contribution of this dissertation is that it reports how a VR-TEI system can be designed, built, and evaluated for a target spatial ability. We hope this research also contributes to bridging some knowledge gaps between interaction design, cognitive science, and STEM learning

Games and Rules

Why do we play games and why do we play them on computers? The contributors of »Games and Rules« take a closer look at the core of each game and the motivational system that is the game mechanics. Games are control circuits that organize the game world with their (joint) players and establish motivations in a dedicated space, a »Magic Circle«, whereas game mechanics are constructs of rules designed for interactions that provide gameplay. Those rules form the base for all the excitement and frustration we experience in games. This anthology contains individual essays by authors with backgrounds in Game Design and Game Studies, who lead the discourse to get to the bottom of game mechanics in video games and the real world