Interface design to support situation awareness in virtual puppetry

Virtual Heritage is the use of digital media to reconstruct cultures and cultural artifacts as they are today or as they might have been in the past. The central element is usually a threedimensional computer model of a person, place, or thing. Frequently, these are ancient monuments, temples, homes, and other social spaces (Jacobson, 2008). The goal of Virtual Heritage is to draw viewers into the virtual world and allow them to directly experience the overall context of the environment. This phenomenon is known to researchers as “presence.” It is a long held belief that the increased presence yields better the opportunities for deeper learning (Devine, 2007)

Tangible User Interfaces and Metaphors for 3D Navigation

The most fundamental and common 3D interaction is the control of the virtual camera or viewpoint, commonly referred to as navigation. The navigational requirements of controlling multiple degrees of freedom and maintaining adequate spatial awareness are big challenges to many users. Many tasks additionally demand large portions of cognitive effort from the user for non-navigational aspects. Therefore, new solutions that are simple and naturally efficient are in high demand. These major challenges to 3D navigation have yet to be satisfactorily addressed, and as a result, there has yet to be a declaration of a suitable unified 3D interaction technique or metaphor. We present a new domain and task independent 3D navigation metaphor, Navigational Puppetry, which we intend to be a candidate for the navigational portion of a unifying 3D interaction metaphor. The major components of the metaphor - the puppet, puppeteer, stage, and puppet-view - enable a new meta-navigational perspective and provide the user with a graspable navigational avatar, within a multiple-view perspective, that allows them to ‘reach’ within the virtual world and manipulate the viewpoint directly. We position this metaphor as a distinct articulation of the front wave of a puppetry related trend in recent 3D navigation solutions. The metaphor was implemented into a tangible user interface prototype called the Navi-Teer. Two usability studies and a unique spatial audio experiment were completed to observe and demonstrate, respectively, the metaphor’s benefits of tactile intimacy, spatial orientation, easy capture of complex input and support for collaboration

Affordable avatar control system for personal robots

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2009.Includes bibliographical references (p. 76-79).Social robots (personal robots) emphasize individualized social interaction and communication with people. To maximize communication capacity of a personal robot, designers make it more anthropomorphic (or zoomorphic), and people tend to interact more naturally with such robots. However, adapting anthropomorphism (or zoomorphism) in social robots makes morphology of a robot more complex; thus, it becomes harder to control robots with existing interfaces. The Huggable is a robotic Teddy bear platform developed by the Personal Robots Group at the MIT Media Lab. It has its specific purpose in healthcare, elderly care, education, and family communication. It is important that a user can successfully convey the meaningful context in a dialogue via the robot's puppeteering interface. I investigate relevant technologies to develop a robotic puppetry system for a zoomorphic personal robot and develop three different puppeteering interfaces to control the robot: the website interface, wearable interface, and sympathetic interface. The wearable interface was examined through a performance test and the web interface was examined through a user study.by Jun Ki Lee.S.M

Sustainable Technology and Elderly Life

The coming years will see an exponential increase in the proportion of elderly people in our society. This accelerated growth brings with it major challenges in relation to the sustainability of the system. There are different aspects where these changes will have a special incidence: health systems and their monitoring; the development of a framework in which the elderly can develop their daily lives satisfactorily; and in the design of intelligent cities adapted to the future sociodemographic profile. The discussion of the challenges faced, together with the current technological evolution, can show possible ways of meeting the challenges. There are different aspects where these changes will have a special incidence: health systems and their monitoring; the development of a framework in which the elderly can develop their daily lives satisfactorily; and in the design of intelligent cities adapted to the future sociodemographic profile. This special issue discusses various ways in which sustainable technologies can be applied to improve the lives of the elderly. Six articles on the subject are featured in this volume. From a systematic review of the literature to the development of gamification and health improvement projects. The articles present suggestive proposals for the improvement of the lives of the elderly. The volume is a resource of interest for the scientific community, since it shows different research gaps in the current state of the art. But it is also a document that can help social policy makers and people working in this domain to planning successful projects

Show and Tell: Learning with Interactive Videoconferencing in Kindergarten

The research investigated how kindergartners make meaning using interactive videoconferencing. The study explored two research questions: 1) What types of meanings are being formed by the kindergartners during interactive videoconferences and, 2) What are the nature of young children\u27s emerging inquiries and dialogue surrounding their use of interactive videoconferencing in their classroom? The study embodied a Vygotskian perspective as the theoretical framework in order to meet demands associated with the young participants\u27 vulnerability, developmental appropriateness, and the students\u27 interactive learning environment. Employing an ethnographic, participant observation methodology, the research design was informed by three criteria: 1) a pilot study, 2) Miles and Huberman\u27s (1994a) recurring themes in qualitative data analysis, and 3) literature review results emphasizing the nuances of contemporary culture. Field observation occurred from October 2007 through February 2008 in a Southwestern Pennsylvania kindergarten classroom. Students participated in 7 videoconferences with distant peers or content experts. Data from a gingerbread and puppetry videoconference and an astronomy program were selected for further analysis based on their ability to illustrate poignant examples of how the kindergartners formed meaning during collaborations. Data analysis procedures involved the importing of dialogue from videoconferencing transcriptions, field notes, and other artifacts into the ATLAS.ti qualitative data analysis software for open coding, data display, and grounded theory development. Results developed from open coding and concept maps in ATLAS.ti informed the following theory development. First, learning with interactive videoconferencing in kindergarten supports meaning making from four Vygotskian tenets: 1) the social origins of learning, 2) sign and tool use through mediated activity, 3) the importance of language, and 4) support for the zone of proximal development. Additionally, the students\u27 meaning making involved the tenets\u27 entwinement rather than the solitary occurrence of individual tenets. Regarding the kindergartners\u27 emerging inquiries, during sustained interactive videoconferencing levels, children\u27s inquiries and dialogue evidenced exploratory talk that was purposeful, reflective and self-directed. It also indicated comfort with the technology. This study is unique in its multidisciplinary application of Vygotskian learning theory to kindergartners\u27 meaning making with videoconferencing and provides a foundation for extended use of qualitative methods to examine young children\u27s\u27 learning with technology

The Quest for Life and Intelligence in Digital Puppets.

Performers and artists wishing to make collaborative improvisations using three-dimensional computer graphics will encounter the following difficulty: the animation process lacks the functionality required for spontaneous, serendipitous, real-time interaction. While human motion capture makes such real-time interaction and the corresponding spontaneity possible, it lacks the expressivity required for non-realistic characterisation. My practice-based research study proposes digital puppetry as a solution to this problem. My focus is on credibility as opposed to realism: the expectation is that the characters will behave in a puppet-like manner as opposed to manifesting the aesthetics – in terms of both movement and visual design – found in human actor-based motion capture and three- dimensional animated cartoons. The practical element is motivated by three imperatives: to improvise, to collaborate and to perform live. The primary question this study sets out to answer is: Is it possible to achieve the same spontaneity and animation[1] in digital puppets as it is with tangible puppets and, if so, what are the mechanisms involved? Unable to find a three-dimensional computer graphics digital puppetry software and hardware solution, I have devised what I call the GLOPPID[2] method, where GLOPPID is an acronym for Goniometric[3] Live Organic Performance Puppetry Improvisation Digitalia. The GLOPPID method comprises an artistic approach and a practical solution in the form of a Human Digital Puppetry Interface. It uses off-the-shelf three-dimensional computer animation software, which I have incorporated into a pipeline customised to suit my creative process. This pipeline is configured to transform ready-made computer graphics models into digital puppets that can be used as collaborators, thereby allowing the performer to experience the same kind of spontaneity as is possible in physical puppet performance. My thesis asserts that it is possible to improvise with digital puppets, and I have devised my own solution in order to do this. I argue that the real-time, improvised manipulation of digital puppets offers creatively advantageous opportunities for spontaneity and expressivity. My research presents the technique of digital puppetry as an expansion of what I call the pro- puppetry thesis – the idea that puppets have dramatic advantages over human actors. It also contributes to the ontological discourses surrounding the Human-Machine Interface (HMI), trans-embodiment, the post-human, the illusion of life, and cybernetics. In addition, it explores how algorithms can be used in the arts, particularly in performance (see Kleber & Trojanowska 2019, p.101). It makes a timely contribution to the pool of knowledge, because I see digital puppets as zeitgeists – apt vehicles for human hopes and fears surrounding the digital and existential angst that is part of the fabric of 21st-century life. The professional digital puppetry practice undertaken and discussed in this study requires both general, transferable human-machine interacting skills, and the specific digital puppetry skills necessary for project phases such as rigging and manipulation. This practical approach prioritises the physical, as opposed to the psycho-physical. Informed by the theories and practice of human dramatic technique practitioners such as Decroux and Lecoq, it takes the basic building blocks of movement identifiable in the segmented anatomy and rotational articulations of the actor-as-puppet, rod, glove and string puppets, and configures them in their digital counterpart in order to accentuate the odd by means of atypical combinations, economy of motion, and asymmetry. My approach is underpinned by the idea that a puppet is uninhibited or influenced by its own ego, backstory, or emotions. Nevertheless, these properties are present in a channeled, and therefore changed, form that emanates from the Human in the Loop, namely, the digital puppeteer. In digital puppetry, the protagonist is a digital puppet operated by a human who is embedded at the core of the activity, and who simultaneously witnesses their own emotional responses as they are acted out in front of them. My work demonstrates that non-realistic, expressive approaches to movement performance derived from human physical theatre techniques (including theatre clowning), combined with the use of algorithm-assisted techniques of rigging and manipulation, mean that the puppets are not under the complete control of their puppeteers. Instead, their rogue nature and irrationality enables digital puppets to satirise and subvert notions surrounding computer-generated imagery and artificial general intelligence, while avoiding exile in the “uncanny valley” (Mori 1970). In this study the term puppet is used both literally and metaphorically. Deployed literally, puppet refers to a figure or object that is manipulated in real time in the presence of a puppeteer, in both tangible and intangible material formats. The term is used metaphorically in the sense that a puppet can be seen as an analogue of a human being that acts as a mirror, reflecting aspects of the human condition or predicament. My work expands upon an understanding of the quest for ‘life’ in physical puppets and the corresponding development of their ur-narrative – as described by Kohler & Jones (2009, p.346) – and extends this to include digital puppets. By combining the concept of this quest with Rokeby’s idea of technology as a “prosthetic of philosophy” (2019, p.107) and with the notion of art as a mirror, I explore how distinctive features present in digital puppetry practice can be used to express truths about being human. The work employs a practice-as-research methodology that provides moments for reflection during the creative process, and reflection on the creative outcomes: reflection-in-action and reflection-on-action. The thesis can be expressed in the form of the following formula: Extemporising with puppets + the ur-narrative of puppets + art as a mirror + technology as a mirror = a pro-digital-puppetry thesis on HMI, AGI + what it is to be human [1] The term animation used here refers not to the techniques of incremental, frame-by-frame posing of characters, but to the act of bringing them to life. [2] ‘Gloppid’ is also the name of a glove-puppet character invented for an ecologically-based travelling show performed during the 1980s (Childs 1988). [3] Goniometric refers to the measurement of the range of motion in a joint

Crafting Play: Little Big Planet

In the contemporary era of Web 2.0, high-tech consumer culture is increasingly engaged in the production of ‘user-generated content’ (UGC) for digital multicast. The tension between global homogeneity and the potential of technology to support multiple voices, histories and viewpoints is of central interest. The new DIY craft movement is successfully adopting Internet technologies to go straight to market as the digital generation increasingly engages in analogue craft practice. The swell of interest in craft values, both in objects and in hands-on feel and process exhibited in blogs such as Wonderland and distribution aggregators like Etsy, offers a productive frame that connects the digital and the analogue. Whether this reveals any anxiety about the intangibility of the digital or points to an increased creativity inspired by UGC remains open to question. The ‘feedback loop’ (to use Schechner’s (2002) term for the connection between an individual’s behavior and what they observe on street, stage and screen) between digital and real world practice, although far from literal, provides a frame for the dialogue between game form and culture at large. This paper teases out aspects of this feedback loop using examples from Sony’s PS3 series Little Big Planet (2008). The argument presented here does not deal with narratological or ludic structures and only tips its hat at the much broader field of fan culture but foregrounds context, style and characterization in its approach to analysis. The rationale for this approach is two-fold; first through the weight Media Molecule, developers of the game, give to visual communication and secondly through the prioritization of the invitation to create over and above the provision of a full triple-A title more typical of a console launch game

Investigating User Experience Using Gesture-based and Immersive-based Interfaces on Animation Learners

Creating animation is a very exciting activity. However, the long and laborious process can be extremely challenging. Keyframe animation is a complex technique that takes a long time to complete, as the procedure involves changing the poses of characters through modifying the time and space of an action, called frame-by-frame animation. This involves the laborious, repetitive process of constantly reviewing results of the animation in order to make sure the movement-timing is accurate. A new approach to animation is required in order to provide a more intuitive animating experience. With the evolution of interaction design and the Natural User Interface (NUI) becoming widespread in recent years, a NUI-based animation system is expected to allow better usability and efficiency that would benefit animation. This thesis investigates the effectiveness of gesture-based and immersive-based interfaces as part of animation systems. A practice-based element of this research is a prototype of the hand gesture interface, which was created based on experiences from reflective practices. An experimental design is employed to investigate the usability and efficiency of gesture-based and immersive-based interfaces in comparison to the conventional GUI/WIMP interface application. The findings showed that gesture-based and immersive-based interfaces are able to attract animators in terms of the efficiency of the system. However, there was no difference in their preference for usability with the two interfaces. Most of our participants are pleasant with NUI interfaces and new technologies used in the animation process, but for detailed work and taking control of the application, the conventional GUI/WIMP is preferable. Despite the awkwardness of devising gesture-based and immersive-based interfaces for animation, the concept of the system showed potential for a faster animation process, an enjoyable learning system, and stimulating interest in a kinaesthetic learning experience

Shadows, touch and digital puppeteering: a media archaeological approach

Aims The practical aim of this research project is to create a multi-touch digital puppetry system that simulates shadow theatre environments and translates gestural acts of touch into live and expressive control of virtual shadow figures. The research is focussed on the qualities of movement achievable through the haptics of single and multi-touch control of the digital puppets in the simulation. An associated aim is to create a collaborative environment where multiple performers can control dynamic animation and scenography, and create novel visualisations and narratives. The conceptual aim is to link traditional and new forms of puppetry seeking cultural significance in the ‘remediation’ of old forms that avail themselves of new haptic resources and collaborative interfaces. The thesis evaluates related prior art where traditional worlds of shadow performance meet new media, digital projection and 3D simulation, in order to investigate how changing technical contexts transform the potential of shadows as an expressive medium. Methodology The thesis uses cultural analysis of relevant documentary material to contextualise the practical work by relating the media archaeology of 2D puppetry—shadows, shadowgraphs and silhouettes—to landmark work in real-time computer graphics and performance animation. The survey considers the work of puppeteers, animators, computer graphics specialists and media artists. Through practice and an experimental approach to critical digital creativity, the study provides practical evidence of multiple iterations of controllable physics-based animation delivering expressive puppet motion through touch and multiuser interaction. Video sequences of puppet movement and written observational analysis document the intangible aspects of animation in performance. Through re-animation of archival shadow puppets, the study presents an emerging artistic media archaeological method. The major element of this method has been the restoration of a collection of Turkish Karagöz Shadow puppets from the Institut International de la Marionnette (Charleville, France) into a playable digital form. Results The thesis presents a developing creative and analytical framework for digital shadow puppetry. It proposes a media archaeological method for working creatively with puppet archives that unlock the kinetic and expressive potential of restored figures. The interaction design introduces novel approaches to puppetry control systems—using spring networks—with objects under physics-simulation that demonstrate emergent expressive qualities. The system facilitates a dance of agency¹ between puppeteer and digital instrument. The practical elements have produced several software iterations and a tool-kit for generating elegant, nuanced multi-touch shadow puppetry. The study presents accidental discoveries—serendipitous benefits of open-ended practical exploration. For instance: the extensible nature of the control system means novel input—other than touch—can provide exciting potential for accessible user interaction, e.g. with gaze duration and eye direction. The study also identifies limitations including the rate of software change and obsolescence, the scope of physics-based animation and failures of simulation. Originality/value The work has historical value in that it documents and begins a media archaeology of digital puppetry, an animated phenomenon of increasing academic and commercial interest. The work is of artistic value providing an interactive approach to making digital performance from archival material in the domain of shadow theatre. The work contributes to the electronic heritage of existing puppetry collections. The study establishes a survey of digital puppetry, setting a research agenda for future studies. Work may proceed to digitise, rig and create collaborative and web-mediated touch-based motion control systems for 2D and 3D puppets. The present study thus provides a solid platform to restore past performances and create new work from old, near forgotten-forms. ¹ Following Andrew Pickering, puppetry is ‘a temporally extended back-and-forth dance of human and non-human agency in which activity and passivity on both sides are reciprocally intertwined’ PICKERING, A. 2010. Material Culture and the Dance of Agency. In: BEAUDRY, M. C. & HICKS, D. (eds.) Oxford Handbook of Material Culture Studies. Oxford University Press.