Puppet Narrator: utilizing motion sensing technology in storytelling for young children

Using avatars in storytelling to assist narration has proved to be beneficial on promoting creativity, collaboration and intimacy among young children. Development of novel Human Computer Interaction (HCI) techniques provides us with new possibilities to explore the training aspects of storytelling by creating new ways of interaction. In this paper, we design and develop a novel digital puppetry storytelling system - Puppet Narrator for young children, utilizing depth motion sensing technology as the HCI method. More than merely allowing children to narrate orally, our system allows them to use hand gestures to play with a virtual puppet and manipulate it to interact with virtual items in virtual environment to assist narration. Under this novel pattern of interaction, children’s narrative ability can be trained and the competencies of cognition and motor coordination can also be nourished

Hand gesture-based interactive puppetry system to assist storytelling for children

© 2016 The Author(s)Digital techniques have been used to assist narrative and storytelling, especially in many pedagogical practices. With the rapid development of HCI techniques, saturated with digital media in their daily lives, young children, demands more interactive learning methods and meaningful immersive learning experiences. In this paper, we propose a novel hand gesture-based puppetry storytelling system which provides a more intuitive and natural human computer interaction method for young children to develop narrative ability in virtual story world. Depth motion sensing and hand gestures control technology is utilized in the implementation of user-friendly interaction. Young players could intuitively use hand gestures to manipulate virtual puppet to perform story and interact with different items in virtual environment to assist narration. Based on the result of the evaluation, this novel digital storytelling system shows positive pedagogical functions on children’s narrating ability as well as the competencies of cognitive and motor coordination. The usability of the system is preliminary examined in our test, and the results which showed that young children can benefit from playing with Puppet Narrator

Virtual puppetry: assisting the elder's life review.

This paper will describe "VoicingElder", an expressive storytelling platform for the senior population, using virtual puppets. VoicingElder combines reminiscent oral storytelling with virtual puppetry to develop a deeper understanding of self-worth. This form of storytelling through puppets is inspired by traditional Korean shamanistic ritual and the philosophy of han. VoicingElder aims to fuse Korean shamanistic ritual and twenty-first century virtual interactive technology to examine the current state of elder hood in Western culture, and to contribute in positive ways to the new elder hood that we face today

PRESENCE AND BODY-OWNERSHIP: THE CASE OF THE EXERCISING AVATAR

This theoretical study on the sport exercises of virtual bodies (avatars) in virtual Fitness Clubs discusses the sense of being in the Virtual World (VW) in relation to virtual exercises and concepts on body ownership and presence. Based on interdisciplinary research and data, the purpose of this theoretical reflection is to contribute to the formulation of a more coherent theoretical model surrounding the issue of sport exercises and the virtual body. This matter emerges as a significant topic in the field of sport sociology and sport science in general and provides fertile ground for multiple analyses for future research.  Article visualizations

Captura de movimento e animação como meio de criação e treinamento em performance

In 2014 a group of students, faculty and staff at the Ohio State University’s (OSU) Department of Theatre, Department of Design and Advanced Computing Center for the Arts and Design (ACCAD) joined their creative energies to devise the play “There is No Silence” based on the life and works of the iconic French mime Marcel Marceau. Marceau’s history of being the first performance artist motion captured at ACCAD in 2001 afforded the group artistic and technological opportunities to integrate digital animation and performance of real-time virtual avatars into their work. The process of devising revealed parallels between the practices of mime art and performing with virtual environments and offered new insights for devising with media.Em 2014, um grupo de estudantes, professores e funcionários do Departamento de Teatro, Departamento de Design e do Centro de Computação Avançada para Artes e Design (ACCAD) da Ohio StateUniversity (OSU) se uniram para elaborar e criar a peça “Thereis no silence” baseada na vida e obra do importante mímico francês Marcel Marceau. Em 2001, a possibilidade de realizar, pela primeira vez na história, uma performance de Marceau, por meio de captura de movimento feito pelo ACCAD, proporcionou ao grupo a oportunidade artística e tecnológica de integrar animação digital e performance de avatares virtuais em tempo real em seu trabalho. O processo de elaboração revelou semelhanças entre a prática damímica e a atuação em ambientes virtuais, oferecendo assim, novas possibilidades criação com novas mídias

Lessons from digital puppetry - Updating a design framework for a perceptual user interface

While digital puppeteering is largely used just to augment full body motion capture in digital production, its technology and traditional concepts could inform a more naturalized multi-modal human computer interaction than is currently used with the new perceptual systems such as Kinect. Emerging immersive social media networks with their fully live virtual or augmented environments and largely inexperienced users would benefit the most from this strategy. This paper intends to define digital puppeteering as it is currently understood, and summarize its broad shortcomings based on expert evaluation. Based on this evaluation it will suggest updates and experiments using current perceptual technology and concepts in cognitive processing for existing human computer interaction taxonomy. This updated framework may be more intuitive and suitable in developing extensions to an emerging perceptual user interface for the general public

To Affinity and Beyond: Interactive Digital Humans as a Human Computer Interface

The field of human computer interaction is increasingly exploring the use of more natural, human-like user interfaces to build intelligent agents to aid in everyday life. This is coupled with a move to people using ever more realistic avatars to represent themselves in their digital lives. As the ability to produce emotionally engaging digital human representations is only just now becoming technically possible, there is little research into how to approach such tasks. This is due to both technical complexity and operational implementation cost. This is now changing as we are at a nexus point with new approaches, faster graphics processing and enabling new technologies in machine learning and computer vision becoming available. I articulate the issues required for such digital humans to be considered successfully located on the other side of the phenomenon known as the Uncanny Valley. My results show that a complex mix of perceived and contextual aspects affect the sense making on digital humans and highlights previously undocumented effects of interactivity on the affinity. Users are willing to accept digital humans as a new form of user interface and they react to them emotionally in previously unanticipated ways. My research shows that it is possible to build an effective interactive digital human that crosses the Uncanny Valley. I directly explore what is required to build a visually realistic digital human as a primary research question and I explore if such a realistic face provides sufficient benefit to justify the challenges involved in building it. I conducted a Delphi study to inform the research approaches and then produced a complex digital human character based on these insights. This interactive and realistic digital human avatar represents a major technical undertaking involving multiple teams around the world. Finally, I explored a framework for examining the ethical implications and signpost future research areas

Striking a Balance between Property and Personality. The Case of the Avatars

Virtual worlds, as powerful social platforms of intense human interaction, gather millions of users worldwide, producing massive economies of their own, giving rise to the birth of complex social relationships and the formation of virtual communities. By enabling the creativity of the player and figuring as an outstanding example of new online collaborative environments, virtual worlds emerge as context for creation, allowing for users to undertake a digital alter-ego and become artists, creators and authors. Nevertheless, such digital egos are not merely creations, but a reflex of their creators, an extension of their personalities and indicia of their identities. As a result, this paper perceives the avatar not only as a property item (avatar as the player’s or [game-developer’s] property) but also, and simultaneously, as a reflex of our personality and identity (avatar as the projection of one self in the virtual domain, as part of an individual persona). Bearing in mind such hybrid configuration, and looking at the disputes over property rights in virtual words, this essay makes three fundamental arguments. Firstly, it proposes a re-interpretation of intellectual property rights (namely of copyright law) according to its underlying utilitarian principles, as such principles seem to have been forgotten or neglected in the sphere of virtual worlds. The idea is to re-balance the uneven relationship between game owners and players perpetuated by the end-user license agreements (EULAs), recognising property rights to users over their own virtual creations. In order to evaluate whether a user’s contribution to the virtual world amounts to an original and creative work and is worthy of copyright protection, the essay proposes the image of a jigsaw puzzle as a tool and criteria to carry out such examination

Actor & Avatar: A Scientific and Artistic Catalog

What kind of relationship do we have with artificial beings (avatars, puppets, robots, etc.)? What does it mean to mirror ourselves in them, to perform them or to play trial identity games with them? Actor & Avatar addresses these questions from artistic and scholarly angles. Contributions on the making of "technical others" and philosophical reflections on artificial alterity are flanked by neuroscientific studies on different ways of perceiving living persons and artificial counterparts. The contributors have achieved a successful artistic-scientific collaboration with extensive visual material

Towards a Framework for Embodying Any-Body through Sensory Translation and Proprioceptive Remapping: A Pilot Study

We address the problem of physical avatar embodiment and investi- gate the most general factors that may allow a person to “wear” an- other body, different from her own. A general approach is required to exploit the fact that an avatar can have any kind of body. With this pilot study we introduce a conceptual framework for the design of non-anthropomorphic embodiment, to foster immersion and user engagement. The person is interfaced with the avatar, a robot, through a system that induces a divergent internal sensorimotor mapping while controlling the avatar, to create an immersive expe- rience. Together with the conceptual framework, we present two implementations: a prototype tested in the lab and an interactive in- stallation exhibited to general public. These implementations consist of a wheeled robot, and control and sensory feedback systems. The control system includes mechanisms that both detect and resist the user’s movement, increasing the sense of connection with the avatar; the feedback system is a virtual reality (VR) environment represent- ing the avatar’s unique perception, combining sensor and control in- formation to generate visual cues. Data gathered from users indicate that the systems implemented following the proposed framework create a challenging and engaging experience, thus providing solid ground for further developments