A Mimetic Strategy to Engage Voluntary Physical Activity In Interactive Entertainment

We describe the design and implementation of a vision based interactive entertainment system that makes use of both involuntary and voluntary control paradigms. Unintentional input to the system from a potential viewer is used to drive attention-getting output and encourage the transition to voluntary interactive behaviour. The iMime system consists of a character animation engine based on the interaction metaphor of a mime performer that simulates non-verbal communication strategies, without spoken dialogue, to capture and hold the attention of a viewer. The system was developed in the context of a project studying care of dementia sufferers. Care for a dementia sufferer can place unreasonable demands on the time and attentional resources of their caregivers or family members. Our study contributes to the eventual development of a system aimed at providing relief to dementia caregivers, while at the same time serving as a source of pleasant interactive entertainment for viewers. The work reported here is also aimed at a more general study of the design of interactive entertainment systems involving a mixture of voluntary and involuntary control.Comment: 6 pages, 7 figures, ECAG08 worksho

Emotion resonance and divergence: a semiotic analysis of music and sound in 'The Lost Thing', an animated short film and 'Elizabeth' a film trailer

Music and sound contributions of interpersonal meaning to film narratives may be different from or similar to meanings made by language and image, and dynamic interactions between several modalities may generate new story messages. Such interpretive potentials of music and voice sound in motion pictures are rarely considered in social semiotic investigations of intermodality. This paper therefore shares two semiotic studies of distinct and combined music, English speech and image systems in an animated short film and a promotional filmtrailer. The paper considers the impact of music and voice sound on interpretations of film narrative meanings. A music system relevant to the analysis of filmic emotion is proposed. Examples show how music and intonation contribute meaning to lexical, visual and gestural elements of the cinematic spaces. Also described are relations of divergence and resonance between emotion types in various couplings of music, intonation, words and images across story phases. The research is relevant to educational knowledge about sound, and semiotic studies of multimodality

Gulliver project: performers and visitors

This paper discusses two projects in our research environment. The Gulliver project, an ambitious project conceived by some artists connected to our research efforts, and the Aveiro-project, as well ambitious, but with goals that can be achieved beause of technological developments, rather than be dependent on artistic and 'political' (read: financial) sources. Both projects are on virtual and augmented reality. The main goal is to design inhabited environments, where 'inhabited' refers to autonomous agents and agents that represent humans, realtime or off-line, visiting the virtual environment and interacting with other agents. The Gulliver environment has been designed by two artists: Matjaz Stuk and Alena Hudcovicova. The Aveiro project is a research effort of a group of researchers trying to design models of intelligence and interaction underlying the behavior of (groups of) agents inhabiting virtual worlds. In this paper we survey the current state of both projects and we discuss current and future attempts to have music performances by virtual and real performers in these environments

Agent Assistance: From Problem Solving to Music Teaching

We report on our research on agents that act and behave in a web learning environment. This research is part of a general approach to agents acting and behaving in virtual environments where they are involved in providing information, performing transactions, demonstrating products and, more generally, assisting users or visitors of the web environment in doing what they want or have been asked to do. While initially we hardly provided our agents with 'teaching knowledge', we now are in the process of making such knowledge explicit, especially in models that take into account that assisting and teaching takes place in a visualized and information-rich environment. Our main (embodied) tutor-agent is called Jacob; it knows about the Towers of Hanoi, a well-known problem that is offered to CS students to learn about recursion. Other agents we are working on assist a visitor in navigating in a virtual world or help the visitor in getting information. We are now designing a music teacher - using knowledge of software engineering and how to design multi-modal interactions, from previous projects

Exploring the Affective Loop

Research in psychology and neurology shows that both body and mind are involved when experiencing emotions (Damasio 1994, Davidson et al. 2003). People are also very physical when they try to communicate their emotions. Somewhere in between beings consciously and unconsciously aware of it ourselves, we produce both verbal and physical signs to make other people understand how we feel. Simultaneously, this production of signs involves us in a stronger personal experience of the emotions we express. Emotions are also communicated in the digital world, but there is little focus on users' personal as well as physical experience of emotions in the available digital media. In order to explore whether and how we can expand existing media, we have designed, implemented and evaluated /eMoto/, a mobile service for sending affective messages to others. With eMoto, we explicitly aim to address both cognitive and physical experiences of human emotions. Through combining affective gestures for input with affective expressions that make use of colors, shapes and animations for the background of messages, the interaction "pulls" the user into an /affective loop/. In this thesis we define what we mean by affective loop and present a user-centered design approach expressed through four design principles inspired by previous work within Human Computer Interaction (HCI) but adjusted to our purposes; /embodiment/ (Dourish 2001) as a means to address how people communicate emotions in real life, /flow/ (Csikszentmihalyi 1990) to reach a state of involvement that goes further than the current context, /ambiguity/ of the designed expressions (Gaver et al. 2003) to allow for open-ended interpretation by the end-users instead of simplistic, one-emotion one-expression pairs and /natural but designed expressions/ to address people's natural couplings between cognitively and physically experienced emotions. We also present results from an end-user study of eMoto that indicates that subjects got both physically and emotionally involved in the interaction and that the designed "openness" and ambiguity of the expressions, was appreciated and understood by our subjects. Through the user study, we identified four potential design problems that have to be tackled in order to achieve an affective loop effect; the extent to which users' /feel in control/ of the interaction, /harmony and coherence/ between cognitive and physical expressions/,/ /timing/ of expressions and feedback in a communicational setting, and effects of users' /personality/ on their emotional expressions and experiences of the interaction

An investigation of the role of background music in IVWs for learning

Empirical evidence is needed to corroborate the intuitions of gamers and game developers in understanding the benefits of Immersive Virtual Worlds (IVWs) as a learning environment and the role that music plays within these environments. We report an investigation to determine if background music of the genre typically found in computer‐based role‐playing games has an effect on learning in a computer‐animated history lesson about the Macquarie Lighthouse within an IVW. In Experiment 1, music stimuli were created from four different computer game soundtracks. Seventy‐two undergraduate students watched the presentation and completed a survey including biographical details, questions on the historical material presented and questions relating to their perceived level of immersion. While the tempo and pitch of the music was unrelated to learning, music conditions resulted in a higher number of accurately remembered facts than the no music condition. One soundtrack showed a statistically significant improvement in memorisation of facts over other music conditions. Also an interaction between the levels of perceived immersion and ability to accurately remember facts was observed. Experiment 2, involving 48 undergraduate students, further investigated the effect of music, sense of immersion and how different display systems affect memory for facts

The Evolution of Stop-motion Animation Technique Through 120 Years of Technological Innovations

Stop-motion animation history has been put on paper by several scholars and practitioners who tried to organize 120 years of technological innovations and material experiments dealing with a huge literature. Bruce Holman (1975), Neil Pettigrew (1999), Ken Priebe (2010), Stefano Bessoni (2014), and more recently Adrián Encinas Salamanca (2017), provided the most detailed even tough partial attempts of systematization, and designed historical reconstructions by considering specific periods of time, film lengths or the use of stop-motion as special effect rather than an animation technique. This article provides another partial historical reconstruction of the evolution of stop-motion and outlines the main events that occurred in the development of this technique, following criteria based on the innovations in the technology of materials and manufacturing processes that have influenced the fabrication of puppets until the present day. The systematization follows a chronological order and takes into account events that changed the technique of a puppets’ manufacturing process as a consequence of the use of either new fabrication processes or materials. Starting from the accident that made the French film-pioneer Georges Méliès discover the trick of the replacement technique at the end of the nineteenth century, the reconstruction goes through 120 years of experiments and films. “Build up” puppets fabricated by the Russian puppet animator Ladislaw Starevicz with insect exoskeletons, the use of clay puppets and the innovations introduced by LAIKA entertainment in the last decade such as Stereoscopic photography and the 3D computer printed replacement pieces, and then the increasing influence of digital technologies in the process of puppet fabrication are some of the main considered events. Technology transfers, new materials’ features, innovations in the way of animating puppets, are the main aspects through which this historical analysis approaches the previously mentioned events. This short analysis is supposed to remind and demonstrate that stop-motion animation is an interdisciplinary occasion of both artistic expression and technological experimentation, and that its evolution and aesthetic is related to cultural, geographical and technological issues. Lastly, if the technology of materials and processes is a constantly evolving field, what future can be expected for this cinematographic technique? The article ends with this open question and without providing an answer it implicitly states the role of stop-motion as a driving force for innovations that come from other fields and are incentivized by the needs of this specific sector