Expressiveness of real-time motion captured avatars influences perceived animation realism and perceived quality of social interaction in virtual reality

Using motion capture to enhance the realism of social interaction in virtual reality (VR) is growing in popularity. However, the impact of different levels of avatar expressiveness on the user experience is not well understood. In the present study we manipulated levels of face and body expressiveness of avatars while investigating participant perceptions of animation realism and interaction quality when disclosing positive and negative experiences in VR. Moderate positive associations were observed between perceptions of animation realism and interaction quality. Post-experiment questions revealed that many of our participants (approximately 40 %) indicated the avatar with the highest face and body expressiveness as having the most realistic face and body expressions. The same proportion also indicated the avatar with the highest face and body expressiveness as being the most comforting and enjoyable avatar to interact with. Our results suggest that higher levels of face and body expressiveness are important for enhancing perceptions of realism and interaction quality within a social interaction in VR using motion capture

Enhancing computer-human interaction with animated facial expressions

Thesis (M.S.)--Massachusetts Institute of Technology, Dept. of Architecture, 1991.Includes bibliographical references (leaves 87-93).by Brent Cabot James Britton.M.S

Presence 2005: the eighth annual international workshop on presence, 21-23 September, 2005 University College London (Conference proceedings)

OVERVIEW (taken from the CALL FOR PAPERS) Academics and practitioners with an interest in the concept of (tele)presence are invited to submit their work for presentation at PRESENCE 2005 at University College London in London, England, September 21-23, 2005. The eighth in a series of highly successful international workshops, PRESENCE 2005 will provide an open discussion forum to share ideas regarding concepts and theories, measurement techniques, technology, and applications related to presence, the psychological state or subjective perception in which a person fails to accurately and completely acknowledge the role of technology in an experience, including the sense of 'being there' experienced by users of advanced media such as virtual reality. The concept of presence in virtual environments has been around for at least 15 years, and the earlier idea of telepresence at least since Minsky's seminal paper in 1980. Recently there has been a burst of funded research activity in this area for the first time with the European FET Presence Research initiative. What do we really know about presence and its determinants? How can presence be successfully delivered with today's technology? This conference invites papers that are based on empirical results from studies of presence and related issues and/or which contribute to the technology for the delivery of presence. Papers that make substantial advances in theoretical understanding of presence are also welcome. The interest is not solely in virtual environments but in mixed reality environments. Submissions will be reviewed more rigorously than in previous conferences. High quality papers are therefore sought which make substantial contributions to the field. Approximately 20 papers will be selected for two successive special issues for the journal Presence: Teleoperators and Virtual Environments. PRESENCE 2005 takes place in London and is hosted by University College London. The conference is organized by ISPR, the International Society for Presence Research and is supported by the European Commission's FET Presence Research Initiative through the Presencia and IST OMNIPRES projects and by University College London

A Comprehensive Review of Data-Driven Co-Speech Gesture Generation

Gestures that accompany speech are an essential part of natural and efficient embodied human communication. The automatic generation of such co-speech gestures is a long-standing problem in computer animation and is considered an enabling technology in film, games, virtual social spaces, and for interaction with social robots. The problem is made challenging by the idiosyncratic and non-periodic nature of human co-speech gesture motion, and by the great diversity of communicative functions that gestures encompass. Gesture generation has seen surging interest recently, owing to the emergence of more and larger datasets of human gesture motion, combined with strides in deep-learning-based generative models, that benefit from the growing availability of data. This review article summarizes co-speech gesture generation research, with a particular focus on deep generative models. First, we articulate the theory describing human gesticulation and how it complements speech. Next, we briefly discuss rule-based and classical statistical gesture synthesis, before delving into deep learning approaches. We employ the choice of input modalities as an organizing principle, examining systems that generate gestures from audio, text, and non-linguistic input. We also chronicle the evolution of the related training data sets in terms of size, diversity, motion quality, and collection method. Finally, we identify key research challenges in gesture generation, including data availability and quality; producing human-like motion; grounding the gesture in the co-occurring speech in interaction with other speakers, and in the environment; performing gesture evaluation; and integration of gesture synthesis into applications. We highlight recent approaches to tackling the various key challenges, as well as the limitations of these approaches, and point toward areas of future development.Comment: Accepted for EUROGRAPHICS 202

Robot mediated communication: Enhancing tele-presence using an avatar

In the past few years there has been a lot of development in the field of tele-presence. These developments have caused tele-presence technologies to become easily accessible and also for the experience to be enhanced. Since tele-presence is not only used for tele-presence assisted group meetings but also in some forms of Computer Supported Cooperative Work (CSCW), these activities have also been facilitated. One of the lingering issues has to do with how to properly transmit presence of non-co-located members to the rest of the group. Using current commercially available tele-presence technology it is possible to exhibit a limited level of social presence but no physical presence. In order to cater for this lack of presence a system is implemented here using tele-operated robots as avatars for remote team members and had its efficacy tested. This testing includes both the level of presence that can be exhibited by robot avatars but also how the efficacy of these robots for this task changes depending on the morphology of the robot. Using different types of robots, a humanoid robot and an industrial robot arm, as tele-presence avatars, it is found that the humanoid robot using an appropriate control system is better at exhibiting a social presence. Further, when compared to a voice only scenario, both robots proved significantly better than with only voice in terms of both cooperative task solving and social presence. These results indicate that using an appropriate control system, a humanoid robot can be better than an industrial robot in these types of tasks and the validity of aiming for a humanoid design behaving in a human-like way in order to emulate social interactions that are closer to human norms. This has implications for the design of autonomous socially interactive robot systems

The Uncanny Valley Effect

The Uncanny Valley Effect (UVE) first emerged as a warning against making industrial robots appear so highly human-like that they could unsettle the real humans around them. It proposed a specific pattern of negative emotional responses to entities that were almost but not quite human, and has been proposed as the reason why some entities such as dolls, mannequins and zombies may appear unsettling. The aim of this thesis was to move beyond an anecdotal explanation to understand more about the perception of near-human faces, and how this compares to the perception of human and non-human faces. The aims were to explore the relationship between the human-likeness of faces and emotional responses to them, to understand reactions to and descriptions of near-human faces, to explore aspects of how near-human faces are processed and to explore whether mismatched emotional expressions might contribute to the perception of some near-human faces as eerie. Five studies were carried out using face images whose human-likeness was systematically controlled or measured. A non-linear relationship between human-likeness and eeriness was found, but the near-human faces were not always the eeriest images. Near-human faces were found to be subject to the effects of inversion, and inversion was found to heighten perceptions of eeriness. Faces were created which contained mismatched emotional expressions, and the blends combining happy faces with angry or fearful eyes were rated as the most eerie. Incongruities between aspects of appearance or behaviour had been cited as explanations for the UVE in the past but this thesis presents the first evidence that differences in eeriness may result from incongruities between emotional expressions. Directions for future research have been suggested to explore these findings in a wider context and to understand more about the UVE

The role of empathy between cognition and aesthetic sense-making : the case of tragedy and the myth of Medea

Adopting a phenomenological and cognitive semiotic approach, the dissertation defines and discusses empathy and the empathic process highlighting its role in intersubjective contexts and encounters, as well as a tool for comprehension of the unity between observer and artistic object. This peculiar form of knowledge pertains to the original experience of outer objects’ perception, which is influenced by memory of previous knowledge, reservoir of meaning in the phenomenal unity with the other (Zahavi 2010, 2012), and different cognitive devices (metaphors, narrative and semantic domains) which contribute to the shift from observation to comprehension. The dissertation aims to explain how empathy’s epistemic attitude allows an embodied form of understanding, which occurs in the narratological experience of another subject’s states and the aesthetic contemplation of artistic works. In this context, meaning is profoundly linked with emotional, physical and mental dimensions of sense-making: the dissertation analyses visual, embodied and cognitive reactions evoked by aesthetic stimuli and the way spectators’ patterns of consciousness are able to emotionally respond to the experience of artworks. Furthermore, the theoretical study on empathy is applied to the examination of tragedy, acknowledging its relevance in shaping the evolution of literary and performative representation through time. Addressing the significative particularities of tragedy and its characters, the research will focus on the myth of Medea across history, both in its literary depiction by Euripides (432 BC) and the film adaptation directed by Pier Paolo Pasolini (1969).Adoptando uma abordagem baseada na fenomenologia e na semiótica cognitiva, a dissertação define e discute a empatia e o processo empático destacando o seu papel em contextos e encontros intersubjectivos, bem como enquanto instrumento para a compreensão da ligação entre observador e objecto artístico. Esta forma particular de conhecimento pertence à experiência original da percepção dos objectos exteriores, que é influenciada pela memória de conhecimentos anteriores, reservatório de significado na unidade fenomenal com o outro (Zahavi 2010, 2012), e diferentes dispositivos cognitivos (metáforas, domínios narrativos e semânticos) que contribuem para a passagem da observação à compreensão. A dissertação visa explicar como a atitude epistémica da empatia permite uma forma coporealizada (embodied) de compreensão, que ocorre na experiência narratológica dos estados de outro sujeito e na contemplação estética das obras artísticas. Neste contexto, o significado está profundamente ligado às dimensões emocional, física e mental do processo de fazer sentido: a dissertação analisa as reacções visuais, corporais e cognitivas evocadas pelos estímulos estéticos e a forma como os padrões de consciência dos espectadores são capazes de responder emocionalmente à experiência das obras de arte. Além disso, o estudo teórico sobre empatia é aplicado ao estudo da tragédia, reconhecendo a sua relevância na forma de evolução desta representação literária e performativa ao longo do tempo. Abordando as particularidades significativas da tragédia e das suas personagens, este trabalho centrar-se-á no mito de Medeia ao longo da história, tanto na sua representação literária por Eurípedes (432 a.C.) como na adaptação cinematográfica de Pier Paolo Pasolini (1969)

Designing Embodied Interactive Software Agents for E-Learning: Principles, Components, and Roles

Embodied interactive software agents are complex autonomous, adaptive, and social software systems with a digital embodiment that enables them to act on and react to other entities (users, objects, and other agents) in their environment through bodily actions, which include the use of verbal and non-verbal communicative behaviors in face-to-face interactions with the user. These agents have been developed for various roles in different application domains, in which they perform tasks that have been assigned to them by their developers or delegated to them by their users or by other agents. In computer-assisted learning, embodied interactive pedagogical software agents have the general task to promote human learning by working with students (and other agents) in computer-based learning environments, among them e-learning platforms based on Internet technologies, such as the Virtual Linguistics Campus (www.linguistics-online.com). In these environments, pedagogical agents provide contextualized, qualified, personalized, and timely assistance, cooperation, instruction, motivation, and services for both individual learners and groups of learners. This thesis develops a comprehensive, multidisciplinary, and user-oriented view of the design of embodied interactive pedagogical software agents, which integrates theoretical and practical insights from various academic and other fields. The research intends to contribute to the scientific understanding of issues, methods, theories, and technologies that are involved in the design, implementation, and evaluation of embodied interactive software agents for different roles in e-learning and other areas. For developers, the thesis provides sixteen basic principles (Added Value, Perceptible Qualities, Balanced Design, Coherence, Consistency, Completeness, Comprehensibility, Individuality, Variability, Communicative Ability, Modularity, Teamwork, Participatory Design, Role Awareness, Cultural Awareness, and Relationship Building) plus a large number of specific guidelines for the design of embodied interactive software agents and their components. Furthermore, it offers critical reviews of theories, concepts, approaches, and technologies from different areas and disciplines that are relevant to agent design. Finally, it discusses three pedagogical agent roles (virtual native speaker, coach, and peer) in the scenario of the linguistic fieldwork classes on the Virtual Linguistics Campus and presents detailed considerations for the design of an agent for one of these roles (the virtual native speaker)