Talk to the Virtual Hands: Self-Animated Avatars Improve Communication in Head-Mounted Display Virtual Environments

Background When we talk to one another face-to-face, body gestures accompany our speech. Motion tracking technology enables us to include body gestures in avatar-mediated communication, by mapping one's movements onto one's own 3D avatar in real time, so the avatar is self-animated. We conducted two experiments to investigate (a) whether head-mounted display virtual reality is useful for researching the influence of body gestures in communication; and (b) whether body gestures are used to help in communicating the meaning of a word. Participants worked in pairs and played a communication game, where one person had to describe the meanings of words to the other. Principal Findings In experiment 1, participants used significantly more hand gestures and successfully described significantly more words when nonverbal communication was available to both participants (i.e. both describing and guessing avatars were self-animated, compared with both avatars in a static neutral pose). Participants ‘passed’ (gave up describing) significantly more words when they were talking to a static avatar (no nonverbal feedback available). In experiment 2, participants' performance was significantly worse when they were talking to an avatar with a prerecorded listening animation, compared with an avatar animated by their partners' real movements. In both experiments participants used significantly more hand gestures when they played the game in the real world. Conclusions Taken together, the studies show how (a) virtual reality can be used to systematically study the influence of body gestures; (b) it is important that nonverbal communication is bidirectional (real nonverbal feedback in addition to nonverbal communication from the describing participant); and (c) there are differences in the amount of body gestures that participants use with and without the head-mounted display, and we discuss possible explanations for this and ideas for future investigation

Virtual Reality Interaction Techniques For Individuals With Autism Spectrum Disorder

Virtual reality (VR) systems are seeing growing use for training individuals with Autism Spectrum Disorder (ASD). Although these systems indicate effective use of VR for training, there is little work in the literature evaluating different VR interaction techniques for this audience. In this paper, different VR interaction techniques are explored in the Virtual Reality for Vocational Rehabilitation (VR4VR) system and additional data analysis on top of our previously published preliminary results [1] was performed via a user study with nine individuals with ASD and ten neurotypical individuals. The participants tried six vocational training modules of the VR4VR system. In these modules, tangible object manipulation, haptic device, touch and snap and touchscreen were tested for object selection and manipulation; real walking and walk-in-place were tested for locomotion; and head mounted display and curtain screen were tested for display. Touchscreen and tangible interaction methods were preferred by the individuals with ASD. The walk-in-place locomotion technique were found frustrating and difficult to perform by the individuals with ASD. Curtain display received higher preference scores from individuals with ASD although they accepted the HMD as well. The observations and findings of the study are expected to give insight into the poorly explored area of experience of individuals with ASD with various interaction techniques in VR

The effect of translational and rotational body-based information on navigation

Physical locomotion provides internal (body-based) sensory information about the translational and rotational components of movement. This chapter starts by summarizing the characteristics of model-, small- and large-scale VE applications, and attributes of ecological validity that are important for the application of navigation research. The type of navigation participants performed, the scale and spatial extent of the environment, and the richness of the visual scene are used to provide a framework for a review of research into the effect of body-based information on navigation. The review resolves contradictions between previous studies' findings, identifies types of navigation interface that are suited to different applications, and highlights areas in which further research is needed. Applications that take place in small-scale environments, where maneuvering is the most demanding aspect of navigation, will benefit from full-walking interfaces. However, collision detection may not be needed because users avoid obstacles even when they are below eye-level. Applications that involve large-scale spaces (e.g., buildings or cities) just need to provide the translational component of body-based information, because it is only in unusual scenarios that the rotational component of body-based information produces any significant benefit. This opens up the opportunity of combining linear treadmill and walking-in-place interfaces with projection displays that provide a wide field of view

Where robots and virtual agents meet : a survey of social interaction across Milgram's Reality-Virtuality Continuum

Traditionally, social interaction research has concentrated on either fully virtually embodied agents (e.g. embodied conversational agents) or fully physically embodied agents (e.g. robots). For some time, however, both areas have started augmenting their agents’ capabilities for social interaction using ubiquitous and intelligent environments. We are placing different agent systems for social interaction along Milgram’s Reality-Virtuality Continuum—according to the degree they are embodied in a physical, virtual or mixed reality environment—and show systems that follow the next logical step in this progression, namely social interaction in the middle of Milgram’s continuum, that is, agents richly embodied in the physical and virtual world. This paper surveys the field of social interaction research with embodied agents with a particular view towards their embodiment forms and highlights some of the advantages and issues associated with the very recent field of social interaction with mixed reality agents.Science Foundation Irelan