Virtual Hand Illusion Induced by Visuomotor Correlations

Background: Our body schema gives the subjective impression of being highly stable. However, a number of easily-evoked illusions illustrate its remarkable malleability. In the rubber-hand illusion, illusory ownership of a rubber-hand is evoked by synchronous visual and tactile stimulation on a visible rubber arm and on the hidden real arm. Ownership is concurrent with a proprioceptive illusion of displacement of the arm position towards the fake arm. We have previously shown that this illusion of ownership plus the proprioceptive displacement also occurs towards a virtual 3D projection of an arm when the appropriate synchronous visuotactile stimulation is provided. Our objective here was to explore whether these illusions (ownership and proprioceptive displacement) can be induced by only synchronous visuomotor stimulation, in the absence of tactile stimulation.Methodology/Principal Findings: To achieve this we used a data-glove that uses sensors transmitting the positions of fingers to a virtually projected hand in the synchronous but not in the asynchronous condition. The illusion of ownership was measured by means of questionnaires. Questions related to ownership gave significantly larger values for the synchronous than for the asynchronous condition. Proprioceptive displacement provided an objective measure of the illusion and had a median value of 3.5 cm difference between the synchronous and asynchronous conditions. In addition, the correlation between the feeling of ownership of the virtual arm and the size of the drift was significant.Conclusions/Significance: We conclude that synchrony between visual and proprioceptive information along with motor activity is able to induce an illusion of ownership over a virtual arm. This has implications regarding the brain mechanisms underlying body ownership as well as the use of virtual bodies in therapies and rehabilitation

Enhancing our lives with immersive virtual reality

Virtual reality (VR) started about 50 years ago in a form we would recognize today [stereo head-mounted display (HMD), head tracking, computer graphics generated images] – although the hardware was completely different. In the 1980s and 1990s, VR emerged again based on a different generation of hardware (e.g., CRT displays rather than vector refresh, electromagnetic tracking instead of mechanical). This reached the attention of the public, and VR was hailed by many engineers, scientists, celebrities, and business people as the beginning of a new era, when VR would soon change the world for the better. Then, VR disappeared from public view and was rumored to be “dead.” In the intervening 25 years a huge amount of research has nevertheless been carried out across a vast range of applications – from medicine to business, from psychotherapy to industry, from sports to travel. Scientists, engineers, and people working in industry carried on with their research and applications using and exploring different forms of VR, not knowing that actually the topic had already passed away. The purpose of this article is to survey a range of VR applications where there is some evidence for, or at least debate about, its utility, mainly based on publications in peer-reviewed journals. Of course not every type of application has been covered, nor every scientific paper (about 186,000 papers in Google Scholar): in particular, in this review we have not covered applications in psychological or medical rehabilitation. The objective is that the reader becomes aware of what has been accomplished in VR, where the evidence is weaker or stronger, and what can be done. We start in Section 1 with an outline of what VR is and the major conceptual framework used to understand what happens when people experience it – the concept of “presence.” In Section 2, we review some areas where VR has been used in science – mostly psychology and neuroscience, the area of scientific visualization, and some remarks about its use in education and surgical training. In Section 3, we discuss how VR has been used in sports and exercise. In Section 4, we survey applications in social psychology and related areas – how VR has been used to throw light on some social phenomena, and how it can be used to tackle experimentally areas that cannot be studied experimentally in real life. We conclude with how it has been used in the preservation of and access to cultural heritage. In Section 5, we present the domain of moral behavior, including an example of how it might be used to train professionals such as medical doctors when confronting serious dilemmas with patients. In Section 6, we consider how VR has been and might be used in various aspects of travel, collaboration, and industry. In Section 7, we consider mainly the use of VR in news presentation and also discuss different types of VR. In the concluding Section 8, we briefly consider new ideas that have recently emerged – an impossible task since during the short time we have written this page even newer ideas have emerged! And, we conclude with some general considerations and speculations. Throughout and wherever possible we have stressed novel applications and approaches and how the real power of VR is not necessarily to produce a faithful reproduction of “reality” but rather that it offers the possibility to step outside of the normal bounds of reality and realize goals in a totally new and unexpected way. We hope that our article will provoke readers to think as paradigm changers, and advance VR to realize different worlds that might have a positive impact on the lives of millions of people worldwide, and maybe even help a little in saving the planet

Violating body movement semantics: Neural signatures of self-generated and external-generated errors

How do we recognize ourselves as the agents of our actions? Do we use the same error detection mechanisms to monitor self-generated vs. externally imposed actions? Using event-related brain potentials (ERPs), we identified two different error-monitoring loops involved in providing a coherent sense of the agency of our actions. In the first ERP experiment, the participants were embodied in a virtual body (avatar) while performing an error-prone fast reaction time task. Crucially, in certain trials, participants were deceived regarding their own actions, i.e., the avatar movement did not match the participant's movement. Self-generated real errors and false (avatar) errors showed very different ERP signatures and with different processing latencies: while real errors showed a classical frontal-central error-related negativity (Ne/ERN), peaking 100 ms after error commission, false errors elicited a larger and delayed parietal negative component (at about 350–400 ms). The violation of the sense of agency elicited by false avatar errors showed a strong similarity to ERP signatures related to semantic or conceptual violations (N400 component). In a follow-up ERP control experiment, a subset of the same participants merely acted as observers of the avatar correct and error movements. This experimental situation did not elicit the N400 component associated with agency violation. Thus, the results show a clear neural dissociation between internal and external error-monitoring loops responsible for distinguishing our self-generated errors from those imposed externally, opening new avenues for the study of the mental processes underlying the integration of internal and sensory feedback information while being actors of our own actions

Is my hand connected to my body? The impact of body continuity and arm alignment on the virtual hand illusion

When a rubber hand is placed on a table top in a plausible position as if part of a person"s body, and is stroked synchronously with the person"s corresponding hidden real hand, an illusion of ownership over the rubber hand can occur (Botvinick and Cohen 1998). A similar result has been found with respect to a virtual hand portrayed in a virtual environment, a virtual hand illusion (Slater et al. 2008). The conditions under which these illusions occur have been the subject of considerable study. Here we exploited the flexibility of virtual reality to examine four contributory factors: visuo-tactile synchrony while stroking the virtual and the real arms, body continuity, alignment between the real and virtual arms, and the distance between them. We carried out three experiments on a total of 32 participants where these factors were varied. The results show that the subjective illusion of ownership over the virtual arm and the time to evoke this illusion are highly dependent on synchronous visuo-tactile stimulation and on connectivity of the virtual arm with the rest of the virtual body. The alignment between the real and virtual arms and the distance between these were less important. It was found that proprioceptive drift was not a sensitive measure of the illusion, but was only related to the distance between the real and virtual arms

The Rocketbox Library and the Utility of Freely Available Rigged Avatars

As part of the open sourcing of the Microsoft Rocketbox avatar library for research and academic purposes, here we discuss the importance of rigged avatars for the Virtual and Augmented Reality (VR, AR) research community. Avatars, virtual representations of humans, are widely used in VR applications. Furthermore many research areas ranging from crowd simulation to neuroscience, psychology, or sociology have used avatars to investigate new theories or to demonstrate how they influence human performance and interactions. We divide this paper in two main parts: the first one gives an overview of the different methods available to create and animate avatars. We cover the current main alternatives for face and body animation as well introduce upcoming capture methods. The second part presents the scientific evidence of the utility of using rigged avatars for embodiment but also for applications such as crowd simulation and entertainment. All in all this paper attempts to convey why rigged avatars will be key to the future of VR and its wide adoption

First Person Experience of Body Transfer in Virtual Reality

Background: Altering the normal association between touch and its visual correlate can result in the illusory perception of a fake limb as part of our own body. Thus, when touch is seen to be applied to a rubber hand while felt synchronously on the corresponding hidden real hand, an illusion of ownership of the rubber hand usually occurs. The illusion has also been demonstrated using visuomotor correlation between the movements of the hidden real hand and the seen fake hand. This type of paradigm has been used with respect to the whole body generating out-of-the-body and body substitution illusions. However, such studies have only ever manipulated a single factor and although they used a form of virtual reality have not exploited the power of immersive virtual reality (IVR) to produce radical transformations in body ownership.Principal Findings: Here we show that a first person perspective of a life-sized virtual human female body that appears to substitute the male subjects' own bodies was sufficient to generate a body transfer illusion. This was demonstrated subjectively by questionnaire and physiologically through heart-rate deceleration in response to a threat to the virtual body. This finding is in contrast to earlier experimental studies that assume visuotactile synchrony to be the critical contributory factor in ownership illusions. Our finding was possible because IVR allowed us to use a novel experimental design for this type of problem with three independent binary factors: (i) perspective position (first or third), (ii) synchronous or asynchronous mirror reflections and (iii) synchrony or asynchrony between felt and seen touch.Conclusions: The results support the notion that bottom-up perceptual mechanisms can temporarily override top down knowledge resulting in a radical illusion of transfer of body ownership. The research also illustrates immersive virtual reality as a powerful tool in the study of body representation and experience, since it supports experimental manipulations that would otherwise be infeasible, with the technology being mature enough to represent human bodies and their motion

Altered visual feedback from an embodied avatar unconsciously influences movement amplitude and muscle activity

Evidence suggests that the sense of the position of our body parts can be surreptitiously deceived, for instance through illusory visual inputs. However, whether altered visual feedback during limb movement can induce substantial unconscious motor and muscular adjustments is not known. To address this question, we covertly manipulated virtual body movements in immersive virtual reality. Participants were instructed to flex their elbow to 90° while tensing an elastic band, as their virtual arm reproduced the same, a reduced (75°), or an amplified (105°) movement. We recorded muscle activity using electromyography, and assessed body ownership, agency and proprioception of the arm. Our results not only show that participants compensated for the avatar’s manipulated arm movement while being completely unaware of it, but also that it is possible to induce unconscious motor adaptations requiring significant changes in muscular activity. Altered visual feedback through body ownership illusions can influence motor performance in a process that bypasses awareness

Beaming into the Rat World: Enabling Real-Time Interaction between Rat and Human Each at Their Own Scale

Immersive virtual reality (IVR) typically generates the illusion in participants that they are in the displayed virtual scene where they can experience and interact in events as if they were really happening. Teleoperator (TO) systems place people at a remote physical destination embodied as a robotic device, and where typically participants have the sensation of being at the destination, with the ability to interact with entities there. In this paper, we show how to combine IVR and TO to allow a new class of application. The participant in the IVR is represented in the destination by a physical robot (TO) and simultaneously the remote place and entities within it are represented to the participant in the IVR. Hence, the IVR participant has a normal virtual reality experience, but where his or her actions and behaviour control the remote robot and can therefore have physical consequences. Here, we show how such a system can be deployed to allow a human and a rat to operate together, but the human interacting with the rat on a human scale, and the rat interacting with the human on the rat scale. The human is represented in a rat arena by a small robot that is slaved to the human’s movements, whereas the tracked rat is represented to the human in the virtual reality by a humanoid avatar. We describe the system and also a study that was designed to test whether humans can successfully play a game with the rat. The results show that the system functioned well and that the humans were able to interact with the rat to fulfil the tasks of the game. This system opens up the possibility of new applications in the life sciences involving participant observation of and interaction with animals but at human scale

Multisensory Stimulation Can Induce an Illusion of Larger Belly Size in Immersive Virtual Reality

Background: Body change illusions have been of great interest in recent years for the understanding of how the brain represents the body. Appropriate multisensory stimulation can induce an illusion of ownership over a rubber or virtual arm, simple types of out-of-the-body experiences, and even ownership with respect to an alternate whole body. Here we use immersive virtual reality to investigate whether the illusion of a dramatic increase in belly size can be induced in males through (a) first person perspective position (b) synchronous visual-motor correlation between real and virtual arm movements, and (c) self-induced synchronous visual-tactile stimulation in the stomach area.Methodology: Twenty two participants entered into a virtual reality (VR) delivered through a stereo head-tracked wide field-of-view head-mounted display. They saw from a first person perspective a virtual body substituting their own that had an inflated belly. For four minutes they repeatedly prodded their real belly with a rod that had a virtual counterpart that they saw in the VR. There was a synchronous condition where their prodding movements were synchronous with what they felt and saw and an asynchronous condition where this was not the case. The experiment was repeated twice for each participant in counter-balanced order. Responses were measured by questionnaire, and also a comparison of before and after self-estimates of belly size produced by direct visual manipulation of the virtual body seen from the first person perspective.Conclusions: The results show that first person perspective of a virtual body that substitutes for the own body in virtual reality, together with synchronous multisensory stimulation can temporarily produce changes in body representation towards the larger belly size. This was demonstrated by (a) questionnaire results, (b) the difference between the self-estimated belly size, judged from a first person perspective, after and before the experimental manipulation, and (c) significant positive correlations between these two measures. We discuss this result in the general context of body ownership illusions, and suggest applications including treatment for body size distortion illnesses

Talk to the Virtual Hands: Self-Animated Avatars Improve Communication in Head-Mounted Display Virtual Environments

Background When we talk to one another face-to-face, body gestures accompany our speech. Motion tracking technology enables us to include body gestures in avatar-mediated communication, by mapping one's movements onto one's own 3D avatar in real time, so the avatar is self-animated. We conducted two experiments to investigate (a) whether head-mounted display virtual reality is useful for researching the influence of body gestures in communication; and (b) whether body gestures are used to help in communicating the meaning of a word. Participants worked in pairs and played a communication game, where one person had to describe the meanings of words to the other. Principal Findings In experiment 1, participants used significantly more hand gestures and successfully described significantly more words when nonverbal communication was available to both participants (i.e. both describing and guessing avatars were self-animated, compared with both avatars in a static neutral pose). Participants ‘passed’ (gave up describing) significantly more words when they were talking to a static avatar (no nonverbal feedback available). In experiment 2, participants' performance was significantly worse when they were talking to an avatar with a prerecorded listening animation, compared with an avatar animated by their partners' real movements. In both experiments participants used significantly more hand gestures when they played the game in the real world. Conclusions Taken together, the studies show how (a) virtual reality can be used to systematically study the influence of body gestures; (b) it is important that nonverbal communication is bidirectional (real nonverbal feedback in addition to nonverbal communication from the describing participant); and (c) there are differences in the amount of body gestures that participants use with and without the head-mounted display, and we discuss possible explanations for this and ideas for future investigation