Effects of prolonged exposure to feedback delay on the qualitative subjective experience of virtual reality

When interacting with virtual environments, feedback delays between making a movement and seeing the visual consequences of that movement are detrimental for the subjective quality of the VR experience. Here we used standard measures of subjective experiences such as ownership, agency and presence to investigate whether prolonged exposure to the delay, and thus the possibility to adapt to it, leads to the recovery of the qualitative experience of VR. Participants performed a target-tracking task in a Virtual Reality environment. We measured the participants' tracking performance in terms of spatial and temporal errors with respect to the target in both No-Delay and Delay conditions. Additionally, participants rated their sense of ``ownership'' of holding a virtual tool, agency and presence on each trial using sliding scales. These single trial ratings were compared to the results of the more traditional questionnaires for ownership and agency and presence for both No-Delay and Delay conditions. We found that the participants' sliding scales ratings corresponded very well to the scores obtained from the traditional questionnaires. Moreover, not only did participants behaviourally adapt to the delay, their ratings of ownership and agency significantly improved with prolonged exposure to the delay. Together the results suggest a tight link between the ability to perform a behavioural task and the subjective ratings of ownership and agency in virtual reality

Predictability is necessary for closed-loop visual feedback delay adaptation

Rohde M, van Dam L, Ernst MO. Predictability is necessary for closed-loop visual feedback delay adaptation. Journal of Vision. 2014;14(3):4.In case of delayed visual feedback during visuomotor tasks, like in some sluggish computer games, humans can modulate their behavior to compensate for the delay. However, opinions on the nature of this compensation diverge. Some studies suggest that humans adapt to feedback delays with lasting changes in motor behavior (aftereffects) and a recalibration of time perception. Other studies have shown little or no evidence for such semipermanent recalibration in the temporal domain. We hypothesize that predictability of the reference signal (target to be tracked) is necessary for semipermanent delay adaptation. To test this hypothesis, we trained participants with a 200 ms visual feedback delay in a visually guided manual tracking task, varying the predictability of the reference signal between conditions, but keeping reference motion and feedback delay constant. In Experiment 1, we focused on motor behavior. Only training in the predictable condition brings about all of the adaptive changes and aftereffects expected from delay adaptation. In Experiment 2, we used a synchronization task to investigate perceived simultaneity (perceptuomotor learning). Supporting the hypothesis, participants recalibrated subjective visuomotor simultaneity only when trained in the predictable condition. Such a shift in perceived simultaneity was also observed in Experiment 3, using an interval estimation task. These results show that delay adaptation in motor control can modulate the perceived temporal alignment of vision and kinesthetically sensed movement. The coadaptation of motor prediction and target prediction (reference extrapolation) seems necessary for such genuine delay adaptation. This offers an explanation for divergent results in the literature

No need to touch this: Bimanual haptic slant adaptation does not require touch

In our daily life, we often interact with objects using both hands raising the question the question to what extent information between the hands is shared. It has, for instance, been shown that curvature adaptation aftereffects can transfer from the adapted hand to the non-adapted hand. However, this transfer only occurred for dynamic exploration, e.g. by moving a single finger over a surface, but not for static exploration when keeping static contact with the surface and combining the information from different parts of the hand. This raises the question to what extent adaptation to object shape is shared between the hands when both hands are used in static fashion simultaneously and the object shape estimates require information from both hands. Here we addressed this question in three experiments using a slant adaptation paradigm. In Experiment 1 we investigated whether an aftereffect of static bimanual adaptation occurs at all and whether it transfers to conditions in which one hand was moving. In Experiment 2 participants adapted either to a felt slanted surface or simply be holding their hands in mid-air at similar positions, to investigate to what extent the effects of static bimanual adaptation are posture-based rather than object based. Experiment 3 further explored the idea that bimanual adaptation is largely posture based. We found that bimanual adaptation using static touch did lead to aftereffects when using the same static exploration mode for testing. However, the aftereffect did not transfer to any exploration mode that included a dynamic component. Moreover, we found similar aftereffects both with and without a haptic surface. Thus, we conclude that static bimanual adaptation is of proprioceptive nature and does not occur at the level at which the object is represented

Visual influence on bimanual haptic slant adaptation.

Adapting to particular features of a haptic shape, for example, the slant of a surface, affects how a subsequently touched shape is perceived (aftereffect). Previous studies showed that this adaptation is largely based on our proprioceptive sense of hand posture, yet the influence of vision on haptic shape adaptation has been relatively unexplored. Here, using a slant-adaptation paradigm, we investigated whether visual information affects haptic adaptation and, if so, how. To this end, we varied the available visual cues during the adaptation period. This process ranged from providing visual information only about the slant of the surface, or the reference frame in which it is presented, to only providing visual information about the location of the fingertips. Additionally, we tested several combinations of these visual cues. We show that, as soon as the visual information can be used as a spatial reference to link the own fingertip position to the surface slant, haptic adaptation is very much reduced. This result means that, under these viewing conditions, vision dominates touch and is one reason why we do not easily adapt to haptic shape in our daily life, because we usually have visual information about both hand and object available simultaneously

Exploration mode affects visuohaptic integration of surface orientation

Plaisier M, van Dam L, Glowania C, Ernst MO. Exploration mode affects visuohaptic integration of surface orientation. Journal of Vision. 2014;14(13):22.We experience the world mostly in a multisensory fashion using a combination of all of our senses. Depending on the modality we can select different exploration strategies for extracting perceptual information. For instance, using touch we can enclose an object in our hand to explore parts of the object in parallel. Alternatively, we can trace the object with a single finger to explore its parts in a serial fashion. In this study we investigated whether the exploration mode (parallel versus serial) affects the way sensory signals are combined. To this end, participants visually and haptically explored surfaces that varied in roll angle and indicated which side of the surface was perceived as higher. In Experiment 1, the exploration mode was the same for both modalities (i.e., both parallel or both serial). In Experiment 2, we introduced a difference in exploration mode between the two modalities (visual exploration was parallel while haptic exploration was serial or vice versa). The results showed that visual and haptic signals were combined in a statistically optimal fashion only when the exploration modes were the same. In case of an asymmetry in the exploration modes across modalities, integration was suboptimal. This indicates that spatial-temporal discrepancies in the acquisition of information in the two senses (i.e., haptic and visual) can lead to the break-down of sensory integration

Immersive-360° theatre: user experience in the virtual auditorium and platform efficacy for current and underserved audiences

The present study explores the efficacy and social potential of immersive-360° theater—live-captured theater performances filmed for virtual reality (VR) viewing—as a remote platform for audiences to view theater. We obtained survey and structured interview responses from 166 and 30 participants, respectively, self-categorized as regular theatergoers, novices, or underserved audiences. We measured immersion, presence, and emotional arousal in the virtual auditorium, technology acceptance, and social perceptions including platform compatibility with traditional theater and use as a psychosocial and accessibility promotion tool. Findings show that in the immersive-360° theater auditorium ratings for presence and immersion are mixed, and the latter is likely to be influenced by external factors including hardware quality and environmental distractors. For most, immersive-360° theater is regarded as a positive tool for psychosocial aspects and accessibility, but many highlighted the absence of social aspects which are central to the traditional theater experience and cannot be replicated in remote conditions. Despite this, the experience was enjoyable for most participants, and crucially, the majority of participants do not perceive immersive-360° theater as a “threat” to its traditional counterpart. Rather, with certain improvements it is seen as a compatible and complementary offering that has potential for use as a digital pipeline for underserved audiences and recruiting new patrons. Suggestions for improving the quality of the VR theater experience and its potential as an accessibility tool included improving headset quality and resolution, additional accessibility and user controls, and the ability to share the experience with somebody else in real time

"The Mystery of the Raddlesham Mumps": A case study for combined storytelling in a theatre play and Virtual Reality

“The Mystery of the Raddlesham Mumps” is a poem by Murray Lachlan Young, aimed at both children and adults. This poem has been adapted as a theatre play with a short prequel as a Virtual Reality (VR)/tablet app. We used this unique combination to explore the potential interaction between these different media elements for the level of “presence” and “immersion” in the story (i.e. the level to which one can imagine oneself within the story at the expense of the sense of physical time and space). The theatre audience had the opportunity to play the VR/tablet app in the foyer before the performance started. After the performance, a questionnaire measured participants’ level of immersion and presence in the theatre play and their enjoyment of both play and app. The results showed that people of all ages interacted with and liked the app. Ratings for the play were also high and did not depend on prior engagement with the app. However, the play was liked more by adults than children, and the reverse was true for the app, suggesting a potential generation shift in multimedia story telling

Knowing Each Random Error of Our Ways, but Hardly Correcting for It: an Instance of Optimal Performance

van Dam L, Ernst MO. Knowing Each Random Error of Our Ways, but Hardly Correcting for It: an Instance of Optimal Performance. PLOS ONE. 2013;8(10): e78757.Random errors are omnipresent in sensorimotor tasks due to perceptual and motor noise. The question is, are humans aware of their random errors on an instance-by-instance basis? The appealing answer would be ‘no’ because it seems intuitive that humans would otherwise immediately correct for the errors online, thereby increasing sensorimotor precision. However, here we show the opposite. Participants pointed to visual targets with varying degree of feedback. After movement completion participants indicated whether they believed they landed left or right of target. Surprisingly, participants' left/right-discriminability was well above chance, even without visual feedback. Only when forced to correct for the error after movement completion did participants loose knowledge about the remaining error, indicating that random errors can only be accessed offline. When correcting, participants applied the optimal correction gain, a weighting factor between perceptual and motor noise, minimizing end-point variance. Together these results show that humans optimally combine direct information about sensorimotor noise in the system (the current random error), with indirect knowledge about the variance of the perceptual and motor noise distributions. Yet, they only appear to do so offline after movement completion, not while the movement is still in progress, suggesting that during movement proprioceptive information is less precise

Alzheimer's disease disrupts alpha and beta-band resting-state oscillatory network connectivity

Objective: Neuroimaging studies in Alzheimer’s disease (AD) yield conflicting results due to selective investigation. We conducted a comprehensive magnetoencephalography study of connectivity changes in AD and healthy ageing in the resting-state. Methods: We performed a whole-brain, source-space assessment of oscillatory neural signalling in multiple frequencies comparing AD patients, elderly and young controls. We compared eyes-open and closed group oscillatory envelope activity in networks obtained through temporal independent component analysis, and calculated whole-brain node-based amplitude and phase connectivity. Results: In bilateral parietotemporal areas, oscillatory envelope amplitude increased with healthy ageing, whereas both local amplitude and node-to-global connectivity decreased with AD. AD-related decreases were spatially specific and restricted to the alpha and beta bands. A significant proportion of the variance in areas of peak group difference was explained by cognitive integrity, in addition to group. None of the groups differed in phase connectivity. Results were highly similar for eyes-open and closed resting-state. Conclusions: These results support the disconnection syndrome hypothesis and suggest that AD shows distinct and unique patterns of disrupted neural functioning, rather than accelerated healthy ageing. Significance: Whole-brain assessments show that disrupted regional oscillatory envelope amplitude and connectivity in the alpha and beta bands play a key role in AD