An Overview of Self-Adaptive Technologies Within Virtual Reality Training

This overview presents the current state-of-the-art of self-adaptive technologies within virtual reality (VR) training. Virtual reality training and assessment is increasingly used for five key areas: medical, industrial & commercial training, serious games, rehabilitation and remote training such as Massive Open Online Courses (MOOCs). Adaptation can be applied to five core technologies of VR including haptic devices, stereo graphics, adaptive content, assessment and autonomous agents. Automation of VR training can contribute to automation of actual procedures including remote and robotic assisted surgery which reduces injury and improves accuracy of the procedure. Automated haptic interaction can enable tele-presence and virtual artefact tactile interaction from either remote or simulated environments. Automation, machine learning and data driven features play an important role in providing trainee-specific individual adaptive training content. Data from trainee assessment can form an input to autonomous systems for customised training and automated difficulty levels to match individual requirements. Self-adaptive technology has been developed previously within individual technologies of VR training. One of the conclusions of this research is that while it does not exist, an enhanced portable framework is needed and it would be beneficial to combine automation of core technologies, producing a reusable automation framework for VR training

Virtual reality for stroke rehabilitation (review)

Published version made available following 12 month embargo from the date of publication [12 Feb 2015] according to publisher policy. Accessed 10/03/2015. Published version available from 13 February 2016

Multimodality in VR: A survey

Virtual reality (VR) is rapidly growing, with the potential to change the way we create and consume content. In VR, users integrate multimodal sensory information they receive, to create a unified perception of the virtual world. In this survey, we review the body of work addressing multimodality in VR, and its role and benefits in user experience, together with different applications that leverage multimodality in many disciplines. These works thus encompass several fields of research, and demonstrate that multimodality plays a fundamental role in VR; enhancing the experience, improving overall performance, and yielding unprecedented abilities in skill and knowledge transfer

User experience of mixed reality applications for healthy ageing : A systematic review

Mixed reality (MR) technologies are being used increasingly to support healthy ageing, but past reviews have concentrated on the efficacy of the technology. This systematic review provides a synthesis of recent experimental studies on the instrumental, emotional and non-instrumental aspects of user experience of healthy older adults in relation to MR-related applications. The review was listed on PROSPERO, utilised a modified PICOS framework, and canvassed all published work between January 2010 to July 2021 that appeared in major databases (Scopus, PubMed, CINAHL, Web of Science, and the Cochrane Library). The literature search revealed 15 eligible studies. Results indicated that all included studies measured the instrumental quality of their applications, all but two studies measured the emotional reactions triggered by gameplay, and only six studies examined participants’ perception of non-instrumental quality of the applications. All included studies focused on improving a health domain such as cognitive or physical training. This suggests that the instrumental quality of the MR applications remains the focus of user experience studies, with far fewer studies examining the non-instrumental quality of the applications. Implications for game design and future research are discussed

Virtual reality for stroke rehabilitation (Review)

Copyright © 2017 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. This review is made available in accordance with Cochrane Database of Systematic Review's repositories policyBackground Virtual reality and interactive video gaming have emerged as recent treatment approaches in stroke rehabilitation with commercial gaming consoles in particular, being rapidly adopted in clinical settings. This is an update of a Cochrane Review published first in 2011 and then again in 2015. Objectives Primary objective: to determine the efficacy of virtual reality compared with an alternative intervention or no intervention on upper limb function and activity. Secondary objectives: to determine the efficacy of virtual reality compared with an alternative intervention or no intervention on: gait and balance, global motor function, cognitive function, activity limitation, participation restriction, quality of life, and adverse events. Search methods We searched the Cochrane Stroke Group Trials Register (April 2017), CENTRAL, MEDLINE, Embase, and seven additional databases. We also searched trials registries and reference lists. Selection criteria Randomised and quasi‐randomised trials of virtual reality ("an advanced form of human‐computer interface that allows the user to 'interact' with and become 'immersed' in a computer‐generated environment in a naturalistic fashion") in adults after stroke. The primary outcome of interest was upper limb function and activity. Secondary outcomes included gait and balance and global motor function. Data collection and analysis Two review authors independently selected trials based on pre‐defined inclusion criteria, extracted data, and assessed risk of bias. A third review author moderated disagreements when required. The review authors contacted investigators to obtain missing information. Main results We included 72 trials that involved 2470 participants. This review includes 35 new studies in addition to the studies included in the previous version of this review. Study sample sizes were generally small and interventions varied in terms of both the goals of treatment and the virtual reality devices used. The risk of bias present in many studies was unclear due to poor reporting. Thus, while there are a large number of randomised controlled trials, the evidence remains mostly low quality when rated using the GRADE system. Control groups usually received no intervention or therapy based on a standard‐care approach. Primary outcome: results were not statistically significant for upper limb function (standardised mean difference (SMD) 0.07, 95% confidence intervals (CI) ‐0.05 to 0.20, 22 studies, 1038 participants, low‐quality evidence) when comparing virtual reality to conventional therapy. However, when virtual reality was used in addition to usual care (providing a higher dose of therapy for those in the intervention group) there was a statistically significant difference between groups (SMD 0.49, 0.21 to 0.77, 10 studies, 210 participants, low‐quality evidence). Secondary outcomes: when compared to conventional therapy approaches there were no statistically significant effects for gait speed or balance. Results were statistically significant for the activities of daily living (ADL) outcome (SMD 0.25, 95% CI 0.06 to 0.43, 10 studies, 466 participants, moderate‐quality evidence); however, we were unable to pool results for cognitive function, participation restriction, or quality of life. Twenty‐three studies reported that they monitored for adverse events; across these studies there were few adverse events and those reported were relatively mild. Authors' conclusions We found evidence that the use of virtual reality and interactive video gaming was not more beneficial than conventional therapy approaches in improving upper limb function. Virtual reality may be beneficial in improving upper limb function and activities of daily living function when used as an adjunct to usual care (to increase overall therapy time). There was insufficient evidence to reach conclusions about the effect of virtual reality and interactive video gaming on gait speed, balance, participation, or quality of life. This review found that time since onset of stroke, severity of impairment, and the type of device (commercial or customised) were not strong influencers of outcome. There was a trend suggesting that higher dose (more than 15 hours of total intervention) was preferable as were customised virtual reality programs; however, these findings were not statistically significant

Video capture virtual reality as a flexible and effective rehabilitation tool

Video capture virtual reality (VR) uses a video camera and software to track movement in a single plane without the need to place markers on specific bodily locations. The user's image is thereby embedded within a simulated environment such that it is possible to interact with animated graphics in a completely natural manner. Although this technology first became available more than 25 years ago, it is only within the past five years that it has been applied in rehabilitation. The objective of this article is to describe the way this technology works, to review its assets relative to other VR platforms, and to provide an overview of some of the major studies that have evaluated the use of video capture technologies for rehabilitation

Multimodality in {VR}: {A} Survey

Virtual reality has the potential to change the way we create and consume content in our everyday life. Entertainment, training, design and manufacturing, communication, or advertising are all applications that already benefit from this new medium reaching consumer level. VR is inherently different from traditional media: it offers a more immersive experience, and has the ability to elicit a sense of presence through the place and plausibility illusions. It also gives the user unprecedented capabilities to explore their environment, in contrast with traditional media. In VR, like in the real world, users integrate the multimodal sensory information they receive to create a unified perception of the virtual world. Therefore, the sensory cues that are available in a virtual environment can be leveraged to enhance the final experience. This may include increasing realism, or the sense of presence; predicting or guiding the attention of the user through the experience; or increasing their performance if the experience involves the completion of certain tasks. In this state-of-the-art report, we survey the body of work addressing multimodality in virtual reality, its role and benefits in the final user experience. The works here reviewed thus encompass several fields of research, including computer graphics, human computer interaction, or psychology and perception. Additionally, we give an overview of different applications that leverage multimodal input in areas such as medicine, training and education, or entertainment; we include works in which the integration of multiple sensory information yields significant improvements, demonstrating how multimodality can play a fundamental role in the way VR systems are designed, and VR experiences created and consumed

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion

Virtual reality based upper extremity rehabilitation following stroke: a review

In the last decade there have been major developments in the creation of interactive virtual scenarios for the rehabilitation of motor deficits following stroke. Virtual reality technology is arising as a promising tool to diagnose, monitor and induce functional recovery after lesions to the nervous system. This evidence has grown in the last few years, as effort has been made to develop virtual scenarios that are built on the knowledge of mechanisms of recovery. In this paper we review the state of the art virtual reality techniques for rehabilitation of functionality of the upper extremities following stroke. We refer to some of the main systems that have been developed within different rehabilitative approaches such as learning by imitation, reinforced feedback, haptic feedback, augmented practice and repetition, video capture virtual reality, exoskeletons, mental practice, action observation and execution, and others. The major findings of these studies show that virtual reality technologies will become a more and more essential ingredient in the treatment of stroke and other disorders of the nervous system.info:eu-repo/semantics/publishedVersio

Designing sensory-substitution devices: Principles, pitfalls and potential1

An exciting possibility for compensating for loss of sensory function is to augment deficient senses by conveying missing information through an intact sense. Here we present an overview of techniques that have been developed for sensory substitution (SS) for the blind, through both touch and audition, with special emphasis on the importance of training for the use of such devices, while highlighting potential pitfalls in their design. One example of a pitfall is how conveying extra information about the environment risks sensory overload. Related to this, the limits of attentional capacity make it important to focus on key information and avoid redundancies. Also, differences in processing characteristics and bandwidth between sensory systems severely constrain the information that can be conveyed. Furthermore, perception is a continuous process and does not involve a snapshot of the environment. Design of sensory substitution devices therefore requires assessment of the nature of spatiotemporal continuity for the different senses. Basic psychophysical and neuroscientific research into representations of the environment and the most effective ways of conveying information should lead to better design of sensory substitution systems. Sensory substitution devices should emphasize usability, and should not interfere with other inter- or intramodal perceptual function. Devices should be task-focused since in many cases it may be impractical to convey too many aspects of the environment. Evidence for multisensory integration in the representation of the environment suggests that researchers should not limit themselves to a single modality in their design. Finally, we recommend active training on devices, especially since it allows for externalization, where proximal sensory stimulation is attributed to a distinct exterior object.This work was supported by the European Union’s Horizon 2020 Research and Innovative Programme under grant agreement no. 643636, “Sound of Vision”.Peer Reviewe