Facilitating Self-monitored Physical Rehabilitation with Virtual Reality and Haptic feedback

Physical rehabilitation is essential to recovery from joint replacement operations. As a representation, total knee arthroplasty (TKA) requires patients to conduct intensive physical exercises to regain the knee's range of motion and muscle strength. However, current joint replacement physical rehabilitation methods rely highly on therapists for supervision, and existing computer-assisted systems lack consideration for enabling self-monitoring, making at-home physical rehabilitation difficult. In this paper, we investigated design recommendations that would enable self-monitored rehabilitation through clinical observations and focus group interviews with doctors and therapists. With this knowledge, we further explored Virtual Reality(VR)-based visual presentation and supplemental haptic motion guidance features in our implementation VReHab, a self-monitored and multimodal physical rehabilitation system with VR and vibrotactile and pneumatic feedback in a TKA rehabilitation context. We found that the third point of view real-time reconstructed motion on a virtual avatar overlaid with the target pose effectively provides motion awareness and guidance while haptic feedback helps enhance users' motion accuracy and stability. Finally, we implemented \systemname to facilitate self-monitored post-operative exercises and validated its effectiveness through a clinical study with 10 patients

Evaluating the Effects of Immersive Embodied Interaction on Cognition in Virtual Reality

Virtual reality is on its advent of becoming mainstream household technology, as technologies such as head-mounted displays, trackers, and interaction devices are becoming affordable and easily available. Virtual reality (VR) has immense potential in enhancing the fields of education and training, and its power can be used to spark interest and enthusiasm among learners. It is, therefore, imperative to evaluate the risks and benefits that immersive virtual reality poses to the field of education. Research suggests that learning is an embodied process. Learning depends on grounded aspects of the body including action, perception, and interactions with the environment. This research aims to study if immersive embodiment through the means of virtual reality facilitates embodied cognition. A pedagogical VR solution which takes advantage of embodied cognition can lead to enhanced learning benefits. Towards achieving this goal, this research presents a linear continuum for immersive embodied interaction within virtual reality. This research evaluates the effects of three levels of immersive embodied interactions on cognitive thinking, presence, usability, and satisfaction among users in the fields of science, technology, engineering, and mathematics (STEM) education. Results from the presented experiments show that immersive virtual reality is greatly effective in knowledge acquisition and retention, and highly enhances user satisfaction, interest and enthusiasm. Users experience high levels of presence and are profoundly engaged in the learning activities within the immersive virtual environments. The studies presented in this research evaluate pedagogical VR software to train and motivate students in STEM education, and provide an empirical analysis comparing desktop VR (DVR), immersive VR (IVR), and immersive embodied VR (IEVR) conditions for learning. This research also proposes a fully immersive embodied interaction metaphor (IEIVR) for learning of computational concepts as a future direction, and presents the challenges faced in implementing the IEIVR metaphor due to extended periods of immersion. Results from the conducted studies help in formulating guidelines for virtual reality and education researchers working in STEM education and training, and for educators and curriculum developers seeking to improve student engagement in the STEM fields

Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review

It is generally accepted that augmented feedback, provided by a human expert or a technical display, effectively enhances motor learning. However, discussion of the way to most effectively provide augmented feedback has been controversial. Related studies have focused primarily on simple or artificial tasks enhanced by visual feedback. Recently, technical advances have made it possible also to investigate more complex, realistic motor tasks and to implement not only visual, but also auditory, haptic, or multimodal augmented feedback. The aim of this review is to address the potential of augmented unimodal and multimodal feedback in the framework of motor learning theories. The review addresses the reasons for the different impacts of feedback strategies within or between the visual, auditory, and haptic modalities and the challenges that need to be overcome to provide appropriate feedback in these modalities, either in isolation or in combination. Accordingly, the design criteria for successful visual, auditory, haptic, and multimodal feedback are elaborate

Movement kinematic and postural control differences when performing a visuomotor skill in real and virtual environments

This is the author accepted manuscript.Availability of data, material and code: All relevant data and code is available online from: https://osf.io/h8az7/Immersive technologies, like virtual and mixed reality, pose a novel challenge for our sensorimotor systems as they deliver simulated sensory inputs that may not match those of the natural environment. These include reduced fields of view, missing or inaccurate haptic information, and distortions of 3D space; differences that may impact the control of motor actions. For instance, reach-to-grasp movements without end-point haptic feedback are characterised by slower and more exaggerated movements. A general uncertainty about sensory input may also induce a more conscious form of movement control. We tested whether a more complex skill like golf putting was also characterised by more consciously controlled movement. In a repeated-measures design, kinematics of the putter swing and postural control were compared between i) real-world putting, ii) VR putting, and iii) VR putting with haptic feedback from a real ball (i.e., mixed reality). Differences in putter swing were observed both between the real-world and VR, and between VR conditions with and without haptic information. Further, clear differences in postural control emerged between real and virtual putting, with both VR conditions characterised by larger postural movements, which were more regular and less complex, suggesting a more conscious form of balance control. Conversely, participants actually reported less conscious awareness of their movements in VR. These findings highlight how fundamental movement differences may exist between virtual and natural environments, which may pose challenges for transfer of learning within applications to motor rehabilitation and sport.Leverhulme Trus

Altering User Movement Behaviour in Virtual Environments.

In immersive Virtual Reality systems, users tend to move in a Virtual Environment as they would in an analogous physical environment. In this work, we investigated how user behaviour is affected when the Virtual Environment differs from the physical space. We created two sets of four environments each, plus a virtual replica of the physical environment as a baseline. The first focused on aesthetic discrepancies, such as a water surface in place of solid ground. The second focused on mixing immaterial objects together with those paired to tangible objects. For example, barring an area with walls or obstacles. We designed a study where participants had to reach three waypoints laid out in such a way to prompt a decision on which path to follow based on the conflict between the mismatching visual stimuli and their awareness of the real layout of the room. We analysed their performances to determine whether their trajectories were altered significantly from the shortest route. Our results indicate that participants altered their trajectories in presence of surfaces representing higher walking difficulty (for example, water instead of grass). However, when the graphical appearance was found to be ambiguous, there was no significant trajectory alteration. The environments mixing immaterial with physical objects had the most impact on trajectories with a mean deviation from the shortest route of 60 cm against the 37 cm of environments with aesthetic alterations. The co-existance of paired and unpaired virtual objects was reported to support the idea that all objects participants saw were backed by physical props. From these results and our observations, we derive guidelines on how to alter user movement behaviour in Virtual Environments

Exploring Adaptation-Based Techniques to Create Comfortable Virtual Reality Experiences

Virtual reality (VR) is transitioning from research to widespread consumer use. However, VR sickness - a type of motion sickness associated with VR usage - is believed to be a major impediment to the mass adoption of VR and it is estimatedto affect more than two-thirds of VR users. Previous research also shows that VR sickness affects some vulnerable groups, such as women, more severely than other groups. Although several strategies have been developed to mitigate VR sickness, most of them are not equally effective for all users since the effectiveness of any particular strategy varies across individuals. There are also concerns that some widely used VR sickness mitigation strategies, such as field-of-view (FOV) restriction, may have negative consequences on women. This thesis aims: 1) to provide theoretical understanding of the aspects of VR systems that cause VR sickness to affect some user more than others, with focus on sex differences, and 2) to develop adaptation-based strategies that could mitigate VR sickness for all VR users irrespective of their differences.Towards these goals, I first investigate the effectiveness of FOV restriction in reducing VR sickness across genders, and it’s effects on women’s spatial navigation ability. Then, based on findings from the first set of studies, I develop andempirically evaluate a novel adaptive eye gaze-contingent FOV restrictor that allows users to have a wider visual field while blocking their peripheral FOV. The wider visual field would be beneficial for women’s spatial navigation performance in virtual environments. Finally, I provide a novel standardized adaptation-based training paradigm that supplements existing VR sickness mitigation techniques by allowing the user to best prepare themselves for continued VR use. Evaluation of this strategy suggests that it could reduce and even eliminate VR sickness in susceptible individuals irrespective of their individual differences

Tadpole VR: virtual reality visualization of a simulated tadpole spinal cord

Recent advances in “developmental” approach (combining experimental study with computational modelling) of neural networks produces increasingly large data sets, in both complexity and size. This poses a significant challenge in analyzing, visualizing and understanding not only the spatial structure but also the behavior of such networks. This paper describes a Virtual Reality application for visualization of two biologically accurate computational models that model the anatomical structure of a neural network comprised of 1,500 neurons and over 80,000 connections. The visualization enables a user to observe the complex spatio-temporal interplay between seven unique types of neurons culminating in an observable swimming pattern. We present a detailed description of the design approach for the virtual environment, based on a set of initial requirements, followed up by the implementation and optimization steps. Lastly, the results of a pilot usability study are being presented on how confident participants are in their ability to understand how the alternating firing pattern between the two sides of the tadpole’s body generate swimming motion

Varying Feedback Strategy and Scheduling in Simulator Training: Effects on Learner Perceptions, Initial Learning, and Transfer

This experimental study investigated the effects of visual feedback on initial learning, perceived self-efficacy, workload, near transfer, far transfer, and perceived realism during a simulator-based training task. Prior studies indicate that providing feedback is critical for schema development (Salmoni, Schmidt, & Walter 1984; Sterman, 1994). However, its influence has been shown to dissipate and is not directly proportionate to the frequency at which it is given (Wulf, Shea, & Matschiner, 1998). A total of 54 participants completed the study forming six treatment groups. The independent treatment, visual feedback, was manipulated as scheduling (absolute—every practice trial or relative—every third trial) and strategies (gradual decrease of visual cues within the interface, gradual increase of visual cues within the interface, or a single consistent cue for each trial). Participants completed twelve practice trials of welding under one of six feedback manipulations; then, participants completed twelve practice trials of welding without it. Lastly, participants performed the weld task on actual equipment in a shop area. No treatment showed significant difference among groups with regard to initial learning, retention, near transfer, and far transfer measures. However, a statistical significance was found during initial learning and retention within each treatment group. Findings support empirical evidence that a variability of practice paradigm promotes learning (Lee & Carnahan, 1990; Shea & Morgan, 1979). Learner perceptions of realism suggest that novice learners perceive simulator fidelity as high, however, these perceptions may dissipate as the learner practices. Those groups that involved the greatest number of cues at the onset of practice or having cues available at every other trial reported the greatest amount of workload. All groups reported increases in perceptions of self-efficacy during practice on the simulator, but those perceptions decreased when participants performed the weld task on actual equipment. Findings suggest that contextual-interference of increasing, decreasing, or changing feedback counteracts the guidance effect of feedback as found in previous studies

14th Annual Focus on Creative Inquiry Poster Forum Program

The Focus on Creative Inquiry (FoCI) Poster Forum is an annual event in which Creative Inquiry (CI) teams can present their research and project accomplishments through posters and interactive displays. It is a celebration of student and mentor collaborations and accomplishments! Teams take this opportunity to develop and hone their communication skills

Program

11th Annual Research and Engagement Day Program with descriptions and schedule of events