The Implementation and Validation of a Virtual Environment for Training Powered Wheelchair Manoeuvres

This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.Navigating a powered wheelchair and avoiding collisions is often a daunting task for new wheelchair users. It takes time and practice to gain the coordination needed to become a competent driver and this can be even more of a challenge for someone with a disability. We present a cost-effective virtual reality (VR) application that takes advantage of consumer level VR hardware. The system can be easily deployed in an assessment centre or for home use, and does not depend on a specialized high-end virtual environment such as a Powerwall or CAVE. This paper reviews previous work that has used virtual environments technology for training tasks, particularly wheelchair simulation. We then describe the implementation of our own system and the first validation study carried out using thirty three able bodied volunteers. The study results indicate that at a significance level of 5% then there is an improvement in driving skills from the use of our VR system. We thus have the potential to develop the competency of a wheelchair user whilst avoiding the risks inherent to training in the real world. However, the occurrence of cybersickness is a particular problem in this application that will need to be addressed

Virtual and Mixed Reality Support for Activities of Daily Living

Rehabilitation and training are extremely important process that help people who have suffered some form of trauma to regain their ability to live independently and successfully complete activities of daily living. VR and MR have been used in rehabilitation and training, with examples in a range of areas such as physical and cognitive rehabilitation, and medical training. However, previous research has mainly used non-immersive VR such as using video games on a computer monitor or television. Immersive VR Head-Mounted Displays were first developed in 1965 but the devices were usually large, bulky and expensive. In 2016, the release of low-cost VR HMDs allowed for wider adoption of VR technology. This thesis investigates the impact of these devices in supporting activities of daily living through three novel applications: training driving skills for a powered wheelchair in both VR and MR; and using VR to help with the cognitive rehabilitation of stroke patients. Results from the acceptability study for VR in cognitive rehabilitation showed that patients would be likely to accept VR as a method of rehabilitation. However, factors such as visual issues need to be taken into consideration. The validation study for the Wheelchair-VR project showed promising results in terms of user improvement after the VR training session but the majority of the users experienced symptoms of cybersickness. Wheelchair-MR didn’t show statistically significant results in terms of improvements but did show a mean average improvement compared to the control group. The effects of cybersickness were also greatly reduced compared to VR. We conclude that VR and MR can be used in conjunction with modern games engines to develop virtual environments that can be adapted to accelerate the rehabilitation and training of patients coping with different aspects of daily life

Training Powered Wheelchair Manoeuvres in Mixed Reality

© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other worksWe describe a mixed reality environment that has been designed as an aid for training driving skills for a powered wheelchair. Our motivation is to provide an improvement on a previous virtual reality wheelchair driving simulator, with a particular aim to remove any cybersickness effects. The results of a validation test are presented that involved 35 able bodied volunteers divided into three groups: mixed reality trained, virtual reality trained, and a control group. No significant differences in improvement was found between the groups but there is a notable trend that both the mixed reality and virtual reality groups improved more than the control group. Whereas the virtual reality group experienced discomfort (as measured using a simulator sickness questionnaire), the mixed reality group experienced no side effects

Training Powered Wheelchair Manoeuvres in Mixed Reality

© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other worksWe describe a mixed reality environment that has been designed as an aid for training driving skills for a powered wheelchair. Our motivation is to provide an improvement on a previous virtual reality wheelchair driving simulator, with a particular aim to remove any cybersickness effects. The results of a validation test are presented that involved 35 able bodied volunteers divided into three groups: mixed reality trained, virtual reality trained, and a control group. No significant differences in improvement was found between the groups but there is a notable trend that both the mixed reality and virtual reality groups improved more than the control group. Whereas the virtual reality group experienced discomfort (as measured using a simulator sickness questionnaire), the mixed reality group experienced no side effects

Explainable shared control in assistive robotics

Shared control plays a pivotal role in designing assistive robots to complement human capabilities during everyday tasks. However, traditional shared control relies on users forming an accurate mental model of expected robot behaviour. Without this accurate mental image, users may encounter confusion or frustration whenever their actions do not elicit the intended system response, forming a misalignment between the respective internal models of the robot and human. The Explainable Shared Control paradigm introduced in this thesis attempts to resolve such model misalignment by jointly considering assistance and transparency. There are two perspectives of transparency to Explainable Shared Control: the human's and the robot's. Augmented reality is presented as an integral component that addresses the human viewpoint by visually unveiling the robot's internal mechanisms. Whilst the robot perspective requires an awareness of human "intent", and so a clustering framework composed of a deep generative model is developed for human intention inference. Both transparency constructs are implemented atop a real assistive robotic wheelchair and tested with human users. An augmented reality headset is incorporated into the robotic wheelchair and different interface options are evaluated across two user studies to explore their influence on mental model accuracy. Experimental results indicate that this setup facilitates transparent assistance by improving recovery times from adverse events associated with model misalignment. As for human intention inference, the clustering framework is applied to a dataset collected from users operating the robotic wheelchair. Findings from this experiment demonstrate that the learnt clusters are interpretable and meaningful representations of human intent. This thesis serves as a first step in the interdisciplinary area of Explainable Shared Control. The contributions to shared control, augmented reality and representation learning contained within this thesis are likely to help future research advance the proposed paradigm, and thus bolster the prevalence of assistive robots.Open Acces

Train vs. play: Evaluating the effects of gamified and non-gamified wheelchair skills training using virtual reality

This study compares the influence of a gamified and a non-gamified virtual reality (VR) environment on wheelchair skills training. In specific, the study explores the integration of gamification elements and their influence on wheelchair driving performance in VR-based training. Twenty-two non-disabled participants volunteered for the study, of whom eleven undertook the gamified VR training, and eleven engaged in the non-gamified VR training. To measure the efficacy of the VR-based wheelchair skills training, we captured the heart rate (HR), number of joystick movements, completion time, and number of collisions. In addition, an adapted version of the Wheelchair Skills Training Program Questionnaire (WSTP-Q), the Igroup Presence Questionnaire (IPQ), and the Simulator Sickness Questionnaire (SSQ) questionnaires were administered after the VR training. The results showed no differences in wheelchair driving performance, the level of involvement, or the ratings of presence between the two environments. In contrast, the perceived cybersickness was statistically higher for the group of participants who trained in the non-gamified VR environment. Remarkably, heightened cybersickness symptoms aligned with increased HR, suggesting physiological connections. As such, while direct gamification effects on the efficacy of VR-based wheelchair skills training were not statistically significant, its potential to amplify user engagement and reduce cybersickness is evident

Virtual Reality Interface Factors in a Power Wheelchair Simulator

Power wheelchairs (PWCs) can improve users’ quality of life by enabling them to participate in the activities of daily living, decreasing their dependence on human assistance. PWC users are faced with restricted environments, with limited space to manoeuvre, and are therefore vulnerable to collisions and injuries. To use a PWC effectively and safely, individuals must undertake training and assessment of their competency. There is significant potential for the use of virtual reality in the training and assessment of PWC users. To date, there is no standard tool available for PWC assessment and training. Rather, clinics use their own observation measurer and assessment is often largely based on guesswork. Several simulators have been developed to help the training of PWC users, yet the study of virtual assessment is an under-researched area. In fact, most simulators offer only very limited functionality and rely solely on client-centric information. For the development of a useful simulator, it is important to identify and evaluate interface factors affecting perception, behaviour, experience, and driving performance from both the user’s and clinician’s perspectives. In this thesis, issues with current PWC simulators were identified and investigated, with the intention of providing a suitable research platform for the advancement of bringing PWC simulator into clinical use. The aspects investigated include the interaction device, perception and behaviour, and virtual assessment. Three systems were developed to test each of these areas by incorporating theories and techniques from computer science and human-computer interaction. The first experiment answered the question, “which input devices are necessary and appropriate, and which virtual input device representations can and should be implemented for PWC simulation?” A proprietary PWC joystick was compared to a standard gaming joystick, and driving performance and experience were measured. Four experimental conditions (comprising two virtual input modalities and their two real-world counterparts) were studied. The findings suggest that performance is enhanced when the PWC joystick is represented and that the gaming joystick is adequate for PWC simulation. The second study investigated the question, “how do immersion factors influence behaviour, perception and sense of presence when navigating a PWC simulator?” The evaluated immersion factors include display type (head mounted display vs. monitor), field of view (changeable vs. static), and avatar presence (present vs. absent). User perception (explicit judgement of doorframe passability) and embedded behaviour (implicit measure of gap passability) were measured, based on the user’s decisions during the experiment. The results show that all three factors affect the user’s sense of presence. The display type affected both perceptual and behavioural measures, whereas field of view only affected behavioural measures. The final experiment explored the question, “how accurately can clinicians assess driving tasks in the virtual environment compared to the real world?” This study evaluated the effect of three observational techniques (viewpoints) on clinician assessment of PWC driving tasks. In addition, perceived ease of use, confidence level, and sense of presence were also examined. Observational techniques include walk, orbit, and standard viewpoints. The findings of this study suggest that clinicians could make accurate judgments and experience a high confidence level when they were able to walk or orbit the viewpoint. The results from all experiments provide general design guidelines for future virtual reality applications, in particular, PWC simulator design

A standardised and cost-effective VR approach for powered wheelchair training

Mastering wheelchair driving skills is essential for the safety of wheelchair users (WUs), yet the acquisition of these skills can be challenging, and training resources can be costly or not available. Technologies such as virtual reality (VR) have grown in popularity as they can provide a motivating training environment without the risks found in real-life training. However, these approaches often deploy navigation controllers which are different from the ones WUs utilise, and do not use a standardised approach in assessing the acquisition of skills. We propose a VR training system based on the wheelchair skills training program (WSTP) and utilizing a sensor device that can be retrofitted to any joystick and communicates wirelessly with a Head-Mounted Display. In this paper, we present a first-validation study with fourteen able-bodied participants, split between a VR test group and a non-VR control group. To determine the acquisition of skills, participants complete tasks in real-life before and after the VR training, where completion time and length of joystick movements are measured. We also assess our system using heart rate measurements, the WSTP questionnaire, the simulator sickness questionnaire and the igroup presence questionnaire. We found that the VR training facilitates the acquisition of skills for more challenging tasks; thus, our system has the potential of being used for training skills of powered wheelchair users, with the benefit of conducting the training in safely and in a low-cost setup