Upper extremity rehabilitation using interactive virtual environments

Stroke affects more than 700,000 people annually in the U.S. It is the leading cause of major disability. Recovery of upper extremity function remains particularly resistant to intervention, with 80% to 95% of persons demonstrating residual upper extremity impairments lasting beyond six months after the stroke. The NJIT Robot Assistive Virtual Rehabilitation (NJIT-RAVR) system has been developed to study optimal strategies for rehabilitation of arm and hand function. Several commercial available devices, such as HapticMaster™, Cyberglove™, trakSTAR™ and Cybergrasp™, have been integrated and 11 simulations were developed to allow users to interact with virtual environments. Visual interfaces used in these simulations were programmed either in Virtools or in C++ using the Open GL library. Stereoscopic glasses were used to enhance depth perception and to present movement targets to the subjects in a 3-dimensional stereo working space. Adaptive online and offline algorithms were developed that provided appropriate task difficulty to optimize the outcomes. A pilot study was done on four stroke patients and two children with cerebral palsy to demonstrate the usability of this robot-assisted VR system. The RAVR system performed well without unexpected glitches during two weeks of training. No subjects experienced side effects such as dizziness, nausea or disorientation while interacting with the virtual environment. Each subject was able to finish the training, either with or without robotic adaptive assistance. To investigate optimal therapeutic approaches, forty stroke subjects were randomly assigned to two groups: Hand and Arm training Together (HAT) and Hand and Arm training Separately (HAS). Each group was trained in similar virtual reality training environments for three hours a day, four days a week for two weeks. In addition, twelve stroke subjects participated as a control group. They received conventional rehabilitation training of similar intensity and duration as the HAS and HAT groups. Clinical outcome measurements included the Jebsen Test of Hand Function, the Wolf Motor Function Test, and the ReachGrasp test. Secondary outcome measurements were calculated from kinematic and kinetic data collected during training in real time at 100 Hz. Both HAS and HAT groups showed significant improvement in clinical and kinematic outcome measurements. Clinical improvement compared favorably to the randomized clinical trials reported in the literature. However, there was no significant improvement difference between the two groups. Subjects from the control group improved in clinical measurements and in the ReachGrasp test. Compared to the control group, the ReachGrasp test showed a larger increase in movement speed during reaching and in the efficiency of lifting an object from the table in the combined HAS and HAT group. The NJIT-RAVR system was further modified to address the needs of children with hemiplegia due to Cerebral Palsy. Thirteen children with cerebral palsy participated in the total of nine sessions of one hour training that lasted for three weeks. Nine of the children were trained using the RAVR system alone, and another four had training with the combined Constraint-Induced Movement therapy and RAVR therapy. As a group, the children demonstrated improved performance across measurements of the Arm Range of Motion (AROM), motor function, kinematics and motor control. While subjects\u27 responses to the games varied, they performed each simulation while maintaining attention sufficient to improve in both robotic task performance and in measures of motor function

Tilt simulation : virtual reality based upper extremity stroke rehabilitation

The primary objective of this research is to design a recreational rehabilitation videogame that interactively encourages purposeful upper extremity gross motor movements. The simulation is also capable of continuous game modification to fit changing therapy goals, to match the needs of the players, and to provide continued motivation while capturing the interactive repetition. This thesis explains the design and features of this latest simulation - Tilt. Tilt uses physics to develop an engaging training experience and provides a realistic approach to virtual reality simulation including friction, elasticity and collisions between objects. It is designed to train upper extremity function as a unit involving multiple modalities simultaneously, either unilaterally or bilaterally. It is the latest addition to the NJIT Robot Assisted Virtual Rehabilitation (RAVR) system. It Employs the Cyber Glove and Flock of Birds systems to interface with the real world. This allows training motor function of patients that come to use in day to day life like making use of hands, fingers and shoulders to pick small objects on table, moving them and placing them elsewhere

A Person-Centric Design Framework for At-Home Motor Learning in Serious Games

abstract: In motor learning, real-time multi-modal feedback is a critical element in guided training. Serious games have been introduced as a platform for at-home motor training due to their highly interactive and multi-modal nature. This dissertation explores the design of a multimodal environment for at-home training in which an autonomous system observes and guides the user in the place of a live trainer, providing real-time assessment, feedback and difficulty adaptation as the subject masters a motor skill. After an in-depth review of the latest solutions in this field, this dissertation proposes a person-centric approach to the design of this environment, in contrast to the standard techniques implemented in related work, to address many of the limitations of these approaches. The unique advantages and restrictions of this approach are presented in the form of a case study in which a system entitled the "Autonomous Training Assistant" consisting of both hardware and software for guided at-home motor learning is designed and adapted for a specific individual and trainer. In this work, the design of an autonomous motor learning environment is approached from three areas: motor assessment, multimodal feedback, and serious game design. For motor assessment, a 3-dimensional assessment framework is proposed which comprises of 2 spatial (posture, progression) and 1 temporal (pacing) domains of real-time motor assessment. For multimodal feedback, a rod-shaped device called the "Intelligent Stick" is combined with an audio-visual interface to provide feedback to the subject in three domains (audio, visual, haptic). Feedback domains are mapped to modalities and feedback is provided whenever the user's performance deviates from the ideal performance level by an adaptive threshold. Approaches for multi-modal integration and feedback fading are discussed. Finally, a novel approach for stealth adaptation in serious game design is presented. This approach allows serious games to incorporate motor tasks in a more natural way, facilitating self-assessment by the subject. An evaluation of three different stealth adaptation approaches are presented and evaluated using the flow-state ratio metric. The dissertation concludes with directions for future work in the integration of stealth adaptation techniques across the field of exergames.Dissertation/ThesisDoctoral Dissertation Computer Science 201

Technology-supported training of arm-hand skills in stroke

Impaired arm-hand performance is a serious consequence of stroke that is associated with reduced self-efficacy and poor quality of life. Task-oriented arm training is a therapy approach that is known to improve skilled arm-hand performance, even in chronic stages after stroke. At the start of this project, little knowledge had been consolidated regarding taskoriented arm training characteristics, especially in the field of technology-supported rehabilitation. The feasibility and effects of technology-supported client-centred task-oriented training on skilled arm-hand performance had not been investigated but to a very limited degree. Reviewing literature on rehabilitation and motor learning in stroke led to the identification of therapy oriented criteria for rehabilitation technology aiming to influence skilled arm-hand performance (chapter 2). Most rehabilitation systems reported in literature to date are robotic systems that are aimed at providing an engaging exercise environment and feedback on motor performance. Both, feedback and engaging exercises are important for motivating patients to perform a high number of exercise repetitions and prolonged training, which are important factors for motor learning. The review also found that current rehabilitation technology is focussed mainly on providing treatment at a function level, thereby improving joint range of motion, muscle strength and parameters such as movement speed and smoothness of movement during analytical movements. However, related research has found no effects of robot-supported training at the activity level. The review concluded that a challenge exists for upper extremity rehabilitation technology in stroke patients to also provide more patienttailored task-oriented arm-hand training in natural environments to support the learning of skilled arm-hand performance. Besides mapping the strengths of different technological solutions, the use of outcome measures and training protocols needs to become more standardized across similar interventions, in order to help determine which training solutions are most suitable for specific patient categories. Chapter 4 contributes towards such a standardization of outcome measurement. A concept is introduced which may guide the clinician/researcher to choose outcome measures for evaluating specific and generalized training effects. As an initial operationalization of this concept, 28 test batteries that have been used in 16 task-oriented training interventions were rated as to whether measurement components were measured by the test. Future research is suggested that elaborates the concept with information on the relative weighing of components in each test, with more test batteries (which may lead to additional components) and by adding more test properties into the concept (e.g. psychometric properties of the tests, possible floor- or ceiling effects). Task-oriented training is one of the training approaches that has been shown to be beneficial for skilled arm-hand performance after stroke. Important mechanisms for motor learning that are identified are patient motivation for such training, and the learning of efficient goaloriented movement strategies and task-specific problem solving. In this thesis we operationalize task-oriented training in terms of 15 components (chapter 3). A systematic review that included 16 randomized controlled trials using task-oriented training in stroke patients, evaluated the effects of these training components on skilled arm-hand performance. The number of training components used in an intervention aimed at improving arm-hand performance after stroke was not associated with the post-treatment effect size. Distributed practice and feedback were associated with the largest post-intervention effect sizes. Random practice and use of clear functional training goals were associated with the largest follow-up effect sizes. It may be that training components that optimize the storage of learned motor performance in the long-term memory are associated with larger treatment effects. Unfortunately, feedback, random practice and distributed practice were reported in very few of the included randomized controlled trials (in only 6,3 and 1 out of the 17 studies respectively). Client-centred training, i.e. training on exercises that support goals that are selected by the patients themselves, improves patient motivation for training. Motivation in turn has proven to positively influence motor learning in stroke patients, as attention during training is heightened and storage of information in the long-term memory improves. Chapter 5 reports on an interview of 40 stroke patients, investigating into training preferences. A list of 46 skills, ranked according to descending training preference scores, was provided that can be used for implementation of exercises in rehabilitation technology, in order for technologysupported training to be client-centred. Chapter 6 introduces T-TOAT, a technology supported task-oriented arm training method that was developed together with colleagues at Adelante (Hoensbroek, NL). T-TOAT enables the implementation of exercises that support task-oriented training in rehabilitation technology. The training method is applicable for different technological systems, e.g. robot and sensor systems, or in combination with functional electrical stimulation, etc. To enable the use of TTOAT for training with the Haptic Master Robot (MOOG-FCS, NL), special software named Haptic TOAT was developed in Adelante together with colleagues at the Centre of Technology in Care of Zuyd University (chapter 6). The software enables the recording of the patient’s movement trajectories, given task constraints and patient possibilities, using the Haptic Master as a recording device. A purpose-made gimbal was attached to the endeffector, leaving the hand free for the use and manipulating objects. The recorded movement can be replayed in a passive mode or in an active mode (active, active-assisted or activeresisted). Haptic feedback is provided when the patient deviates from the recorded movement trajectory, as the patient receives the sensation of bouncing into a wall, as well as feeling a spring that pulls him/her back to the recorded path. The diameter of the tunnel around the recorded trajectory (distance to the wall), and the spring force can be adjusted for each patient. An ongoing clinical trial in which chronic stroke patients train with Haptic-TOAT examines whether Haptic Master provides additional value compared to supporting the same exercises by video-instruction only. Together with Philips Research Europe (Eindhoven,Aachen), the T-TOAT method has been implemented in a sensor based prototype, called Philips Stroke Rehabilitation Exerciser. This system included movement tracking sensors and an exercise board interacting with real life objects. A very strong feature of the system is that feedback is provided to patients (real-time and after exercise performance), based on a comparison of the patient’s exercise performance to individual targets set by the therapist. Chapter 7 reports on a clinical trial investigating arm-hand treatment outcome and patient motivation for technology-supported task-oriented training in chronic stroke patients. It was found that 8 weeks of T-TOAT training improved arm-hand performance in chronic stroke patients significantly on Fugl-Meyer, Action Research Arm Test, and Motor Activity Log. An improvement was found in health-related quality of life. Training effects lasted at least 6 months post-training. Participants reported feeling intrinsically motivated and competent to use the system. The results of this study showed that T-TOAT is feasible. Despite the small number of stroke patients tested (n=9), significant and clinically relevant improvements in skilled arm-hand performance were found. In conclusion, this thesis has made several contributions. It motivated the need for clientcentred task-oriented training, which it has operationalized in terms of 15 components. Four of these 15 components were identified as most beneficial for the patient. A prioritized inventory of arm-hand training preferences of stroke patients was compiled by means of an interview study of 40 subacute and chronic stroke patients. T-TOAT, a method for technology-supported, client-centred, task-oriented training, was conceived and implemented in two target technologies (Haptic Master and Philips Stroke Rehabilitation Exerciser). Its feasibility was demonstrated in a clinical trial showing substantial and durable benefits for the stroke patients. Finally, the thesis contributes towards the standardization of outcome measures which is necessary for charting progress and guiding future developments of technology-supported stroke rehabilitation. Methodological considerations were discussed and several suggestions for future research were presented. The variety of treatment approaches and the various ways of support and challenge that are offered by existing rehabilitation technologies hold a large potential for offering a variety of extra training opportunities to stroke patients that may improve their arm-hand performance. Such solutions will be of increasing importance, to alleviate therapists and reduce economic pressure on the health care system, as the stroke incidence is increasing rapidly over the coming decades

Review of the Augmented Reality Systems for Shoulder Rehabilitation

Literature shows an increasing interest for the development of augmented reality (AR) applications in several fields, including rehabilitation. Current studies show the need for new rehabilitation tools for upper extremity, since traditional interventions are less effective than in other body regions. This review aims at: Studying to what extent AR applications are used in shoulder rehabilitation, examining wearable/non-wearable technologies employed, and investigating the evidence supporting AR effectiveness. Nine AR systems were identified and analyzed in terms of: Tracking methods, visualization technologies, integrated feedback, rehabilitation setting, and clinical evaluation. Our findings show that all these systems utilize vision-based registration, mainly with wearable marker-based tracking, and spatial displays. No system uses head-mounted displays, and only one system (11%) integrates a wearable interface (for tactile feedback). Three systems (33%) provide only visual feedback; 66% present visual-audio feedback, and only 33% of these provide visual-audio feedback, 22% visual-audio with biofeedback, and 11% visual-audio with haptic feedback. Moreover, several systems (44%) are designed primarily for home settings. Three systems (33%) have been successfully evaluated in clinical trials with more than 10 patients, showing advantages over traditional rehabilitation methods. Further clinical studies are needed to generalize the obtained findings, supporting the effectiveness of the AR applications

Assessment of a hand exoskeleton on proximal and distal training in virtual environments for robot mediated upper extremity rehabilitation

Stroke is the leading cause of disability in the United States with approximately 800,000 cases per year. This cerebral vascular accident results in neurological impairments that reduce limb function and limit the daily independence of the individual. Evidence suggests that therapeutic interventions with repetitive motor training can aid in functional recovery of the paretic limb. Robotic rehabilitation may present an exercise intervention that can improve training and induce motor plasticity in individuals with stroke. An active (motorized) hand exoskeleton that provides support for wrist flexion/extension, abduction/adduction, pronation/supination, and finger pinch is integrated with a pre-existing 3-Degree of Freedom (DOF) haptic robot (Haptic Master, FCS Moog) to determine the efficacy of increased DOF during proximal and distal training in Upper Extremity (UE) rehabilitation. Subjects are randomly assigned into four groups to evaluate the significance of increased DOF during virtual training: Haptic Master control group (HM), Haptic Master with Gripper (HMG), Haptic Master with Wrist (HMW), and Haptic Master with Gripper and Wrist (HMWG). Each subject group performs a Pick and Place Task in a virtual environment where the distal hand exoskeleton is mapped to the virtual representation of the hand. Subjects are instructed to transport as many virtual cubes as possible to a specified target in the allotted time period of 120s. Three cube sizes are assessed to determine efficacy of the assistive end-effector. An additional virtual task, Mailbox Task, is performed to determine the effect of training and the ability to transfer skills between virtual settings in an unfamiliar environment. The effects of viewing mediums are also investigated to determine the effect of immersion on performance using an Oculus Rift as an HMD compared to conventional projection displays. It is hypothesized that individuals with both proximal and complete distal hand control (HMWG) will see increased benefit during the Pick and Place Task than individuals without the complete distal attachment, as assisted daily living tasks are often accomplished with coordinated arm and hand movement. The purpose of this study is to investigate the additive effect of increased degrees of freedom at the hand through task-specific training of the upper arm in a virtual environment, validate the ability to transfer skills obtained in a virtual environment to an untrained task, and determine the effects of viewing mediums on performance. A feasibility study is conducted in individuals with stroke to determine if the modular gripper can assist pinch movements. These investigations represent a comprehensive investigation to assess the potential benefits of assistive devices in a virtual reality setting to retrain lost function and increase efficacy in motor control in populations with motor impairments

Handwriting Correction System using Wearable Sleeve with Optimal Tactor Configuration

Handwriting remains an elusive skill with practice worksheets being the common method of learning. Since these worksheets provide only visual feedback and no quantitative feedback, it can often be a challenge to improve. For children with learning disabilities, learning handwriting skills is one of the most difficult tasks. We propose a handwriting training system that uses off-the-shelf webcam, a pen tracking software and a haptic sleeve which provides active feedback to the user based on their deviation from the original pattern. The sleeve has 4 individual motors that vibrate at different intensities based on the direction (right, left, up or down) and severity of the deviation (\u3c 1cm, 1cm – 3cm, \u3e 3cm). Different motor placements around the forearm are evaluated for vibro-tactile feedback accuracy and time response, and a novel spaced-ring configuration is proposed. This paper provides details on the system architecture and sleeve characterization, and the results show promise in utilizing the system for self-correction and visual-motor skills development. The results from sleeve characterization suggest the applicability of the spaced-ring configuration (perceived feedback accuracy \u3e 98%, time response \u3c 1s) in other vibrotactile hand guidance systems, in addition to handwriting correction. Recommendations on tactor placements around the forearm are provided

Haptic wearables as sensory replacement, sensory augmentation and trainer - a review

Sensory impairments decrease quality of life and can slow or hinder rehabilitation. Small, computationally powerful electronics have enabled the recent development of wearable systems aimed to improve function for individuals with sensory impairments. The purpose of this review is to synthesize current haptic wearable research for clinical applications involving sensory impairments. We define haptic wearables as untethered, ungrounded body worn devices that interact with skin directly or through clothing and can be used in natural environments outside a laboratory. Results of this review are categorized by degree of sensory impairment. Total impairment, such as in an amputee, blind, or deaf individual, involves haptics acting as sensory replacement; partial impairment, as is common in rehabilitation, involves haptics as sensory augmentation; and no impairment involves haptics as trainer. This review found that wearable haptic devices improved function for a variety of clinical applications including: rehabilitation, prosthetics, vestibular loss, osteoarthritis, vision loss and hearing loss. Future haptic wearables development should focus on clinical needs, intuitive and multimodal haptic displays, low energy demands, and biomechanical compliance for long-term usage