Integrating eye tracking in virtual reality for stroke rehabilitation

This thesis reports on research done for the integration of eye tracking technology into virtual reality environments, with the goal of using it in rehabilitation of patients who suffered from stroke. For the last few years, eye tracking has been a focus on medical research, used as an assistive tool to help people with disabilities interact with new technologies and as an assessment tool to track the eye gaze during computer interactions. However, tracking more complex gaze behaviors and relating them to motor deficits in people with disabilities is an area that has not been fully explored, therefore it became the focal point of this research. During the research, two exploratory studies were performed in which eye tracking technology was integrated in the context of a newly created virtual reality task to assess the impact of stroke. Using an eye tracking device and a custom virtual task, the system developed is able to monitor the eye gaze pattern changes over time in patients with stroke, as well as allowing their eye gaze to function as an input for the task. Based on neuroscientific hypotheses of upper limb motor control, the studies aimed at verifying the differences in gaze patterns during the observation and execution of the virtual goal-oriented task in stroke patients (N=10), and also to assess normal gaze behavior in healthy participants (N=20). Results were found consistent and supported the hypotheses formulated, showing that eye gaze could be used as a valid assessment tool on these patients. However, the findings of this first exploratory approach are limited in order to fully understand the effect of stroke on eye gaze behavior. Therefore, a novel model-driven paradigm is proposed to further understand the relation between the neuronal mechanisms underlying goal-oriented actions and eye gaze behavior

Eye gaze correlates of motor impairment in VR observation of motor actions

Introduction: This article is part of the Focus Theme of Methods of Information in Medicine on “Methodologies, Models and A lgorithms for Patients Rehabilitation”. Objective: Identify eye gaze correlates of motor impairment in a virtual reality motor observation task in a study with healthy participants and stroke patients. Methods: Participants consisted of a group of healthy subjects (N = 20) and a group of stroke survivors (N = 10). Both groups were required to observe a simple reach-and-grab and place-and-release task in a virtual environment. Additionally, healthy subjects were required to observe the task in a normal condition and a constrained movement condition. Eye movements were recorded during the observation task for later analysis.info:eu-repo/semantics/publishedVersio

Development and Evaluation of Tongue Operated Robotic Rehabilitation Paradigm for Stroke Survivors with Upper Limb Paralysis

Stroke is a devastating condition that may cause upper limb paralysis. Robotic rehabilitation with self-initiated and assisted movements is a promising technology that could help restore upper limb function. The objective of this research is to develop and evaluate a tongue-operated exoskeleton that will harness the intention of stroke survivors with upper limb paralysis via tongue motion to control robotic exoskeleton during rehabilitation to achieve functional restoration and improve quality of life. Specifically, a tongue operated assistive technology called the Tongue Drive System is used to harness the tongue gesture to generate commands. And, the generated command is used to control rehabilitation robot such as wrist-based exoskeleton Hand Mentor ProTM (HM) and upper limb-based exoskeleton KINARMTM. Through a pilot experiment with 3 healthy participants, we have demonstrated the functionality of an enhanced TDS-HM with pressure-sensing capability. The system can add a programmable load force to increase the exercise intensity in isotonic mode. Through experiments with healthy and stroke subjects, we have demonstrated that the TDS-KINARM system could accurately translate tongue commands to exoskeleton arm movements, quantify function of the upper limb and perform rehabilitation training. Specifically, all healthy subjects and stroke survivors successfully performed target reaching and tracking tasks in all control modes. One of the stroke patients showed clinically significant improvement. We also analyzed the arm reaching kinematics of healthy subjects in 4 modes (active, active viscous, discrete tongue, and proportional tongue) of TDS-KINARM operation. The results indicated that the proportional tongue mode was a better candidate than the discrete tongue mode for the tongue assisted rehabilitation. This study also provided initial insights into possible kinematic similarities between tongue-operated and voluntary arm movements. Furthermore, the results showed that the viscous resistance to arm motion did not affect kinematics of arm reaching movements significantly. Finally, through a 6 healthy subject experiment, we observed a tendency of a facilitatory effect of adding tongue movement to limb movement on event-related desynchronization in EEG, implying enhanced brain excitability. This effect may contribute to enhanced rehabilitation outcome in stroke survivors using TDS with motor rehabilitation.Ph.D

Safe Haptics-enabled Patient-Robot Interaction for Robotic and Telerobotic Rehabilitation of Neuromuscular Disorders: Control Design and Analysis

Motivation: Current statistics show that the population of seniors and the incidence rate of age-related neuromuscular disorders are rapidly increasing worldwide. Improving medical care is likely to increase the survival rate but will result in even more patients in need of Assistive, Rehabilitation and Assessment (ARA) services for extended periods which will place a significant burden on the world\u27s healthcare systems. In many cases, the only alternative is limited and often delayed outpatient therapy. The situation will be worse for patients in remote areas. One potential solution is to develop technologies that provide efficient and safe means of in-hospital and in-home kinesthetic rehabilitation. In this regard, Haptics-enabled Interactive Robotic Neurorehabilitation (HIRN) systems have been developed. Existing Challenges: Although there are specific advantages with the use of HIRN technologies, there still exist several technical and control challenges, e.g., (a) absence of direct interactive physical interaction between therapists and patients; (b) questionable adaptability and flexibility considering the sensorimotor needs of patients; (c) limited accessibility in remote areas; and (d) guaranteeing patient-robot interaction safety while maximizing system transparency, especially when high control effort is needed for severely disabled patients, when the robot is to be used in a patient\u27s home or when the patient experiences involuntary movements. These challenges have provided the motivation for this research. Research Statement: In this project, a novel haptics-enabled telerobotic rehabilitation framework is designed, analyzed and implemented that can be used as a new paradigm for delivering motor therapy which gives therapists direct kinesthetic supervision over the robotic rehabilitation procedure. The system also allows for kinesthetic remote and ultimately in-home rehabilitation. To guarantee interaction safety while maximizing the performance of the system, a new framework for designing stabilizing controllers is developed initially based on small-gain theory and then completed using strong passivity theory. The proposed control framework takes into account knowledge about the variable biomechanical capabilities of the patient\u27s limb(s) in absorbing interaction forces and mechanical energy. The technique is generalized for use for classical rehabilitation robotic systems to realize patient-robot interaction safety while enhancing performance. In the next step, the proposed telerobotic system is studied as a modality of training for classical HIRN systems. The goal is to first model and then regenerate the prescribed kinesthetic supervision of an expert therapist. To broaden the population of patients who can use the technology and HIRN systems, a new control strategy is designed for patients experiencing involuntary movements. As the last step, the outcomes of the proposed theoretical and technological developments are translated to designing assistive mechatronic tools for patients with force and motion control deficits. This study shows that proper augmentation of haptic inputs can not only enhance the transparency and safety of robotic and telerobotic rehabilitation systems, but it can also assist patients with force and motion control deficiencies

Enhancing brain/neural-machine interfaces for upper limb motor restoration in chronic stroke and cervical spinal cord injury

Operation of assistive exoskeletons based on voluntary control of sensorimotor rhythms (SMR, 8-12 Hz) enables intuitive control of finger or arm movements in severe paralysis after chronic stroke or cervical spinal cord injury (SCI). To improve reliability of such systems outside the laboratory, in particular when brain activity is recorded non-invasively with scalp electroencephalography (EEG), a hybrid EEG/electrooculography (EOG) brain/neural-machine interface (B/NMI) was recently introduced. Besides providing assistance, recent studies indicate that repeated use of such systems can trigger neural recovery. However, important prerequisites have to achieved before broader use in clinical settings or everyday life environments is feasible. Current B/NMI systems predominantly restore hand function, but do not allow simultaneous control of more proximal joints for whole-arm motor coordination as required for most stroke survivors suffering from paralysis in the entire upper limb. Besides paralysis, cognitive impairments including post-stroke fatigue due to the brain lesion reduce the capacity to maintain effortful B/NMI control over a longer period of time. This impedes the applicability in daily life assistance and might even limits the efficacy of neurorehabilitation training. In contrast to stroke survivors, tetraplegics due to cervical SCI lack motor function in both hands. Given that most activities of daily living (ADL) involve bimanual manipulation, e.g., to open the lid of a bottle, bilateral exoskeleton control is required but was not shown yet in tetraplegics. To further enhance B/NMI systems, we first investigated whether B/NMI whole-arm exoskeleton control in hemiplegia after chronic stroke is feasible and safe. In contrast to simple grasping, control of more complex tasks involving the entire upper limb was not feasible with established B/NMIs because high- dimensionality of such multiple joint systems exceeds the bandwidth of these interfaces. Thus, we blended B/NMI control with vision-guidance to receive a semiautonomous whole-arm exoskeleton control. Such setup allowed to divide ADL tasks into a sequence of EEG/EOG-triggered sub-tasks reducing complexity for the user. While, for instance, a drinking task was resolved into EOG-induced reaching, lifting and placing back the cup, grasping and releasing movements were based on intuitive SMR control. Feasibility of such shared vision-guided B/NMI control was assumed when executions were initialized within 3 s (fluent control) and a minimum of 75 % of subtasks were executed within that time (reliable control). We showed feasibility in healthy subjects as well as stroke survivors without report of any side effects documenting safe use. Similarly, feasibility and safety of bilateral B/NMI control after cervical SCI was evaluated. To enable bilateral B/NMI control, established EEG-based grasping and EOG-based releasing or stop commands were complemented with a novel EOG command allowing to switch laterality by performing prolonged horizontal eye movements (>1 s) to the left or to the right. Study results with healthy subjects and tetraplegics document fluent initialization of grasping motions below 3 s as well as safe use as unintended grasping could be stopped before a full motion was conducted. Superiority of novel bilateral control was documented by a higher accuracy of up to 22 % in tetraplegics compared to a bilateral control without prolonged EOG command. Lastly, as reliable B/NMI control is cognitively demanding, e.g., by imagining or attempting the desired movements, we investigated whether heart rate variability (HRV) can be used as biomarker to predict declining control performance, which is often reported in stroke survivors due to their cognitive impairments. Referring to the close brain-heart connection, we showed in healthy subjects that a decline in HRV is specific as well as predictive to a decline in B/NMI control performance within a single training session. The predictive link was revealed by a Granger-causality analysis. In conclusion, we could demonstrate important enhancements in B/NMI control paradigms including complex whole-arm exoskeleton control as well as individual performance monitoring within a training session based on HRV. Both achievements contribute to broaden the use as a standard therapy in stroke neurorehabilitation. Especially the predictive characteristic of HRV paves the way for adaptive B/NMI control paradigms to account for individual differences among impaired stroke survivors. Moreover, we also showed feasibility and safety of a novel implementation for bilateral B/NMI control, which is necessary for reliable operation of two hand-exoskeletons for bimanual ADLs after SCI

Proceedings of the 10th international conference on disability, virtual reality and associated technologies (ICDVRAT 2014)

The proceedings of the conferenc

In-home and remote use of robotic body surrogates by people with profound motor deficits

By controlling robots comparable to the human body, people with profound motor deficits could potentially perform a variety of physical tasks for themselves, improving their quality of life. The extent to which this is achievable has been unclear due to the lack of suitable interfaces by which to control robotic body surrogates and a dearth of studies involving substantial numbers of people with profound motor deficits. We developed a novel, web-based augmented reality interface that enables people with profound motor deficits to remotely control a PR2 mobile manipulator from Willow Garage, which is a human-scale, wheeled robot with two arms. We then conducted two studies to investigate the use of robotic body surrogates. In the first study, 15 novice users with profound motor deficits from across the United States controlled a PR2 in Atlanta, GA to perform a modified Action Research Arm Test (ARAT) and a simulated self-care task. Participants achieved clinically meaningful improvements on the ARAT and 12 of 15 participants (80%) successfully completed the simulated self-care task. Participants agreed that the robotic system was easy to use, was useful, and would provide a meaningful improvement in their lives. In the second study, one expert user with profound motor deficits had free use of a PR2 in his home for seven days. He performed a variety of self-care and household tasks, and also used the robot in novel ways. Taking both studies together, our results suggest that people with profound motor deficits can improve their quality of life using robotic body surrogates, and that they can gain benefit with only low-level robot autonomy and without invasive interfaces. However, methods to reduce the rate of errors and increase operational speed merit further investigation.Comment: 43 Pages, 13 Figure

Enhancement of Robot-Assisted Rehabilitation Outcomes of Post-Stroke Patients Using Movement-Related Cortical Potential

Post-stroke rehabilitation is essential for stroke survivors to help them regain independence and to improve their quality of life. Among various rehabilitation strategies, robot-assisted rehabilitation is an efficient method that is utilized more and more in clinical practice for motor recovery of post-stroke patients. However, excessive assistance from robotic devices during rehabilitation sessions can make patients perform motor training passively with minimal outcome. Towards the development of an efficient rehabilitation strategy, it is necessary to ensure the active participation of subjects during training sessions. This thesis uses the Electroencephalography (EEG) signal to extract the Movement-Related Cortical Potential (MRCP) pattern to be used as an indicator of the active engagement of stroke patients during rehabilitation training sessions. The MRCP pattern is also utilized in designing an adaptive rehabilitation training strategy that maximizes patients’ engagement. This project focuses on the hand motor recovery of post-stroke patients using the AMADEO rehabilitation device (Tyromotion GmbH, Austria). AMADEO is specifically developed for patients with fingers and hand motor deficits. The variations in brain activity are analyzed by extracting the MRCP pattern from the acquired EEG data during training sessions. Whereas, physical improvement in hand motor abilities is determined by two methods. One is clinical tests namely Fugl-Meyer Assessment (FMA) and Motor Assessment Scale (MAS) which include FMA-wrist, FMA-hand, MAS-hand movements, and MAS-advanced hand movements’ tests. The other method is the measurement of hand-kinematic parameters using the AMADEO assessment tool which contains hand strength measurements during flexion (force-flexion), and extension (force-extension), and Hand Range of Movement (HROM)

Desarrollo y análisis de estrategias avanzadas de interacción en sistemas robóticos complejos de rehabilitación y asistencia

Los últimos informes indican que la incidencia de los accidentes cerebro vasculares van en aumento. Los supervivientes de un accidente cerebro vascular generalmente experimentan hemiparesia, lo que provoca un deterioro de las extremidades que implica un notable deterioro de la calidad de vida. La escasez de recursos junto con la necesidad de rehabilitación y asistencia que presentan estas personas, hacen que el cuidado y los ejercicios de rehabilitación que proporcionan las plataformas robóticas de rehabilitación cobren aún más importancia en la actualidad y en los próximos años. La presente Tesis Doctoral se centra en el desarrollo de una arquitectura multimodal capaz de implementar sistemas robóticos complejos de rehabilitación y asistencia. Con esta arquitectura, se plantea implementar y evaluar los siguientes sistemas. Primero, llevar a cabo la implementación de un sistema complejo de robótica asistencial. Después, desarrollar un sistema para la realización de terapias competitivas. Seguidamente, implementar un sistema para realizar terapias cooperativas. Y finalmente, desarrollar un sistema capaz de implementar terapias de tele-rehabilitación mediante una estrategia de teleoperación maestro-esclavo. Los principales resultados de esta tesis se han publicado en dos artículos en revistas incluidas en el Journal Citation Reports (JCR). La publicación “Feasibility and safety of shared EEG/EOG and vision-guided autonomous whole-arm exoskeleton control to perform activities of daily living” se ha evaluado la implementación de un sistema complejo de robótica asistencial basado en la arquitectura multimodal desarrollada en esta tesis, en el que se ha integrado con éxito multitud de sensores junto con diferentes interfaces de control y dispositivos robóticos para la creación de un sistema autónomo capaz de ayudar a un usuario a realizar actividades de la vida diaria. Por otro lado, en la publicación “Differences in Physiological Reactions Due to a Competitive Rehabilitation Game Modality”, se han analizado los cambios que se producen en el estado afectivo del paciente en una terapia de neurorrehabilitación asistida por robots debidos a una modalidad de juego multijugador de tipo competitivo. Finalmente, cabe destacar que uno de los resultados de la presente tesis ha dado lugar a la patente ES1234596U: Dispositivo robótico interconectable para rehabilitación de extremidades