Exploring the bases for a mixed reality stroke rehabilitation system, Part II: Design of Interactive Feedback for upper limb rehabilitation

abstract: Background Few existing interactive rehabilitation systems can effectively communicate multiple aspects of movement performance simultaneously, in a manner that appropriately adapts across various training scenarios. In order to address the need for such systems within stroke rehabilitation training, a unified approach for designing interactive systems for upper limb rehabilitation of stroke survivors has been developed and applied for the implementation of an Adaptive Mixed Reality Rehabilitation (AMRR) System. Results The AMRR system provides computational evaluation and multimedia feedback for the upper limb rehabilitation of stroke survivors. A participant's movements are tracked by motion capture technology and evaluated by computational means. The resulting data are used to generate interactive media-based feedback that communicates to the participant detailed, intuitive evaluations of his performance. This article describes how the AMRR system's interactive feedback is designed to address specific movement challenges faced by stroke survivors. Multimedia examples are provided to illustrate each feedback component. Supportive data are provided for three participants of varying impairment levels to demonstrate the system's ability to train both targeted and integrated aspects of movement. Conclusions The AMRR system supports training of multiple movement aspects together or in isolation, within adaptable sequences, through cohesive feedback that is based on formalized compositional design principles. From preliminary analysis of the data, we infer that the system's ability to train multiple foci together or in isolation in adaptable sequences, utilizing appropriately designed feedback, can lead to functional improvement. The evaluation and feedback frameworks established within the AMRR system will be applied to the development of a novel home-based system to provide an engaging yet low-cost extension of training for longer periods of time.The electronic version of this article is the complete one and can be found online at: https://jneuroengrehab.biomedcentral.com/articles/10.1186/1743-0003-8-5

Data-driven learning for robot physical intelligence

The physical intelligence, which emphasizes physical capabilities such as dexterous manipulation and dynamic mobility, is essential for robots to physically coexist with humans. Much research on robot physical intelligence has achieved success on hyper robot motor capabilities, but mostly through heavily case-specific engineering. Meanwhile, in terms of robot acquiring skills in a ubiquitous manner, robot learning from human demonstration (LfD) has achieved great progress, but still has limitations handling dynamic skills and compound actions. In this dissertation, a composite learning scheme which goes beyond LfD and integrates robot learning from human definition, demonstration, and evaluation is proposed. This method tackles advanced motor skills that require dynamic time-critical maneuver, complex contact control, and handling partly soft partly rigid objects. Besides, the power of crowdsourcing is brought to tackle case-specific engineering problem in the robot physical intelligence. Crowdsourcing has demonstrated great potential in recent development of artificial intelligence. Constant learning from a large group of human mentors breaks the limit of learning from one or a few mentors in individual cases, and has achieved success in image recognition, translation, and many other cyber applications. A robot learning scheme that allows a robot to synthesize new physical skills using knowledge acquired from crowdsourced human mentors is proposed. The work is expected to provide a long-term and big-scale measure to produce advanced robot physical intelligence

Neurorehabilitation using the virtual reality based Rehabilitation Gaming System: methodology, design, psychometrics, usability and validation

<p>Abstract</p> <p>Background</p> <p>Stroke is a frequent cause of adult disability that can lead to enduring impairments. However, given the life-long plasticity of the brain one could assume that recovery could be facilitated by the harnessing of mechanisms underlying neuronal reorganization. Currently it is not clear how this reorganization can be mobilized. Novel technology based neurorehabilitation techniques hold promise to address this issue. Here we describe a Virtual Reality (VR) based system, the Rehabilitation Gaming System (RGS) that is based on a number of hypotheses on the neuronal mechanisms underlying recovery, the structure of training and the role of individualization. We investigate the psychometrics of the RGS in stroke patients and healthy controls.</p> <p>Methods</p> <p>We describe the key components of the RGS and the psychometrics of one rehabilitation scenario called Spheroids. We performed trials with 21 acute/subacute stroke patients and 20 healthy controls to study the effect of the training parameters on task performance. This allowed us to develop a Personalized Training Module (PTM) for online adjustment of task difficulty. In addition, we studied task transfer between physical and virtual environments. Finally, we assessed the usability and acceptance of the RGS as a rehabilitation tool.</p> <p>Results</p> <p>We show that the PTM implemented in RGS allows us to effectively adjust the difficulty and the parameters of the task to the user by capturing specific features of the movements of the arms. The results reported here also show a consistent transfer of movement kinematics between physical and virtual tasks. Moreover, our usability assessment shows that the RGS is highly accepted by stroke patients as a rehabilitation tool.</p> <p>Conclusions</p> <p>We introduce a novel VR based paradigm for neurorehabilitation, RGS, which combines specific rehabilitative principles with a psychometric evaluation to provide a personalized and automated training. Our results show that the RGS effectively adjusts to the individual features of the user, allowing for an unsupervised deployment of individualized rehabilitation protocols.</p

Data analytics for image visual complexity and kinect-based videos of rehabilitation exercises

With the recent advances in computer vision and pattern recognition, methods from these fields are successfully applied to solve problems in various domains, including health care and social sciences. In this thesis, two such problems, from different domains, are discussed. First, an application of computer vision and broader pattern recognition in physical therapy is presented. Home-based physical therapy is an essential part of the recovery process in which the patient is prescribed specific exercises in order to improve symptoms and daily functioning of the body. However, poor adherence to the prescribed exercises is a common problem. In our work, we explore methods for improving home-based physical therapy experience. We begin by proposing DyAd, a dynamically difficulty adjustment system which captures the trajectory of the hand movement, evaluates the user's performance quantitatively and adjusts the difficulty level for the next trial of the exercise based on the performance measurements. Next, we introduce ExerciseCheck, a remote monitoring and evaluation platform for home-based physical therapy. ExerciseCheck is capable of capturing exercise information, evaluating the performance, providing therapeutic feedback to the patient and the therapist, checking the progress of the user over the course of the physical therapy, and supporting the patient throughout this period. In our experiments, Parkinson patients have tested our system at a clinic and in their homes during their physical therapy period. Our results suggests that ExerciseCheck is a user-friendly application and can assist patients by providing motivation, and guidance to ensure correct execution of the required exercises. As the second application, and within computer vision paradigm, we focus on visual complexity, an image attribute that humans can subjectively evaluate based on the level of details in the image. Visual complexity has been studied in psychophysics, cognitive science, and, more recently, computer vision, for the purposes of product design, web design, advertising, etc. We first introduce a diverse visual complexity dataset which compromises of seven image categories. We collect the ground-truth scores by comparing the pairwise relationship of images and then convert the pairwise scores to absolute scores using mathematical methods. Furthermore, we propose a method to measure the visual complexity that uses unsupervised information extraction from intermediate convolutional layers of deep neural networks. We derive an activation energy metric that combines convolutional layer activations to quantify visual complexity. The high correlations between ground-truth labels and computed energy scores in our experiments show superiority of our method compared to the previous works. Finally, as an example of the relationship between visual complexity and other image attributes, we demonstrate that, within the context of a category, visually more complex images are more memorable to human observers

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion

Eye quietness and quiet eye in expert and novice golf performance: an electrooculographic analysis

Quiet eye (QE) is the final ocular fixation on the target of an action (e.g., the ball in golf putting). Camerabased eye-tracking studies have consistently found longer QE durations in experts than novices; however, mechanisms underlying QE are not known. To offer a new perspective we examined the feasibility of measuring the QE using electrooculography (EOG) and developed an index to assess ocular activity across time: eye quietness (EQ). Ten expert and ten novice golfers putted 60 balls to a 2.4 m distant hole. Horizontal EOG (2ms resolution) was recorded from two electrodes placed on the outer sides of the eyes. QE duration was measured using a EOG voltage threshold and comprised the sum of the pre-movement and post-movement initiation components. EQ was computed as the standard deviation of the EOG in 0.5 s bins from –4 to +2 s, relative to backswing initiation: lower values indicate less movement of the eyes, hence greater quietness. Finally, we measured club-ball address and swing durations. T-tests showed that total QE did not differ between groups (p = .31); however, experts had marginally shorter pre-movement QE (p = .08) and longer post-movement QE (p < .001) than novices. A group × time ANOVA revealed that experts had less EQ before backswing initiation and greater EQ after backswing initiation (p = .002). QE durations were inversely correlated with EQ from –1.5 to 1 s (rs = –.48 - –.90, ps = .03 - .001). Experts had longer swing durations than novices (p = .01) and, importantly, swing durations correlated positively with post-movement QE (r = .52, p = .02) and negatively with EQ from 0.5 to 1s (r = –.63, p = .003). This study demonstrates the feasibility of measuring ocular activity using EOG and validates EQ as an index of ocular activity. Its findings challenge the dominant perspective on QE and provide new evidence that expert-novice differences in ocular activity may reflect differences in the kinematics of how experts and novices execute skills

Functional changes in the lower extremity after non-immersive virtual reality and physiotherapy following stroke

Objective: To analyse the effect of virtual reality (VR) therapy combined with conventional physiotherapy on balance, gait and motor functional disturbances, and to determine whether there is an influence on motor recovery in the subacute (&lt; 6 months) or chronic (&gt; 6 months) phases after stroke. Methods: A total of 59 stroke inpatients (mean age 60.3 years (standard deviation (SD) 14.8); 14.0 months (SD 25.7) post-stroke) were stratified into 2 groups: subacute (n = 31) and chronic (n = 28). Clinical scales (Fugl-Meyer lower extremity (FM LE); Functional Independence Measure (FIM); Berg Balance Scale (BBS); Functional Ambulation Category (FAC); modified Ashworth scale (MAS); 10-metre walk test (10MWT); and kinematic parameters during specific motor tasks in sitting and standing position (speed; time; jerk; spatial error; length) were applied before and after treatment. The VR treatment lasted for 15 sessions, 5 days/week, 1 h/day. Results: The subacute group underwent significant change in all variables, except MAS and length. The chronic group underwent significant improvement in clinical scales, except MAS and kinematics. Motor impairment improved in the severe = 19 FM LE points, moderate 20-28 FM LE points, mild = 29 FM LE points. Neither time since stroke onset nor affected hemisphere differed significantly between groups. The correlations were investigated between the clinical scales and the kinematic parameters of the whole sample. Moreover, FM LE, BBS, MAS, and speed showed high correlations (R2&gt; 0.70) with independent variables. Conclusion: VR therapy combined with conventional physiotherapy can contribute to functional improvement in the subacute and chronic phases after stroke

The Future of Humanoid Robots

This book provides state of the art scientific and engineering research findings and developments in the field of humanoid robotics and its applications. It is expected that humanoids will change the way we interact with machines, and will have the ability to blend perfectly into an environment already designed for humans. The book contains chapters that aim to discover the future abilities of humanoid robots by presenting a variety of integrated research in various scientific and engineering fields, such as locomotion, perception, adaptive behavior, human-robot interaction, neuroscience and machine learning. The book is designed to be accessible and practical, with an emphasis on useful information to those working in the fields of robotics, cognitive science, artificial intelligence, computational methods and other fields of science directly or indirectly related to the development and usage of future humanoid robots. The editor of the book has extensive R&D experience, patents, and publications in the area of humanoid robotics, and his experience is reflected in editing the content of the book