Detecting affective states in virtual rehabilitation

Virtual rehabilitation supports motor training following stroke by means of tailored virtual environments. To optimize therapy outcome, virtual rehabilitation systems automatically adapt to the different patients' changing needs. Adaptation decisions should ideally be guided by both the observable performance and the hidden mind state of the user. We hypothesize that some affective aspects can be inferred from observable metrics. Here we present preliminary results of a classification exercise to decide on 4 states; tiredness, tension, pain and satisfaction. Descriptors of 3D hand movement and finger pressure were collected from 2 post-stroke participants while they practice on a virtual rehabilitation platform. Linear Support Vector Machine models were learnt to unfold a predictive relation between observation and the affective states considered. Initial results are promising (ROC Area under the curve (mean±std): 0.713 ± 0.137). Confirmation of these opens the door to incorporate surrogates of mind state into the algorithm deciding on therapy adaptation

The development of an adaptive upper-limb stroke rehabilitation robotic system

<p>Abstract</p> <p>Background</p> <p>Stroke is the primary cause of adult disability. To support this large population in recovery, robotic technologies are being developed to assist in the delivery of rehabilitation. This paper presents an automated system for a rehabilitation robotic device that guides stroke patients through an upper-limb reaching task. The system uses a decision theoretic model (a partially observable Markov decision process, or POMDP) as its primary engine for decision making. The POMDP allows the system to automatically modify exercise parameters to account for the specific needs and abilities of different individuals, and to use these parameters to take appropriate decisions about stroke rehabilitation exercises.</p> <p>Methods</p> <p>The performance of the system was evaluated by comparing the decisions made by the system with those of a human therapist. A single patient participant was paired up with a therapist participant for the duration of the study, for a total of six sessions. Each session was an hour long and occurred three times a week for two weeks. During each session, three steps were followed: (A) after the system made a decision, the therapist either agreed or disagreed with the decision made; (B) the researcher had the device execute the decision made by the therapist; (C) the patient then performed the reaching exercise. These parts were repeated in the order of A-B-C until the end of the session. Qualitative and quantitative question were asked at the end of each session and at the completion of the study for both participants.</p> <p>Results</p> <p>Overall, the therapist agreed with the system decisions approximately 65% of the time. In general, the therapist thought the system decisions were believable and could envision this system being used in both a clinical and home setting. The patient was satisfied with the system and would use this system as his/her primary method of rehabilitation.</p> <p>Conclusions</p> <p>The data collected in this study can only be used to provide insight into the performance of the system since the sample size was limited. The next stage for this project is to test the system with a larger sample size to obtain significant results.</p

Development of a Dynamical Systems Model and Adaptive Intervention Strategy for Stroke Rehabilitation

Each year, approximately 795000 people experience stroke in the United States. After stroke onset, about 80% of patients suffer from hemiparesis, the weakness of face or limb on one side. These people outside clinical setting may develop learned nonuse, which may result in long-term limitation in the outcome of motor recovery. Interventions such as the Constraint Induced Movement Therapy has shown promise in reversing nonuse. However, many chronic individuals do not have access to such training programs. Therefore, some novel tools capable of continuous monitoring patients\u27 health status and furthermore providing appropriate interventions for patients in ambient setting is required to optimize stroke rehabilitation.Dynamical systems modeling combined with wearable technologies may allow to quantitatively describe nonuse evolution. We developed and validated a pendulum-based dynamical model using experimental and simulated motion data. Without direct access to internal torques, we proposed an inverse dynamics-based metric to quantify and compare motor performance between limbs. The primary outcome measure is RMSE between the simulated driving torque for experimental and reference motions. Using RMSEs, we defined a novel within-person comparison factor w participant limb [w], and compared it to the Fugl-Mayer Assessment score. Our dynamic model is capable of mimicking upper-extremity shoulder flexion dynamics. RMSE is sensitive to differences in motor performance between limbs for both groups. Finally, the factor w participant limb [w] is related to post-stroke severity. The arm dynamical model may have great potential for monitoring time-varying motor impairment using noninvasive sensing.Markov decision process (MDP) is a comparatively simple approach of simulation modelling. We implemented MDP to understand the primary factors behind human dynamic decision making on limb choice during rehabilitation. The model showed good performance in understanding the crucial motivators (or barriers) underlying patients\u27 behaviors. We found that a patient with higher motivation, greater perceived benefits of paretic-limb use, and milder motor impairment, would show a better adherence to using paretic limb in physical activity, which suggests that we may provide related interventions in clinical practice to promote a better recovery outcome. MDP modelling may be suggestive in designing cost-effective adaptive intervention for stroke rehabilitation

Unobtrusive inference of affective states in virtual rehabilitation from upper limb motions: a feasibility study

Virtual rehabilitation environments may afford greater patient personalization if they could harness the patient's affective state. Four states: anxiety, pain, engagement and tiredness (either physical or psychological), were hypothesized to be inferable from observable metrics of hand location and gripping strength -relevant for rehabilitation-. Contributions are; (a) multiresolution classifier built from Semi-Naïve Bayesian classifiers, and (b) establishing predictive relations for the considered states from the motor proxies capitalizing on the proposed classifier with recognition levels sufficient for exploitation. 3D hand locations and gripping strength streams were recorded from 5 post-stroke patients whilst undergoing motor rehabilitation therapy administered through virtual rehabilitation along 10 sessions over 4 weeks. Features from the streams characterized the motor dynamics, while spontaneous manifestations of the states were labelled from concomitant videos by experts for supervised classification. The new classifier was compared against baseline support vector machine (SVM) and random forest (RF) with all three exhibiting comparable performances. Inference of the aforementioned states departing from chosen motor surrogates appears feasible, expediting increased personalization of virtual motor neurorehabilitation therapies

A review of computer vision-based approaches for physical rehabilitation and assessment

The computer vision community has extensively researched the area of human motion analysis, which primarily focuses on pose estimation, activity recognition, pose or gesture recognition and so on. However for many applications, like monitoring of functional rehabilitation of patients with musculo skeletal or physical impairments, the requirement is to comparatively evaluate human motion. In this survey, we capture important literature on vision-based monitoring and physical rehabilitation that focuses on comparative evaluation of human motion during the past two decades and discuss the state of current research in this area. Unlike other reviews in this area, which are written from a clinical objective, this article presents research in this area from a computer vision application perspective. We propose our own taxonomy of computer vision-based rehabilitation and assessment research which are further divided into sub-categories to capture novelties of each research. The review discusses the challenges of this domain due to the wide ranging human motion abnormalities and difficulty in automatically assessing those abnormalities. Finally, suggestions on the future direction of research are offered

Assessment-driven selection and adaptation of exercise difficulty in robot-assisted therapy: a pilot study with a hand rehabilitation robot

Background Selecting and maintaining an engaging and challenging training difficulty level in robot-assisted stroke rehabilitation remains an open challenge. Despite the ability of robotic systems to provide objective and accurate measures of function and performance, the selection and adaptation of exercise difficulty levels is typically left to the experience of the supervising therapist. Methods We introduce a patient-tailored and adaptive robot-assisted therapy concept to optimally challenge patients from the very first session and throughout therapy progress. The concept is evaluated within a four-week pilot study in six subacute stroke patients performing robot-assisted rehabilitation of hand function. Robotic assessments of both motor and sensory impairments of hand function conducted prior to the therapy are used to adjust exercise parameters and customize difficulty levels. During therapy progression, an automated routine adapts difficulty levels from session to session to maintain patients’ performance around a target level of 70%, to optimally balance motivation and challenge. Results Robotic assessments suggested large differences in patients’ sensorimotor abilities that are not captured by clinical assessments. Exercise customization based on these assessments resulted in an average initial exercise performance around 70% (62% ± 20%, mean ± std), which was maintained throughout the course of the therapy (64% ± 21%). Patients showed reduction in both motor and sensory impairments compared to baseline as measured by clinical and robotic assessments. The progress in difficulty levels correlated with improvements in a clinical impairment scale (Fugl-Meyer Assessment) (r s = 0.70), suggesting that the proposed therapy was effective at reducing sensorimotor impairment. Conclusions Initial robotic assessments combined with progressive difficulty adaptation have the potential to automatically tailor robot-assisted rehabilitation to the individual patient. This results in optimal challenge and engagement of the patient, may facilitate sensorimotor recovery after neurological injury, and has implications for unsupervised robot-assisted therapy in the clinic and home environment.ISSN:1743-000

A Survey of Applications and Human Motion Recognition with Microsoft Kinect

Microsoft Kinect, a low-cost motion sensing device, enables users to interact with computers or game consoles naturally through gestures and spoken commands without any other peripheral equipment. As such, it has commanded intense interests in research and development on the Kinect technology. In this paper, we present, a comprehensive survey on Kinect applications, and the latest research and development on motion recognition using data captured by the Kinect sensor. On the applications front, we review the applications of the Kinect technology in a variety of areas, including healthcare, education and performing arts, robotics, sign language recognition, retail services, workplace safety training, as well as 3D reconstructions. On the technology front, we provide an overview of the main features of both versions of the Kinect sensor together with the depth sensing technologies used, and review literatures on human motion recognition techniques used in Kinect applications. We provide a classification of motion recognition techniques to highlight the different approaches used in human motion recognition. Furthermore, we compile a list of publicly available Kinect datasets. These datasets are valuable resources for researchers to investigate better methods for human motion recognition and lower-level computer vision tasks such as segmentation, object detection and human pose estimation