Home-based physical therapy with an interactive computer vision system

In this paper, we present ExerciseCheck. ExerciseCheck is an interactive computer vision system that is sufficiently modular to work with different sources of human pose estimates, i.e., estimates from deep or traditional models that interpret RGB or RGB-D camera input. In a pilot study, we first compare the pose estimates produced by four deep models based on RGB input with those of the MS Kinect based on RGB-D data. The results indicate a performance gap that required us to choose the MS Kinect when we tested ExerciseCheck with Parkinson’s disease patients in their homes. ExerciseCheck is capable of customizing exercises, capturing exercise information, evaluating patient performance, providing therapeutic feedback to the patient and the therapist, checking the progress of the user over the course of the physical therapy, and supporting the patient throughout this period. We conclude that ExerciseCheck is a user-friendly computer vision application that can assist patients by providing motivation and guidance to ensure correct execution of the required exercises. Our results also suggest that while there has been considerable progress in the field of pose estimation using deep learning, current deep learning models are not fully ready to replace RGB-D sensors, especially when the exercises involved are complex, and the patient population being accounted for has to be carefully tracked for its “active range of motion.”Published versio

A Telerehabilitation System for the Selection, Evaluation and Remote Management of Therapies

Telerehabilitation systems that support physical therapy sessions anywhere can help save healthcare costs while also improving the quality of life of the users that need rehabilitation. The main contribution of this paper is to present, as a whole, all the features supported by the innovative Kinect-based Telerehabilitation System (KiReS). In addition to the functionalities provided by current systems, it handles two new ones that could be incorporated into them, in order to give a step forward towards a new generation of telerehabilitation systems. The knowledge extraction functionality handles knowledge about the physical therapy record of patients and treatment protocols described in an ontology, named TRHONT, to select the adequate exercises for the rehabilitation of patients. The teleimmersion functionality provides a convenient, effective and user-friendly experience when performing the telerehabilitation, through a two-way real-time multimedia communication. The ontology contains about 2300 classes and 100 properties, and the system allows a reliable transmission of Kinect video depth, audio and skeleton data, being able to adapt to various network conditions. Moreover, the system has been tested with patients who suffered from shoulder disorders or total hip replacement.This research was funded by the Spanish Ministry of Economy and Competitiveness grant number FEDER/TIN2016-78011-C4-2R

Motion Capture for Telemedicine: A Review of Nintendo Wii, Microsoft Kinect, and PlayStation Move

Access to healthcare has been and continues to be difficult for many around the world. With the introduction of telemedicine, this impediment to attaining medical care has been lifted. Although many avenues of telemedicine exist (and have yet to exist), the use of home video game consoles such as the Nintendo Wii®, Microsoft Kinect®, and PlayStation Move® can be used to measure patient progress outside of the office. Due to the nature of each individual console/system, some unique characteristics exist that allow each system to provide its own clinical potential. A comparative analysis of the clinical implications of the Nintendo Wii®, Microsoft Kinect®, and PlayStation Move® showed that with its ease of use and dynamic accuracy, the Microsoft Kinect® offered the most benefit. With further exploration, using the Microsoft Kinect® for telemedicine will be able to improve medical efficiency and hopefully health outcomes

Design and test of an automated version of the modified Jebsen test of hand function using Microsoft Kinect

Abstract Background The present paper describes the design and evaluation of an automated version of the Modified Jebsen Test of Hand Function (MJT) based on the Microsoft Kinect sensor. Methods The MJT was administered twice to 11 chronic stroke subjects with varying degrees of hand function deficits. The test times of the MJT were evaluated manually by a therapist using a stopwatch, and automatically using the Microsoft Kinect sensor. The ground truth times were assessed based on inspection of the video-recordings. The agreement between the methods was evaluated along with the test-retest performance. Results The results from Bland-Altman analysis showed better agreement between the ground truth times and the automatic MJT time evaluations compared to the agreement between the ground truth times and the times estimated by the therapist. The results from the test-retest performance showed that the subjects significantly improved their performance in several subtests of the MJT, indicating a practice effect. Conclusions The results from the test showed that the Kinect can be used for automating the MJT

Design and Development of ReMoVES Platform for Motion and Cognitive Rehabilitation

Exergames have recently gained popularity and scientific reliability in the field of assistive computing technology for human well-being. The ReMoVES platform, developed by the author, provides motor and cognitive exergames to be performed by elderly or disabled people, in conjunction with traditional rehabilitation. Data acquisition during the exercise takes place through Microsoft Kinect, Leap Motion and touchscreen monitor. The therapist is provided with feedback on patients' activity over time in order to assess their weakness and correct inaccurate movement attitudes. This work describes the technical characteristics of the ReMoVES platform, designed to be used by multiple locations such as rehabilitation centers or the patient's home, while providing a centralized data collection server. The system includes 15 exergames, developed from scratch by the author, with the aim of promoting motor and cognitive activity through patient entertainment. The ReMoVES platform differs from similar solutions for the automatic data processing features in support of the therapist. Three methods are presented: based on classic data analysis, on Support Vector Machine classification, and finally on Recurrent Neural Networks. The results describe how it is possible to discern patient gaming sessions with adequate performance from those with incorrect movements with an accuracy of up to 92%. The system has been used with real patients and a data database is made available to the scientific community. The aim is to encourage the dissemination of such data to lay the foundations for a comparison between similar studies

深度センサーを用いた脳卒中後の運動機能の自動評価システム開発に関する研究

脳卒中後の身体機能の評価に用いられる脳卒中機能評価法(SIAS) は，臨床において，特殊な器具を使わずに判定できる言語ルールベースの基準を用いて評価する。ところが，その基準は，曖昧な言語表現で書かれており，目測による評価であることから，その判定が曖昧になり，恣意性が含まれがちになる。そのため，同一の被験者に対して異なる結果が生じることもあり，それを回避するためには，定量的に評価するシステムが必要である。実際，モーションキャプチャーシステムを使って定量的な評価の試みもなされているが，利用者や使用者の負担が大きく，日常的に臨床で用いるのは困難である。臨床において，理学療法士による曖昧性を含む言語的ルールに基づいた評価判定方法は，すでに日常的に使用されており，自動的に，一意的に評価するような新たなシステムに置き換えることは困難である。よって，システムを構築するにあたり，理学療法士一人ひとりが，目視による計測・判定に近づけるために調整を行うことのできるパラメータを具備する必要がある。そこで，本研究では，安価な深度センサー類，特にKinect やLeapMotion を用いて，日常的に用いられる新たなSIAS の定量的な評価システムを構築する。ここで開発するSIAS の評価システムについて，(1) Kinect の関節検知機能，(2) LeapMotion の関節検知機能，を用いるものと，(3) 深度画像から身体の特徴部位等を検出するアルゴリズムを作りこみ，それを用いて評価するものに分ける。(1) では，麻痺側運動機能に含まれる膝口テスト，股関節屈曲テスト，膝関節伸展テスト，足パットテスト，(2) では，手指テスト，視空間認知検査，(3) では，体幹機能に含まれる腹筋力テストと垂直性テスト，関節可動域測定に含まれる肩関節と足関節について，角度計測値を用いた評価システムが含まれる。このシステムを用いて，身体に問題のない若年成人者を対象に実験を行い，また，実際の対象者となる高齢者や片麻痺者に対する実証実験も同時に行った。その結果，本システムと理学療法士とが評価した結果を比較したところ，高い一致率を示した。理学療法士をはじめとする専門家が各々の判定基準を持ちながら評価する際にも，本研究で開発したシステムを併用して，本システムが出力する数値データによる評価の裏付けが可能である。このことから，本システムは新たに提案できる定量的な評価システムのプロトタイプになると考える。Stroke Impairment Assessment Set (SIAS) is used to evaluate bodily function after stroke. In daily clinical treatment, SIAS is evaluated by using fuzzy linguistic rules without special equipment, which tends to include personal arbitrariness. This may lead to different results among physical therapists for a same client. A quantitative evaluation systems are required to avoid difference evaluations between testers. In fact, motion capture systems have been applied for quantitative measurement methods, but it costs expensive and forces troublesome tasks to clients and operators.SIAS is conducted with linguistic fuzzy rules, thus it is difficult to change it to automatic unified evaluation methods even if it exists. Therefore, it is necessary to develop systems with changeable parameters to adjust each physical therapist. In this study, a new quantitative evaluation system is developed by using low-cost portable depth sensors such as Kinect and Leap Motion for SIAS.Here, systems with three categories are developed: (1) Kinect applications using body joint detection function, (2) Leap Motion applications for finger detection, and (3) depth sensor applications by finding the feature of the body shape that cannot be detected properly by the joint detection. In (1), algorithms for testing paralysis motor functions are developed including knee mouth test, hip flexion test, knee extension test and foot pat test by using joint detected function. In (2), for finger test and visuospatial test systems are developed. In (3), evaluation systems are developed for trunk function inspection in abdominal test and vertical tests, and range of motion inspection in shoulder and ankle joints, by using depth data.Experimental study conducted with healthy young persons, elderly persons and hemiplegia persons. The results of experiments show that the measurement values and the judgements were very similar between the results by this system and physical therapists. Even when the SIAS is tested by physical therapists as the traditional way, it is possible for this system to supply various data to strengthen the judgement. In this way, this prototype can also be applied to various quantitative evaluation methods other than SIAS.甲南大学令和元年度(2019年度