Physical rehabilitation based on kinect serious games: ThG therapy game

This thesis presents a serious game platform developed using Unity 3D game Engine and Kinect V2 sensor as a natural user interface. The aim of this work was to provide a tool for objective evaluation of patients’ movements during physiotherapy sessions as well as a pleasant way that may increase patient engagement on training motor rehabilitation exercises. The developed platform based on Kinect V2 sensor detects 3D motion of different body joints and provides data storage capability in a remote database. The platform for patient’s data management during physiotherapy process includes biometric data, some data relevant for physiotherapist related to patient’s clinical history, obtained scores during serious game based training and values of metrics such as the distance between feet during a game, left and right feet usage frequency and execution time for imposed movement associated with game mechanics. A description of technologies and techniques used for development of the platform and some results related to usability of the platform are presented in this thesis.Esta tese apresenta uma plataforma de jogo séria desenvolvida usando o motor de jogo Unity 3D juntamente com o sensor Kinect V2 como uma interface natural de utilizador. O objetivo deste trabalho foi fornecer uma ferramenta para avaliação objetiva dos movimentos dos pacientes durante as sessões de fisioterapia, bem como uma maneira agradá- vel que possa aumentar o envolvimento do paciente nos treinos de reabilitação motora. A plataforma desenvolvida baseada no sensor Kinect V2 deteta o movimento 3D de diferentes articulações do corpo e fornece capacidade de armazenamento de dados em uma base de dados remota. A plataforma que gere só dados do paciente durante o processo de fisioterapia inclui dados biométricos, alguns dados relevantes para fisioterapeuta relacionados com o historial clínico do paciente, pontuações durante o treino e valores de métricas, como a distância entre os pés durante o jogo, o uso do pé esquerdo e direito, frequência e tempo de execução do movimento associado à mecânica do jogo. A tese apresenta a descrição das tecnologias e técnicas utilizadas para o desenvolvimento da plataforma, e alguns resultados relacionados com o uso da plataforma

Implementation of Open Source applications “Serious Game” for rehabilitation

Serious Games and Virtual Reality (VR) are present nowadays as an alternative to traditional rehabilitation therapies. This project describes the workflow to develop videogames for health monitoring as well as a source of entertainment for physiotherapy patients, primarily patients that suffer hemiparesis caused by a neurological disease like a stroke. We propose the last version of Microsoft Kinect sensors as low cost game controller and the software Unity to develop Open Source Rehabilitation Serious Games. These Serious Games try to imitate physiotherapy sessions performed in movement recovery therapies, reducing the waiting list of patients together with time and costs to hospitals. The premise is that the gameplay makes patients execute upper body exercises alongside equilibrium training, meanwhile they are monitored extracting useful data and results for the physicians.Ingeniería Biomédic

Safety and Feasibility of a First-Person View, Full-Body Interaction Game for Telerehabilitation Post-Stroke

This study explored the feasibility and safety of pairing the Microsoft Kinect® sensor with the Oculus Rift® Head Mounted Display (HMD) as a telerehabilitation technology platform for persons post-stroke. To test initial safety, fourteen participants without disabilities (age 30 ± 8.8 years) engaged in a game-based task using the Microsoft Kinect® with a first-person view using the Oculus Rift®. These tasks were repeated for five participants post-stroke (age 56 ± 3.0 years). No significant adverse events occurred in either study population. When using the Oculus Rift® HMD, three participants without disabilities reported dizziness and nausea. All of the participants post-stroke required hands-on assistance for balance and fall prevention. The intensive nature of physical support necessary for this type of interaction limits the application as a telerehabilitation intervention.  Given the increasing availability of HMDs for commercial use, it is crucial that the safety of immersive games and technologies for telerehabilitation is fully explored

An Internet- and Kinect-Based Multiple Sclerosis Fitness Intervention Training With Pilates Exercises: Development and Usability Study

background: balance impairments are common in people with multiple sclerosis (MS), with reduced ability to maintain position and delayed responses to postural adjustments. Pilates is a popular alternative method for balance training that may reduce the rapid worsening of symptoms and the increased risk of secondary conditions (eg, depression) that are frequently associated with physical inactivity.objective: In this paper, we aimed to describe the design, development, and usability testing of MS Fitness Intervention Training (MS-FIT), a Kinect-based tool implementing Pilates exercises customized for MS. methods: MS-FIT has been developed using a user-centered design approach (design, prototype, user feedback, and analysis) to gain the target user's perspective. a team composed of 1 physical therapist, 2 game programmers, and 1 game designer developed the first version of MS-FIT that integrated the knowledge and experience of the team with MS literature findings related to pilates exercises and balance interventions based on exergames. MS-FIT, developed by using the Unity 3D (Unity Technologies) game engine software with kinect Sensor V2 for Windows, implements exercises for breathing, posture, and balance. Feedback from an Italian panel of experts in MS rehabilitation (neurologists, physiatrists, physical therapists, 1 statistician, and 1 bioengineer) and people with MS was collected to customize the tool for use in MS. The context of MS-FIT is traveling around the world to visit some of the most important cities to learn the aspects of their culture through pictures and stories. At each stay of the travel, the avatar of a Pilates teacher shows the user the exercises to be performed. Overall, 9 people with MS (n=4, 44% women; mean age 42.89, SD 11.97 years; mean disease duration 10.19, SD 9.18 years; Expanded Disability Status Scale score 3.17, SD 0.75) were involved in 3 outpatient user test sessions of 30 minutes; MS-FIT's usability was assessed through an ad hoc questionnaire (maximum value=5; higher the score, higher the usability) evaluating easiness to use, playability, enjoyment, satisfaction, and acceptance.Results: A user-centered design approach was used to develop an accessible and challenging tool for balance training. all people with MS (9/9, 100%) completed the user test sessions and answered the ad hoc questionnaire. the average score on each item ranged from 3.78 (SD 0.67) to 4.33 (SD 1.00), which indicated a high usability level. The feedback and suggestions provided by 64% (9/14) of people with MS and 36% (5/14) of therapists involved in the user test were implemented to refine the first prototype to release MS-FIT 2.0. Conclusions: The participants reported that MS-FIT was a usable tool. It is a promising system for enhancing the motivation and engagement of people with MS in performing exercise with the aim of improving their physical status

Assessment of joint parameters in a Kinect sensor based rehabilitation game

Copyright © 2019 ASME. A Kinect sensor based basketball game is developed for delivering post-stroke exercises in association with a newly developed elbow exoskeleton. Few interesting features such as audio-visual feedback and scoring have been added to the game platform to enhance patient’s engagement during exercises. After playing the game, the performance score has been calculated based on their reachable points and reaching time to measure their current health conditions. During exercises, joint parameters are measured using the motion capture technique of Kinect sensor. The measurement accuracy of Kinect sensor is validated by two comparative studies where two healthy subjects were asked to move elbow joint in front of Kinect sensor wearing the developed elbow exoskeleton. In the first study, the joint information collected from Kinect sensor was compared with the exoskeleton based sensor. In the next study, the length of upperarm and forearm measured by Kinect were compared with the standard anthropometric data. The measurement errors between Kinect and exoskeleton are turned out to be in the acceptable range; 1% for subject 1 and 0.44% for subject 2 in case of joint angle; 5.55% and 3.58% for subject 1 and subject 2 respectively in case of joint torque. The average errors of Kinect measurement as compared to the anthropometric data of the two subjects are 16.52% for upperarm length and 9.87% for forearm length. It shows that Kinect sensor can measure the activity of joint movement with a minimum margin of error

PhyDSLK: a model-driven framework for generating exergames

AbstractIn recent years, we have been witnessing a rapid increase of research on exergames—i.e., computer games that require users to move during gameplay as a form of physical activity and rehabilitation. Properly balancing the need to develop an effective exercise activity with the requirements for a smooth interaction with the software system and an engaging game experience is a challenge. Model-driven software engineering enables the fast prototyping of multiple system variants, which can be very useful for exergame development. In this paper, we propose a framework, PhyDSLK, which eases the development process of personalized and engaging Kinect-based exergames for rehabilitation purposes, providing high-level tools that abstract the technical details of using the Kinect sensor and allows developers to focus on the game design and user experience. The system relies on model-driven software engineering technologies and is made of two main components: (i) an authoring environment relying on a domain-specific language to define the exergame model encapsulating the gameplay that the exergame designer has envisioned and (ii) a code generator that transforms the exergame model into executable code. To validate our approach, we performed a preliminary empirical evaluation addressing development effort and usability of the PhyDSLK framework. The results are promising and provide evidence that people with no experience in game development are able to create exergames with different complexity levels in one hour, after a less-than-two-hour training on PhyDSLK. Also, they consider PhyDSLK usable regardless of the exergame complexity

Fusion of pose and head tracking data for immersive mixed-reality application development

This work addresses the creation of a development framework where application developers can create, in a natural way, immersive physical activities where users experience a 3D first-person perception of full body control. The proposed frame-work is based on commercial motion sensors and a Head-Mounted Display (HMD), and a uses Unity 3D as a unifying environment where user pose, virtual scene and immersive visualization functions are coordinated. Our proposal is exemplified by the development of a toy application showing its practical us

The Design and Implementation of a Kinect-Based Rehabilitation Exercise Monitoring and Guidance System

In preventive and rehabilitative healthcare, physical exercise is a powerful intervention. However, a program may require in the range of thousands of practice repetitions, and many people do not adhere to the program or perform their home exercises incorrectly, making the exercise ineffective, or even dangerous. This thesis research aims to develop a Kinect-based system for rehabilitation exercises monitoring and guidance. In the first step, a feasibility study was carried out on using Kinect for realtime monitoring of rehabilitation exercises while a multi-camera motion tracking system was used to establish the ground truth. In the second step, a Unity-based system was developed to provide realtime monitoring and guidance to patients. The Unity framework was chosen because it enables us to use virtual reality techniques to demonstrate detailed movements to the patient, and to facilitate examination of the quality and quantity of the patient sessions by the clinician. The avatar-based rendering of motion also preserves the privacy of the patients, which is essential for healthcare system

The Design and Implementation of a Kinect-Based Rehabilitation Exercise Monitoring and Guidance System

In preventive and rehabilitative healthcare, physical exercise is a powerful intervention. However, a program may require in the range of thousands of practice repetitions, and many people do not adhere to the program or perform their home exercises incorrectly, making the exercise ineffective, or even dangerous. This thesis research aims to develop a Kinect-based system for rehabilitation exercises monitoring and guidance. In the first step, a feasibility study was carried out on using Kinect for realtime monitoring of rehabilitation exercises while a multi-camera motion tracking system was used to establish the ground truth. In the second step, a Unity-based system was developed to provide realtime monitoring and guidance to patients. The Unity framework was chosen because it enables us to use virtual reality techniques to demonstrate detailed movements to the patient, and to facilitate examination of the quality and quantity of the patient sessions by the clinician. The avatar-based rendering of motion also preserves the privacy of the patients, which is essential for healthcare system

ReHabgame: A non-immersive virtual reality rehabilitation system with applications in neuroscience

This paper proposes the use of a non-immersive virtual reality rehabilitation system ”ReHabgame” developed using Microsoft KinectT M and the ThalmicT M Labs Myo gesture control armband. The ReHabgame was developed based on two third-person video games that provide a feasible possibility of assessing postural control and functional reach tests. It accurately quantifies specific postural control mechanisms including timed standing balance, functional reach tests using real-time anatomical landmark orientation, joint velocity, and acceleration while end trajectories were calculated using an inverse kinematics algorithm. The game was designed to help patients with neurological impairment to be subjected to physiotherapy activity and practice postures of daily activities. The subjective experience of the ReHabgame was studied through the development of an Engagement Questionnaire (EQ) for qualitative, quantitative and Rasch model. The Monte-Carlo Tree Search (MCTS) and Random object (ROG) generator algorithms were used to adapt the physical and gameplay intensity in the ReHabgame based on the Motor Assessment Scale (MAS) and Hierarchical Scoring System (HSS). Rasch analysis was conducted to assess the psychometric characteristics of the ReHabgame and to identify if these are any misfitting items in the game. Rasch rating scale model (RSM) was used to assess the engagement of players in the ReHabgame and evaluate the effectiveness and attractiveness of the game. The results showed that the scales assessing the rehabilitation process met Rasch expectations of reliability, and unidimensionality. Infit and outfit mean squares values are in the range of (0.68 − 1.52) for all considered 16 items. The Root Mean Square Residual (RMSR) and the person separation reliability were acceptable. The item/person map showed that the persons and items were clustered symmetrically