Implementation of Open Source applications “Serious Game” for rehabilitation

Serious Games and Virtual Reality (VR) are present nowadays as an alternative to traditional rehabilitation therapies. This project describes the workflow to develop videogames for health monitoring as well as a source of entertainment for physiotherapy patients, primarily patients that suffer hemiparesis caused by a neurological disease like a stroke. We propose the last version of Microsoft Kinect sensors as low cost game controller and the software Unity to develop Open Source Rehabilitation Serious Games. These Serious Games try to imitate physiotherapy sessions performed in movement recovery therapies, reducing the waiting list of patients together with time and costs to hospitals. The premise is that the gameplay makes patients execute upper body exercises alongside equilibrium training, meanwhile they are monitored extracting useful data and results for the physicians.Ingeniería Biomédic

An Internet- and Kinect-Based Multiple Sclerosis Fitness Intervention Training With Pilates Exercises: Development and Usability Study

background: balance impairments are common in people with multiple sclerosis (MS), with reduced ability to maintain position and delayed responses to postural adjustments. Pilates is a popular alternative method for balance training that may reduce the rapid worsening of symptoms and the increased risk of secondary conditions (eg, depression) that are frequently associated with physical inactivity.objective: In this paper, we aimed to describe the design, development, and usability testing of MS Fitness Intervention Training (MS-FIT), a Kinect-based tool implementing Pilates exercises customized for MS. methods: MS-FIT has been developed using a user-centered design approach (design, prototype, user feedback, and analysis) to gain the target user's perspective. a team composed of 1 physical therapist, 2 game programmers, and 1 game designer developed the first version of MS-FIT that integrated the knowledge and experience of the team with MS literature findings related to pilates exercises and balance interventions based on exergames. MS-FIT, developed by using the Unity 3D (Unity Technologies) game engine software with kinect Sensor V2 for Windows, implements exercises for breathing, posture, and balance. Feedback from an Italian panel of experts in MS rehabilitation (neurologists, physiatrists, physical therapists, 1 statistician, and 1 bioengineer) and people with MS was collected to customize the tool for use in MS. The context of MS-FIT is traveling around the world to visit some of the most important cities to learn the aspects of their culture through pictures and stories. At each stay of the travel, the avatar of a Pilates teacher shows the user the exercises to be performed. Overall, 9 people with MS (n=4, 44% women; mean age 42.89, SD 11.97 years; mean disease duration 10.19, SD 9.18 years; Expanded Disability Status Scale score 3.17, SD 0.75) were involved in 3 outpatient user test sessions of 30 minutes; MS-FIT's usability was assessed through an ad hoc questionnaire (maximum value=5; higher the score, higher the usability) evaluating easiness to use, playability, enjoyment, satisfaction, and acceptance.Results: A user-centered design approach was used to develop an accessible and challenging tool for balance training. all people with MS (9/9, 100%) completed the user test sessions and answered the ad hoc questionnaire. the average score on each item ranged from 3.78 (SD 0.67) to 4.33 (SD 1.00), which indicated a high usability level. The feedback and suggestions provided by 64% (9/14) of people with MS and 36% (5/14) of therapists involved in the user test were implemented to refine the first prototype to release MS-FIT 2.0. Conclusions: The participants reported that MS-FIT was a usable tool. It is a promising system for enhancing the motivation and engagement of people with MS in performing exercise with the aim of improving their physical status

Wheelchair-based game design for older adults

Few leisure activities are accessible to institutionalized older adults using wheelchairs; in consequence, they experience lower levels of perceived health than able-bodied peers. Video games have been shown to be an engaging leisure activity for older adults. In our work, we address the design of wheelchair-accessible motion-based games. We present KINECTWheels, a toolkit designed to integrate wheelchair movements into motion-based games, and Cupcake Heaven, a wheelchair-based video game designed for older adults using wheelchairs. Results of two studies show that KINECTWheels can be applied to make motion-based games wheelchair-accessible, and that wheelchair-based games engage older adults. Through the application of the wheelchair as an enabling technology in play, our work has the potential of encouraging older adults to develop a positive relationship with their wheelchair. Copyright 2013 ACM

Design and Development of ReMoVES Platform for Motion and Cognitive Rehabilitation

Exergames have recently gained popularity and scientific reliability in the field of assistive computing technology for human well-being. The ReMoVES platform, developed by the author, provides motor and cognitive exergames to be performed by elderly or disabled people, in conjunction with traditional rehabilitation. Data acquisition during the exercise takes place through Microsoft Kinect, Leap Motion and touchscreen monitor. The therapist is provided with feedback on patients' activity over time in order to assess their weakness and correct inaccurate movement attitudes. This work describes the technical characteristics of the ReMoVES platform, designed to be used by multiple locations such as rehabilitation centers or the patient's home, while providing a centralized data collection server. The system includes 15 exergames, developed from scratch by the author, with the aim of promoting motor and cognitive activity through patient entertainment. The ReMoVES platform differs from similar solutions for the automatic data processing features in support of the therapist. Three methods are presented: based on classic data analysis, on Support Vector Machine classification, and finally on Recurrent Neural Networks. The results describe how it is possible to discern patient gaming sessions with adequate performance from those with incorrect movements with an accuracy of up to 92%. The system has been used with real patients and a data database is made available to the scientific community. The aim is to encourage the dissemination of such data to lay the foundations for a comparison between similar studies

MaLT - combined motor and language therapy tool for brain injury patients using Kinect

Background: The functional connectivity and structural proximity of elements of the language and motor systems result in frequent co-morbidity post brain injury. Although rehabilitation services are becoming increasingly multidisciplinary and "integrated", treatment for language and motor functions often occurs in isolation. Thus, behavioural therapies which promote neural reorganisation do not reflect the high intersystem connectivity of the neurologically intact brain. As such, there is a pressing need for rehabilitation tools which better reflect and target the impaired cognitive networks. Objectives: The objective of this research is to develop a combined high dosage therapy tool for language and motor rehabilitation. The rehabilitation therapy tool developed, MaLT (Motor and Language Therapy), comprises a suite of computer games targeting both language and motor therapy that use the Kinect sensor as an interaction device. The games developed are intended for use in the home environment over prolonged periods of time. In order to track patients' engagement with the games and their rehabilitation progress, the game records patient performance data for the therapist to interrogate. Methods: MaLT incorporates Kinect-based games, a database of objects and language parameters, and a reporting tool for therapists. Games have been developed that target four major language therapy tasks involving single word comprehension, initial phoneme identification, rhyme identification and a naming task. These tasks have 8 levels each increasing in difficulty. A database of 750 objects is used to programmatically generate appropriate questions for the game, providing both targeted therapy and unique gameplay every time. The design of the games has been informed by therapists and by discussions with a Public Patient Involvement (PPI) group. Results: Pilot MaLT trials have been conducted with three stroke survivors for the duration of 6 to 8 weeks. Patients' performance is monitored through MaLT's reporting facility presented as graphs plotted from patient game data. Performance indicators include reaction time, accuracy, number of incorrect responses and hand use. The resultant games have also been tested by the PPI with a positive response and further suggestions for future modifications made. Conclusion: MaLT provides a tool that innovatively combines motor and language therapy for high dosage rehabilitation in the home. It has demonstrated that motion sensor technology can be successfully combined with a language therapy task to target both upper limb and linguistic impairment in patients following brain injury. The initial studies on stroke survivors have demonstrated that the combined therapy approach is viable and the outputs of this study will inform planned larger scale future trials

A Survey of Applications and Human Motion Recognition with Microsoft Kinect

Microsoft Kinect, a low-cost motion sensing device, enables users to interact with computers or game consoles naturally through gestures and spoken commands without any other peripheral equipment. As such, it has commanded intense interests in research and development on the Kinect technology. In this paper, we present, a comprehensive survey on Kinect applications, and the latest research and development on motion recognition using data captured by the Kinect sensor. On the applications front, we review the applications of the Kinect technology in a variety of areas, including healthcare, education and performing arts, robotics, sign language recognition, retail services, workplace safety training, as well as 3D reconstructions. On the technology front, we provide an overview of the main features of both versions of the Kinect sensor together with the depth sensing technologies used, and review literatures on human motion recognition techniques used in Kinect applications. We provide a classification of motion recognition techniques to highlight the different approaches used in human motion recognition. Furthermore, we compile a list of publicly available Kinect datasets. These datasets are valuable resources for researchers to investigate better methods for human motion recognition and lower-level computer vision tasks such as segmentation, object detection and human pose estimation

Enhancing the measurement of clinical outcomes using Microsoft Kinect

There is a growing body of applications leveraging Microsoft Kinect and the associated Windows Software Development Kit in health and wellness. In particular, this platform has been valuable in developing interactive solutions for rehabilitation including creating more engaging exercise regimens and ensuring that exercises are performed correctly for optimal outcomes. Clinical trials rely upon robust and validated methodologies to measure health status and to detect treatment-related changes over time to enable the efficacy and safety of new drug treatments to be assessed and measured. In many therapeutic areas, traditional outcome measures rely on subjective investigator and patient ratings. Subjective ratings are not always sensitive to detecting small improvements, are subject to inter- and intra-rater variability and limited in their ability to record detailed or subtle aspects of movement and mobility. For these reasons, objective measurements may provide greater sensitivity to detect treatment-related changes where they exist. In this review paper, we explore the use of the Kinect platform to develop low-cost approaches to objectively measure aspects of movement. We consider published applications that measure aspects of gait and balance, upper extremity movement, chest wall motion and facial analysis. In each case, we explore the utility of the approach for clinical trials, and the precision and accuracy of estimates derived from the Kinect output. We conclude that the use of games platforms such as Microsoft Kinect to measure clinical outcomes offer a versatile, easy to use and low-cost approach that may add significant value and utility to clinical drug development, in particular in replacing conventional subjective measures and providing richer information about movement than previously possible in large scale clinical trials, especially in the measurement of gross spatial movements. Regulatory acceptance of clinical outcomes collected in this way will be subject to comprehensive assessment of validity and clinical relevance, and this will require good quality peer-reviewed publications of scientific evidence

Home-based physical therapy with an interactive computer vision system

In this paper, we present ExerciseCheck. ExerciseCheck is an interactive computer vision system that is sufficiently modular to work with different sources of human pose estimates, i.e., estimates from deep or traditional models that interpret RGB or RGB-D camera input. In a pilot study, we first compare the pose estimates produced by four deep models based on RGB input with those of the MS Kinect based on RGB-D data. The results indicate a performance gap that required us to choose the MS Kinect when we tested ExerciseCheck with Parkinson’s disease patients in their homes. ExerciseCheck is capable of customizing exercises, capturing exercise information, evaluating patient performance, providing therapeutic feedback to the patient and the therapist, checking the progress of the user over the course of the physical therapy, and supporting the patient throughout this period. We conclude that ExerciseCheck is a user-friendly computer vision application that can assist patients by providing motivation and guidance to ensure correct execution of the required exercises. Our results also suggest that while there has been considerable progress in the field of pose estimation using deep learning, current deep learning models are not fully ready to replace RGB-D sensors, especially when the exercises involved are complex, and the patient population being accounted for has to be carefully tracked for its “active range of motion.”Published versio

Motion-based technology to support motor skills screening in developing children: A scoping review

Background. Acquiring motor skills is fundamental for children's development since it is linked to cognitive development. However, access to early detection of motor development delays is limited. Aim. This review explores the use and potential of motion-based technology (MBT) as a complement to support and increase access to motor screening in developing children. Methods. Six databases were searched following the PRISMA guidelines to search, select, and assess relevant works where MBT recognised the execution of children's motor skills. Results. 164 studies were analysed to understand the type of MBT used, the motor skills detected, the purpose of using MBT and the age group targeted. Conclusions. There is a gap in the literature aiming to integrate MBT in motor skills development screening and assessment processes. Depth sensors are the prevailing technology offering the largest detection range for children from age 2. Nonetheless, the motor skills detected by MBT represent about half of the motor skills usually observed to screen and assess motor development. Overall, research in this field is underexplored. The use of multimodal approaches, combining various motion-based sensors, may support professionals in the health domain and increase access to early detection programmes.Funding for open access charge: Universidad de Málaga / CBUA

Interactive Games Using Hand-Eye Coordination Method for Autistic Children Therapy

Recent studies have found that imbalanced motor skills (hand-eye coordination) among autism spectrum disorder (ASD) or autistic children cause lower efficiency in daily routines. Autistic children need long-term rehabilitation programs to improve their abilities. However, problems with a lack of motivation to participate in conventional therapy and the high cost of therapy sessions arise over time. Autism children need to do repetitive activities every time therapy sessions been done. Repetitive activities cause children having less interest to participate ongoing therapy sessions. Other than that, the therapy process usually requires a long time to be implemented that cause the relatively high cost had to be paid by the patient's family. The use of technology is seen more effective and less boring for child with autism. However, the use of games technology require gamer to hold game controller. Proposals for motion-based touchless games using NUI technology, such as the Kinect Xbox 360, to be used in rehabilitation are exciting for patients, but the design elements do not meet the requirements of autistic people. Hence, this research proposes a Kinect game based on design elements for autistic rehabilitation. The objective of this research is to identify symptom of eye-hand coordination problem and develop serious game using Kinect technology as the solution for the problem faced by autism children. This technology provides a low-budget solution costs of therapy and games base on 3D sensor without the use of control equipment which must be hold or touch by hand. Moreover, Kinect does not need a controller or additional body-worn attachment during play time. A game prototype was developed and measured, and an evaluation resulted in positive feedback from the user and therapists, thus meeting the objective of this study