Home-based physical therapy with an interactive computer vision system

In this paper, we present ExerciseCheck. ExerciseCheck is an interactive computer vision system that is sufficiently modular to work with different sources of human pose estimates, i.e., estimates from deep or traditional models that interpret RGB or RGB-D camera input. In a pilot study, we first compare the pose estimates produced by four deep models based on RGB input with those of the MS Kinect based on RGB-D data. The results indicate a performance gap that required us to choose the MS Kinect when we tested ExerciseCheck with Parkinson’s disease patients in their homes. ExerciseCheck is capable of customizing exercises, capturing exercise information, evaluating patient performance, providing therapeutic feedback to the patient and the therapist, checking the progress of the user over the course of the physical therapy, and supporting the patient throughout this period. We conclude that ExerciseCheck is a user-friendly computer vision application that can assist patients by providing motivation and guidance to ensure correct execution of the required exercises. Our results also suggest that while there has been considerable progress in the field of pose estimation using deep learning, current deep learning models are not fully ready to replace RGB-D sensors, especially when the exercises involved are complex, and the patient population being accounted for has to be carefully tracked for its “active range of motion.”Published versio

An interface to virtual environments for people who are blind using Wii technology - mental models and navigation

Accessible games, both for serious and for entertainment purposes, would allow inclusion and participation for those with disabilities. Research into the development of accessible games, and accessible virtual environments, is discussed. Research into accessible Virtual Environments has demonstrated great potential for allowing people who are blind to explore new spaces, reducing their reliance on guides, and aiding development of more efficient spatial maps and strategies. Importantly, Lahav and Mioduser (2005, 2008) have demonstrated that, when exploring virtual spaces, people who are blind use more and different strategies than when exploring real physical spaces, and develop relatively accurate spatial representations of them. The present paper describes the design, development and evaluation of a system in which a virtual environment may be explored by people who are blind using Nintendo Wii devices, with auditory and haptic feedback. The nature of the various types of feedback is considered, with the aim of creating an intuitive and usable system. Using Wii technology has many advantages, not least of which are that it is mainstream, readily available and cheap. The potential of the system for exploration and navigation is demonstrated. Results strongly support the possibilities of the system for facilitating and supporting the construction of cognitive maps and spatial strategies. Intelligent support is discussed. Systems such as the present one will facilitate the development of accessible games, and thus enable Universal Design and accessible interactive technology to become more accepted and widespread

Kinect as an access device for people with cerebral palsy: A preliminary study

Cerebral palsy (CP) describes a group of disorders affecting the development of movement and posture, causingactivity limitation. Access to technology can alleviate some of these limitations. Many studies have used vision- based movement capture systems to overcome problems related to discomfort and fear of wearing devices. Incontrast, there has been no research assessing the behavior of vision-based movement capture systems in peoplewith involuntary movements. In this paper, we look at the potential of the Kinect sensor as an assistive technologyfor people with cerebral palsy. We developed a serious game, called KiSens Números, to study the behavior ofKinect in this context and eighteen subjects with cerebral palsy used it to complete a set of sessions. The resultsof the experiments show that Kinect filters some of peoples involuntary movements, confirming the potential ofKinect as an assistive technology for people with motor disabilities

Diseño de entornos de realidad virtual aplicables a sistemas de robótica asistencial: un análisis literario

Virtual Reality (VR) environments can be applied to assistive robotics to improve the effectiveness and the user experience perception in the rehabilitation process due to its innovative nature, getting to entertain patients while they recover their motor functions. This literature review pretends to analyze some design principles of VR environments developed for upper limb rehabilitation processes. The idea is to identify features related to peripheral and central nervous systems, types of information included as feedback to increase the user's levels of immersion having a positive impact on the user's performance and experience during the treatment. A total of 32 articles published in Scopus, IEEE, PubMed, and Web of Science in the last four years were reviewed. We present the article selection process, the division by concepts presented previously, and the guidelines that can be considered for the design of VR environments applicable to assistive robots for upper limbs rehabilitation processes.Los entornos de Realidad Virtual (RV) aplicables a sistemas de robótica asistencial pueden ser diseñados de manera que mejoren la efectividad y la experiencia de usuario de los procesos de rehabilitación debido a su naturaleza novedosa, logrando entretener a los pacientes mientras recuperan sus funciones motoras. Esta revisión literaria pretende analizar los criterios de diseño de entornos de RV utilizados en procesos de rehabilitación de miembro superior, identificando las características de entornos para rehabilitación de problemas asociados el sistema nervioso central y periféricos, los tipos de información que se realimenta al usuario para beneficiar los niveles de inmersión y su impacto en términos del desempeño y la experiencia del usuario en tratamiento. Un total de 32 artículos publicados en revistas indexadas de Scopus, IEEE, PubMed y Web of Science en los últimos cuatro años fueron revisados. Se presenta el proceso de selección de artículos, la división por las temáticas presentadas anteriormente y los lineamientos generales que pueden ser considerados para el diseño de entornos de RV aplicables a robots asistenciales en procesos de rehabilitación de miembro superior

An Overview of Self-Adaptive Technologies Within Virtual Reality Training

This overview presents the current state-of-the-art of self-adaptive technologies within virtual reality (VR) training. Virtual reality training and assessment is increasingly used for five key areas: medical, industrial & commercial training, serious games, rehabilitation and remote training such as Massive Open Online Courses (MOOCs). Adaptation can be applied to five core technologies of VR including haptic devices, stereo graphics, adaptive content, assessment and autonomous agents. Automation of VR training can contribute to automation of actual procedures including remote and robotic assisted surgery which reduces injury and improves accuracy of the procedure. Automated haptic interaction can enable tele-presence and virtual artefact tactile interaction from either remote or simulated environments. Automation, machine learning and data driven features play an important role in providing trainee-specific individual adaptive training content. Data from trainee assessment can form an input to autonomous systems for customised training and automated difficulty levels to match individual requirements. Self-adaptive technology has been developed previously within individual technologies of VR training. One of the conclusions of this research is that while it does not exist, an enhanced portable framework is needed and it would be beneficial to combine automation of core technologies, producing a reusable automation framework for VR training

The use and effect of video game design theory in the creation of game-based systems for upper limb stroke rehabilitation

Upper limb exercise is often neglected during post-stroke rehabilitation. Video games have been shown to be useful in providing environments in which patients can practise repetitive, functionally meaningful movements, and in inducing neuroplasticity. The design of video games is often focused upon a number of fundamental principles, such as reward, goals, challenge and the concept of meaningful play, and these same principles are important in the design of games for rehabilitation. Further to this, there have been several attempts for the strengthening of the relationship between commercial game design and rehabilitative game design, the former providing insight into factors that can increase motivation and engagement with the latter. In this article, we present an overview of various game design principles and the theoretical grounding behind their presence, in addition to attempts made to utilise these principles in the creation of upper limb stroke rehabilitation systems and the outcomes of their use. We also present research aiming to move the collaborative efforts of designers and therapists towards a model for the structured design of these games and the various steps taken concerning the theoretical classification and mapping of game design concepts with intended cognitive and motor outcomes