Augmented reality system for rehabilitation : new approach based on human interaction and biofeedback

University of Technology Sydney. Faculty of Engineering and Information Technology.Rehabilitation is the process of training for someone in order to recover or improve their lost functions caused by neurological deficits. The upper limb rehabilitation system provides relearning of motor skills that are lost due to any neurological injuries via motor rehabilitation training. The process of motor rehabilitation is a form of motor learning via practice or experience. It requires thorough understanding and examination of neural processes involved in producing movement and learning as well as the medical aspects that may affect the central nervous system (CNS) or peripheral nervous system (PNS) in order to develop an effective treatment system. Although there are numerous rehabilitation systems which have been proposed in literatures, a low cost upper limb rehabilitation system that maximizes the functional recovery by stimulating the neural plasticity is not widely available. This is due to lack of motivation during rehabilitation training, lack of real time biofeedback information with complete database, the requirement of one to one attention between physiotherapist and patient, the technique to stimulate human neural plasticity. Therefore, the main objective of this thesis is to develop a novel low cost rehabilitation system that helps recovery not only from loss of physical functions, but also from loss of cognitive functions to fulfill the aforementioned gaps via multimodal technologies such as augmented reality (AR), computer vision and signal processing. In order to fulfill such ambitious objectives, the following contributions have been implemented. Firstly, since improvements in physical functions are targeted, the Rehabilitation system with Biofeedback simulation (RehaBio) is developed. The system enhances user’s motivation via game based therapeutic exercises and biofeedback. For this, AR based therapeutic games are developed to provide eye-hand coordination with inspiration in motivation via immediate audio and visual feedback. All the exercises in RehaBio are developed in a safe training environment for paralyzed patients. In addition to that, real-time biofeedback simulation is developed and integrated to serve in two ways: (1) from the patient’s point of view, the biofeedback simulation motivates the user to execute the movements since it will animate the different muscles in different colors, and (2) from the therapist’s point of view, the muscle simulations and EMG threshold level can be evaluated as patient’s muscle performance throughout the rehabilitation process. Secondly, a new technique that stimulates the human neural plasticity is proposed. This is a virtual human arm (VHA) model that driven by proposed continuous joint angle prediction in real time based on human biological signal, Electromyogram (EMG). The VHA model simulation aims to create the illusion environment in Augmented Reality-based Illusion System (ARIS). Finally, a complete novel upper limb rehabilitation system, Augmented Reality-based Illusion System (ARIS) is developed. The system incorporates some of the developments in RehaBio and real time VHA model to develop the illusion environment. By conducting the rehabilitation training with ARIS, user’s neural plasticity will be stimulated to re-establish the neural pathways and synapses that are able to control mobility. This is achieved via an illusion concept where an illusion scene is created in AR environment to remove the impaired real arm virtually and replace it with VHA model to be perceived as part of the user’s own body. The job of the VHA model in ARIS is when the real arm cannot perform the required task, it will take over the job of the real one and will let the user perceive the sense that the user is still able to perform the reaching movement by their own effort to the destination point. Integration with AR based therapeutic exercises and motivated immediate intrinsic and extrinsic feedback in ARIS leads to serve as a novel upper limb rehabilitation system in a clinical setting. The usability tests and verification process of the proposed systems are conducted and provided with very encouraging results. Furthermore, the developments have been demonstrated to the clinical experts in the rehabilitation field at Port Kembla Hospital. The feedback from the professionals is very positive for both the RehaBio and ARIS systems and they have been recommended to be used in the clinical setting for paralyzed patients

Custom-designed motion-based games for older adults: a review of literature in human-computer interaction

Many older adults, particularly persons living in senior residences and care homes, lead sedentary lifestyles, which reduces their life expectancy. Motion-based video games encourage physical activity and might be an opportunity for these adults to remain active and engaged; however, research efforts in the field have frequently focused on younger audiences and little is known about the requirements and benefits of motion-based games for elderly players. In this paper, we present an overview of motion-based video games and other interactive technologies for older adults. First, we summarize existing approaches towards the definition of motion-based video games – often referred to as exergames – and suggest a categorization of motion-based applications into active video games, exergames, and augmented sports. Second, we use this scheme to classify case studies addressing design efforts particularly directed towards older adults. Third, we analyze these case studies with a focus on potential target audiences, benefits, challenges in their deployment, and future design opportunities to investigate whether motion-based video games can be applied to encourage physical activity among older adults. In this context, special attention is paid to evaluation routines and their implications regarding the deployment of such games in the daily lives of older adults. The results show that many case studies examine isolated aspects of motion-based game design for older adults, and despite the broad range of issues in motion-based interaction for older adults covered by the sum of all research projects, there appears to be a disconnect between laboratory-based research and the deployment of motion-based video games in the daily lives of senior citizens. Our literature review suggests that despite research results suggesting various benefits of motion-based play for older adults, most work in the field of game design for senior citizens has focused on the implementation of accessible user interfaces, and that little is known about the long-term deployment of video games for this audience, which is a crucial step if these games are to be implemented in activity programs of senior residences, care homes, or in therapy

In-home and remote use of robotic body surrogates by people with profound motor deficits

By controlling robots comparable to the human body, people with profound motor deficits could potentially perform a variety of physical tasks for themselves, improving their quality of life. The extent to which this is achievable has been unclear due to the lack of suitable interfaces by which to control robotic body surrogates and a dearth of studies involving substantial numbers of people with profound motor deficits. We developed a novel, web-based augmented reality interface that enables people with profound motor deficits to remotely control a PR2 mobile manipulator from Willow Garage, which is a human-scale, wheeled robot with two arms. We then conducted two studies to investigate the use of robotic body surrogates. In the first study, 15 novice users with profound motor deficits from across the United States controlled a PR2 in Atlanta, GA to perform a modified Action Research Arm Test (ARAT) and a simulated self-care task. Participants achieved clinically meaningful improvements on the ARAT and 12 of 15 participants (80%) successfully completed the simulated self-care task. Participants agreed that the robotic system was easy to use, was useful, and would provide a meaningful improvement in their lives. In the second study, one expert user with profound motor deficits had free use of a PR2 in his home for seven days. He performed a variety of self-care and household tasks, and also used the robot in novel ways. Taking both studies together, our results suggest that people with profound motor deficits can improve their quality of life using robotic body surrogates, and that they can gain benefit with only low-level robot autonomy and without invasive interfaces. However, methods to reduce the rate of errors and increase operational speed merit further investigation.Comment: 43 Pages, 13 Figure