Development of augmented reality rehabilitation games integrated with biofeedback for upper limb

Stroke is one of the leading causes of disability in all over the world. This disability greatly impacts the stroke patients' daily life activities. Thus, rehabilitation exercises are essential for post stroke patients to restore their lost functions gradually for daily life activities. Traditional rehabilitation exercises do not motivate the post stroke patients as they are normally humdrum and required expensive equipments. Therefore, this paper presents the development of low -cost motivating webcam colour based visual tracking augmented reality (AR) system with biofeedback for upper-limb post stroke rehabilitation therapy. Augmented Reality is a novel form of human-computer interface which overlay the computer-generated information on the real world environment rather than replaces it. In the developed AR system, two games; Ping Pong Rehab (PPR) and Balloon Collection Rehab (BCR) are created based on game design principle. PPR game trains shoulder and arm muscles during rehabilitation therapy whilst BCR game trains shoulder, arm and forearm muscles. Both games have been built and integrated with Biograph Infiniti software to monitor the muscles' performance. The integrated system will obtain the biofeedback EMG signals from patients that will be utilised for future developments. It allows the patients to monitor their arms and muscles movements in real time on the display screen via low-cost webcam. The system aims for home based rehabilitation system and friendly used by patients themselves. The developed integrated system has tested with able subject and it worked perfectly during the test

Augmented reality system for rehabilitation : new approach based on human interaction and biofeedback

University of Technology Sydney. Faculty of Engineering and Information Technology.Rehabilitation is the process of training for someone in order to recover or improve their lost functions caused by neurological deficits. The upper limb rehabilitation system provides relearning of motor skills that are lost due to any neurological injuries via motor rehabilitation training. The process of motor rehabilitation is a form of motor learning via practice or experience. It requires thorough understanding and examination of neural processes involved in producing movement and learning as well as the medical aspects that may affect the central nervous system (CNS) or peripheral nervous system (PNS) in order to develop an effective treatment system. Although there are numerous rehabilitation systems which have been proposed in literatures, a low cost upper limb rehabilitation system that maximizes the functional recovery by stimulating the neural plasticity is not widely available. This is due to lack of motivation during rehabilitation training, lack of real time biofeedback information with complete database, the requirement of one to one attention between physiotherapist and patient, the technique to stimulate human neural plasticity. Therefore, the main objective of this thesis is to develop a novel low cost rehabilitation system that helps recovery not only from loss of physical functions, but also from loss of cognitive functions to fulfill the aforementioned gaps via multimodal technologies such as augmented reality (AR), computer vision and signal processing. In order to fulfill such ambitious objectives, the following contributions have been implemented. Firstly, since improvements in physical functions are targeted, the Rehabilitation system with Biofeedback simulation (RehaBio) is developed. The system enhances user’s motivation via game based therapeutic exercises and biofeedback. For this, AR based therapeutic games are developed to provide eye-hand coordination with inspiration in motivation via immediate audio and visual feedback. All the exercises in RehaBio are developed in a safe training environment for paralyzed patients. In addition to that, real-time biofeedback simulation is developed and integrated to serve in two ways: (1) from the patient’s point of view, the biofeedback simulation motivates the user to execute the movements since it will animate the different muscles in different colors, and (2) from the therapist’s point of view, the muscle simulations and EMG threshold level can be evaluated as patient’s muscle performance throughout the rehabilitation process. Secondly, a new technique that stimulates the human neural plasticity is proposed. This is a virtual human arm (VHA) model that driven by proposed continuous joint angle prediction in real time based on human biological signal, Electromyogram (EMG). The VHA model simulation aims to create the illusion environment in Augmented Reality-based Illusion System (ARIS). Finally, a complete novel upper limb rehabilitation system, Augmented Reality-based Illusion System (ARIS) is developed. The system incorporates some of the developments in RehaBio and real time VHA model to develop the illusion environment. By conducting the rehabilitation training with ARIS, user’s neural plasticity will be stimulated to re-establish the neural pathways and synapses that are able to control mobility. This is achieved via an illusion concept where an illusion scene is created in AR environment to remove the impaired real arm virtually and replace it with VHA model to be perceived as part of the user’s own body. The job of the VHA model in ARIS is when the real arm cannot perform the required task, it will take over the job of the real one and will let the user perceive the sense that the user is still able to perform the reaching movement by their own effort to the destination point. Integration with AR based therapeutic exercises and motivated immediate intrinsic and extrinsic feedback in ARIS leads to serve as a novel upper limb rehabilitation system in a clinical setting. The usability tests and verification process of the proposed systems are conducted and provided with very encouraging results. Furthermore, the developments have been demonstrated to the clinical experts in the rehabilitation field at Port Kembla Hospital. The feedback from the professionals is very positive for both the RehaBio and ARIS systems and they have been recommended to be used in the clinical setting for paralyzed patients

Serious Games for Wrist Rehabilitation in Juvenile Idiopathic Arthritis

Rehabilitation is a painful and tiring process involving series of exercises that patients must repeat over a long period. Unfortunately, patients often grow bored, frustrated, and lose motivation making rehabilitation less effective. In the recent years video games have been widely used to implement rehabilitation protocols so as to make the process more entertaining, engaging and to keep patients motivated. In this paper, we present an integrated framework we developed for the wrist rehabilitation of patients affected by Juvenile Idiopathic Arthritis (JIA) following a therapeutic protocol at the Clinica Pediatrica G. e D. De Marchi. The framework comprises four video games and a set modules that let the therapists tune and control the exercises the games implemented, record all the patients actions, replay and analyze the sessions. We present the result of a preliminary validation we performed with four poliarticular JIA patients at the clinic under the supervision of the therapists. Overall, we received good feedback both from the young patients, who enjoyed performing known rehabilitation exercises using video games, and therapists who were satisfied with the framework and its potentials for engaging and motivating the patients

Immersive augmented reality system for the training of pattern classification control with a myoelectric prosthesis

Background!#!Hand amputation can have a truly debilitating impact on the life of the affected person. A multifunctional myoelectric prosthesis controlled using pattern classification can be used to restore some of the lost motor abilities. However, learning to control an advanced prosthesis can be a challenging task, but virtual and augmented reality (AR) provide means to create an engaging and motivating training.!##!Methods!#!In this study, we present a novel training framework that integrates virtual elements within a real scene (AR) while allowing the view from the first-person perspective. The framework was evaluated in 13 able-bodied subjects and a limb-deficient person divided into intervention (IG) and control (CG) groups. The IG received training by performing simulated clothespin task and both groups conducted a pre- and posttest with a real prosthesis. When training with the AR, the subjects received visual feedback on the generated grasping force. The main outcome measure was the number of pins that were successfully transferred within 20 min (task duration), while the number of dropped and broken pins were also registered. The participants were asked to score the difficulty of the real task (posttest), fun-factor and motivation, as well as the utility of the feedback.!##!Results!#!The performance (median/interquartile range) consistently increased during the training sessions (4/3 to 22/4). While the results were similar for the two groups in the pretest, the performance improved in the posttest only in IG. In addition, the subjects in IG transferred significantly more pins (28/10.5 versus 14.5/11), and dropped (1/2.5 versus 3.5/2) and broke (5/3.8 versus 14.5/9) significantly fewer pins in the posttest compared to CG. The participants in IG assigned (mean ± std) significantly lower scores to the difficulty compared to CG (5.2 ± 1.9 versus 7.1 ± 0.9), and they highly rated the fun factor (8.7 ± 1.3) and usefulness of feedback (8.5 ± 1.7).!##!Conclusion!#!The results demonstrated that the proposed AR system allows for the transfer of skills from the simulated to the real task while providing a positive user experience. The present study demonstrates the effectiveness and flexibility of the proposed AR framework. Importantly, the developed system is open source and available for download and further development

Important Parameters for Hand Function Assessment of Stroke Patients

Clinical scales such as Fugl-Meyer Assessment and Motor Assessment Scale are widely used to evaluate stroke patient's motor performance. However, the scoring systems of these assessments provide only rough estimation, making it difficult to objectively quantify impairment and disability or even rehabilitation progress throughout their rehabilitation period. In contrast, robot-based assessments are objective, repeatable, and could potentially reduce the assessment time. However, robot-based assessment scales are not as well established as conventional assessment scale and the correlation to conventional assessment scale is unclear. This paper discusses the important parameters in order to assess the hand function of stroke patients. This knowledge will provide a contribution to the development of a new robot-based assessment device effectively by including the important parameters in the device. The important parameters were included in development of iRest and yielded promising results that illustrate the potential of the important parameters in assessing the hand function of stroke patients

Home-based rehabilitation of the shoulder using auxiliary systems and artificial intelligence: an overview

Advancements in modern medicine have bolstered the usage of home-based rehabilitation services for patients, particularly those recovering from diseases or conditions that necessitate a structured rehabilitation process. Understanding the technological factors that can influence the efficacy of home-based rehabilitation is crucial for optimizing patient outcomes. As technologies continue to evolve rapidly, it is imperative to document the current state of the art and elucidate the key features of the hardware and software employed in these rehabilitation systems. This narrative review aims to provide a summary of the modern technological trends and advancements in home-based shoulder rehabilitation scenarios. It specifically focuses on wearable devices, robots, exoskeletons, machine learning, virtual and augmented reality, and serious games. Through an in-depth analysis of existing literature and research, this review presents the state of the art in home-based rehabilitation systems, highlighting their strengths and limitations. Furthermore, this review proposes hypotheses and potential directions for future upgrades and enhancements in these technologies. By exploring the integration of these technologies into home-based rehabilitation, this review aims to shed light on the current landscape and offer insights into the future possibilities for improving patient outcomes and optimizing the effectiveness of home-based rehabilitation programs.info:eu-repo/semantics/publishedVersio

Novel Bidirectional Body - Machine Interface to Control Upper Limb Prosthesis

Objective. The journey of a bionic prosthetic user is characterized by the opportunities and limitations involved in adopting a device (the prosthesis) that should enable activities of daily living (ADL). Within this context, experiencing a bionic hand as a functional (and, possibly, embodied) limb constitutes the premise for mitigating the risk of its abandonment through the continuous use of the device. To achieve such a result, different aspects must be considered for making the artificial limb an effective support for carrying out ADLs. Among them, intuitive and robust control is fundamental to improving amputees’ quality of life using upper limb prostheses. Still, as artificial proprioception is essential to perceive the prosthesis movement without constant visual attention, a good control framework may not be enough to restore practical functionality to the limb. To overcome this, bidirectional communication between the user and the prosthesis has been recently introduced and is a requirement of utmost importance in developing prosthetic hands. Indeed, closing the control loop between the user and a prosthesis by providing artificial sensory feedback is a fundamental step towards the complete restoration of the lost sensory-motor functions. Within my PhD work, I proposed the development of a more controllable and sensitive human-like hand prosthesis, i.e., the Hannes prosthetic hand, to improve its usability and effectiveness. Approach. To achieve the objectives of this thesis work, I developed a modular and scalable software and firmware architecture to control the Hannes prosthetic multi-Degree of Freedom (DoF) system and to fit all users’ needs (hand aperture, wrist rotation, and wrist flexion in different combinations). On top of this, I developed several Pattern Recognition (PR) algorithms to translate electromyographic (EMG) activity into complex movements. However, stability and repeatability were still unmet requirements in multi-DoF upper limb systems; hence, I started by investigating different strategies to produce a more robust control. To do this, EMG signals were collected from trans-radial amputees using an array of up to six sensors placed over the skin. Secondly, I developed a vibrotactile system to implement haptic feedback to restore proprioception and create a bidirectional connection between the user and the prosthesis. Similarly, I implemented an object stiffness detection to restore tactile sensation able to connect the user with the external word. This closed-loop control between EMG and vibration feedback is essential to implementing a Bidirectional Body - Machine Interface to impact amputees’ daily life strongly. For each of these three activities: (i) implementation of robust pattern recognition control algorithms, (ii) restoration of proprioception, and (iii) restoration of the feeling of the grasped object's stiffness, I performed a study where data from healthy subjects and amputees was collected, in order to demonstrate the efficacy and usability of my implementations. In each study, I evaluated both the algorithms and the subjects’ ability to use the prosthesis by means of the F1Score parameter (offline) and the Target Achievement Control test-TAC (online). With this test, I analyzed the error rate, path efficiency, and time efficiency in completing different tasks. Main results. Among the several tested methods for Pattern Recognition, the Non-Linear Logistic Regression (NLR) resulted to be the best algorithm in terms of F1Score (99%, robustness), whereas the minimum number of electrodes needed for its functioning was determined to be 4 in the conducted offline analyses. Further, I demonstrated that its low computational burden allowed its implementation and integration on a microcontroller running at a sampling frequency of 300Hz (efficiency). Finally, the online implementation allowed the subject to simultaneously control the Hannes prosthesis DoFs, in a bioinspired and human-like way. In addition, I performed further tests with the same NLR-based control by endowing it with closed-loop proprioceptive feedback. In this scenario, the results achieved during the TAC test obtained an error rate of 15% and a path efficiency of 60% in experiments where no sources of information were available (no visual and no audio feedback). Such results demonstrated an improvement in the controllability of the system with an impact on user experience. Significance. The obtained results confirmed the hypothesis of improving robustness and efficiency of a prosthetic control thanks to of the implemented closed-loop approach. The bidirectional communication between the user and the prosthesis is capable to restore the loss of sensory functionality, with promising implications on direct translation in the clinical practice

Rehabilitation Engineering

Population ageing has major consequences and implications in all areas of our daily life as well as other important aspects, such as economic growth, savings, investment and consumption, labour markets, pensions, property and care from one generation to another. Additionally, health and related care, family composition and life-style, housing and migration are also affected. Given the rapid increase in the aging of the population and the further increase that is expected in the coming years, an important problem that has to be faced is the corresponding increase in chronic illness, disabilities, and loss of functional independence endemic to the elderly (WHO 2008). For this reason, novel methods of rehabilitation and care management are urgently needed. This book covers many rehabilitation support systems and robots developed for upper limbs, lower limbs as well as visually impaired condition. Other than upper limbs, the lower limb research works are also discussed like motorized foot rest for electric powered wheelchair and standing assistance device

IoT Based Virtual Reality Game for Physio-therapeutic Patients

Biofeedback therapy trains the patient to control voluntarily the involuntary process of their body. This non-invasive and non-drug treatment is also used as a means to rehabilitate the physical impairments that may follow a stroke, a traumatic brain injury or even in neurological aspects within occupational therapy. The idea behind this study is based on using immersive gaming as a tool for physical rehabilitation that combines the idea of biofeedback and physical computing to get a patient emotionally involved in a game that requires them to do the exercises in order to interact with the game. This game is aimed towards addressing the basic treatment for ‘Frozen Shoulder’. In this work, the physical motions are captured by the wearable ultrasonic sensor attached temporarily to the various limbs of the patient. The data received from the sensors are then sent to the game via serial wireless communication. There are two main aspects to this study: motion capturing and game design. The current status of the application is a single ultrasonic detector. The experimental result shows that physio-therapeutic patients are benefited through the IoT based virtual reality game