Augmented reality system for rehabilitation : new approach based on human interaction and biofeedback

University of Technology Sydney. Faculty of Engineering and Information Technology.Rehabilitation is the process of training for someone in order to recover or improve their lost functions caused by neurological deficits. The upper limb rehabilitation system provides relearning of motor skills that are lost due to any neurological injuries via motor rehabilitation training. The process of motor rehabilitation is a form of motor learning via practice or experience. It requires thorough understanding and examination of neural processes involved in producing movement and learning as well as the medical aspects that may affect the central nervous system (CNS) or peripheral nervous system (PNS) in order to develop an effective treatment system. Although there are numerous rehabilitation systems which have been proposed in literatures, a low cost upper limb rehabilitation system that maximizes the functional recovery by stimulating the neural plasticity is not widely available. This is due to lack of motivation during rehabilitation training, lack of real time biofeedback information with complete database, the requirement of one to one attention between physiotherapist and patient, the technique to stimulate human neural plasticity. Therefore, the main objective of this thesis is to develop a novel low cost rehabilitation system that helps recovery not only from loss of physical functions, but also from loss of cognitive functions to fulfill the aforementioned gaps via multimodal technologies such as augmented reality (AR), computer vision and signal processing. In order to fulfill such ambitious objectives, the following contributions have been implemented. Firstly, since improvements in physical functions are targeted, the Rehabilitation system with Biofeedback simulation (RehaBio) is developed. The system enhances user’s motivation via game based therapeutic exercises and biofeedback. For this, AR based therapeutic games are developed to provide eye-hand coordination with inspiration in motivation via immediate audio and visual feedback. All the exercises in RehaBio are developed in a safe training environment for paralyzed patients. In addition to that, real-time biofeedback simulation is developed and integrated to serve in two ways: (1) from the patient’s point of view, the biofeedback simulation motivates the user to execute the movements since it will animate the different muscles in different colors, and (2) from the therapist’s point of view, the muscle simulations and EMG threshold level can be evaluated as patient’s muscle performance throughout the rehabilitation process. Secondly, a new technique that stimulates the human neural plasticity is proposed. This is a virtual human arm (VHA) model that driven by proposed continuous joint angle prediction in real time based on human biological signal, Electromyogram (EMG). The VHA model simulation aims to create the illusion environment in Augmented Reality-based Illusion System (ARIS). Finally, a complete novel upper limb rehabilitation system, Augmented Reality-based Illusion System (ARIS) is developed. The system incorporates some of the developments in RehaBio and real time VHA model to develop the illusion environment. By conducting the rehabilitation training with ARIS, user’s neural plasticity will be stimulated to re-establish the neural pathways and synapses that are able to control mobility. This is achieved via an illusion concept where an illusion scene is created in AR environment to remove the impaired real arm virtually and replace it with VHA model to be perceived as part of the user’s own body. The job of the VHA model in ARIS is when the real arm cannot perform the required task, it will take over the job of the real one and will let the user perceive the sense that the user is still able to perform the reaching movement by their own effort to the destination point. Integration with AR based therapeutic exercises and motivated immediate intrinsic and extrinsic feedback in ARIS leads to serve as a novel upper limb rehabilitation system in a clinical setting. The usability tests and verification process of the proposed systems are conducted and provided with very encouraging results. Furthermore, the developments have been demonstrated to the clinical experts in the rehabilitation field at Port Kembla Hospital. The feedback from the professionals is very positive for both the RehaBio and ARIS systems and they have been recommended to be used in the clinical setting for paralyzed patients

Real time biosignal-Driven illusion system for upper limb rehabilitation

This paper presents design and development of real time biosignal-driven illusion system: Augmented Reality based Illusion System (ARIS) for upper limb motor rehabilitation. ARIS is a hospital / home based self-motivated whole arm rehabilitation system that aims to improve and restore the lost upper limb functions due to Cerebrovascular Accident (CVA) or stroke. Taking the advantage of human brain plasticity nature, the system incorporates with number of technologies to provide fast recovery by re-establishing the neural pathways and synapses that able to control the mobility. These technologies include Augmented Reality (AR) where illusion environment is developed, computer vision technology to track multiple colors in real time, EMG acquisition system to detect the user intention in real time and 3D modelling library to develop Virtual Arm (VA) model where human biomechanics are applied to mimic the movement of real arm. The system operates according to the user intention via surface electromyography (sEMG) threshold level. In the case of real arm cannot reach to the desired position, VA will take over the job of real arm to complete the exercise. The effectiveness of the developed ARIS has evaluated via questionnaire, graphical and analytical measurements which provided with positive results

Active interaction control applied to a lower limb rehabilitation robot by using EMG recognition and impedance model

Purpose – The purpose of this paper is to propose a seamless active interaction control method integrating electromyography (EMG)-triggered assistance and the adaptive impedance control scheme for parallel robot-assisted lower limb rehabilitation and training. Design/methodology/approach – An active interaction control strategy based on EMG motion recognition and adaptive impedance model is implemented on a six-degrees of freedom parallel robot for lower limb rehabilitation. The autoregressive coefficients of EMG signals integrating with a support vector machine classifier are utilized to predict the movement intention and trigger the robot assistance. An adaptive impedance controller is adopted to influence the robot velocity during the exercise, and in the meantime, the user’s muscle activity level is evaluated online and the robot impedance is adapted in accordance with the recovery conditions. Findings – Experiments on healthy subjects demonstrated that the proposed method was able to drive the robot according to the user’s intention, and the robot impedance can be updated with the muscle conditions. Within the movement sessions, there was a distinct increase in the muscle activity levels for all subjects with the active mode in comparison to the EMG-triggered mode. Originality/value – Both users’ movement intention and voluntary participation are considered, not only triggering the robot when people attempt to move but also changing the robot movement in accordance with user’s efforts. The impedance model here responds directly to velocity changes, and thus allows the exercise along a physiological trajectory. Moreover, the muscle activity level depends on both the normalized EMG signals and the weight coefficients of involved muscles

Augmented Reality based Illusion System with biofeedback

This paper presents the Augmented Reality based Illusion System (ARIS) with biofeedback for upper limb rehabilitation. It aims for fast recovery of motor deficit with motivational approach over traditional upper limb rehabilitation methods. The system incorporates with Augmented Reality (AR) technology to develop upper limb rehabilitation exercise and computer vision with color recognition technique to create 'Fool-the-Brain' concept for fast recovery of neural impairment due to various motor injuries. The rehabilitation exercise in ARIS is aiming to increase the shoulder joint range of motions by performing reaching movements and to strengthen the associated muscles. 'Fool-the-Brain' concept is introduced during performing rehabilitation exercise to perceive artificial visual feedback where user's real impaired arm is covered by Virtual Arm (VA). When the real arm cannot perform the required task, VA will take over the job of real one and will make the user perceives the sense that is still be able to accomplish the task with own effort. Evaluation has performed and results indicate that ARIS with biofeedback is a potential upper limb rehabilitation system for people with upper limb motor deficit. © 2014 IEEE

Computational Intelligence in Electromyography Analysis

Electromyography (EMG) is a technique for evaluating and recording the electrical activity produced by skeletal muscles. EMG may be used clinically for the diagnosis of neuromuscular problems and for assessing biomechanical and motor control deficits and other functional disorders. Furthermore, it can be used as a control signal for interfacing with orthotic and/or prosthetic devices or other rehabilitation assists. This book presents an updated overview of signal processing applications and recent developments in EMG from a number of diverse aspects and various applications in clinical and experimental research. It will provide readers with a detailed introduction to EMG signal processing techniques and applications, while presenting several new results and explanation of existing algorithms. This book is organized into 18 chapters, covering the current theoretical and practical approaches of EMG research

Model-free Optimization of Trajectory And Impedance Parameters on Exercise Robots With Applications To Human Performance And Rehabilitation

This dissertation focuses on the study and optimization of human training and its physiological effects through the use of advanced exercise machines (AEMs). These machines provide an invaluable contribution to advanced training by combining exercise physiology with technology. Unlike conventional exercise machines (CEMs), AEMs provide controllable trajectories and impedances by using electric motors and control systems. Therefore, they can produce various patterns even in the absence of gravity. Moreover, the ability of the AEMs to target multiple physiological systems makes them the best available option to improve human performance and rehabilitation. During the early stage of the research, the physiological effects produced under training by the manual regulation of the trajectory and impedance parameters of the AEMs were studied. Human dynamics appear as not only complex but also unique and time-varying due to the particular features of each person such as its musculoskeletal distribution, level of fatigue,fitness condition, hydration, etc. However, the possibility of the optimization of the AEM training parameters by using physiological effects was likely, thus the optimization objective started to be formulated. Some previous research suggests that a model-based optimization of advanced training is complicated for real-time environments as a consequence of the high level of v complexity, computational cost, and especially the many unidentifiable parameters. Moreover, a model-based method differs from person to person and it would require periodic updates based on physical and psychological variations in the user. Consequently, we aimed to develop a model-free optimization framework based on the use of Extremum Seeking Control (ESC). ESC is a non-model based controller for real-time optimization which its main advantage over similar controllers is its ability to deal with unknown plants. This framework uses a physiological effect of training as bio-feedback. Three different frameworks were performed for single-variable and multi-variable optimization of trajectory and impedance parameters. Based on the framework, the objective is achieved by seeking the optimal trajectory and/or impedance parameters associated with the orientation of the ellipsoidal path to be tracked by the user and the stiffness property of the resistance by using weighted measures of muscle activations

Study of Stability of Time-Domain Features for Electromyographic Pattern Recognition

Background: Significant progress has been made towards the clinical application of human-machine interfaces (HMIs) based on electromyographic (EMG) pattern recognition for various rehabilitation purposes. Making this technology practical and available to patients with motor deficits requires overcoming real-world challenges, such as physical and physiological changes, that result in variations in EMG signals and systems that are unreliable for long-term use. In this study, we aimed to address these challenges by (1) investigating the stability of time-domain EMG features during changes in the EMG signals and (2) identifying the feature sets that would provide the most robust EMG pattern recognition. Methods: Variations in EMG signals were introduced during physical experiments. We identified three disturbances that commonly affect EMG signals: EMG electrode location shift, variation in muscle contraction effort, and muscle fatigue. The impact of these disturbances on individual features and combined feature sets was quantified by changes in classification performance. The robustness of feature sets was evaluated by a stability index developed in this study. Results: Muscle fatigue had the smallest effect on the studied EMG features, while electrode location shift and varying effort level significantly reduced the classification accuracy for most of the features. Under these disturbances, the most stable EMG feature set with combination of four features produced at least 16.0% higher classification accuracy than the least stable set. EMG autoregression coefficients and cepstrum coefficients showed the most robust classification performance of all studied time-domain features. Conclusions: Selecting appropriate EMG feature combinations can overcome the impact of the studied disturbances on EMG pattern classification to a certain extent; however, this simple solution is still inadequate. Stabilizing electrode contact locations and developing effective classifier training strategies are suggested to further improve the robustness of HMIs based on EMG pattern recognition