New control strategies for neuroprosthetic systems

The availability of techniques to artificially excite paralyzed muscles opens enormous potential for restoring both upper and lower extremity movements with\ud neuroprostheses. Neuroprostheses must stimulate muscle, and control and regulate the artificial movements produced. Control methods to accomplish these tasks include feedforward (open-loop), feedback, and adaptive control. Feedforward control requires a great deal of information about the biomechanical behavior of the limb. For the upper extremity, an artificial motor program was developed to provide such movement program input to a neuroprosthesis. In lower extremity control, one group achieved their best results by attempting to meet naturally perceived gait objectives rather than to follow an exact joint angle trajectory. Adaptive feedforward control, as implemented in the cycleto-cycle controller, gave good compensation for the gradual decrease in performance observed with open-loop control. A neural network controller was able to control its system to customize stimulation parameters in order to generate a desired output trajectory in a given individual and to maintain tracking performance in the presence of muscle fatigue. The authors believe that practical FNS control systems must\ud exhibit many of these features of neurophysiological systems

Prediction and control in human neuromusculoskeletal models

Computational neuromusculoskeletal modelling enables the generation and testing of hypotheses about human movement on a large scale, in silico. Humanoid models, which increasingly aim to replicate the full complexity of the human nervous and musculoskeletal systems, are built on extensive prior knowledge, extracted from anatomical imaging, kinematic and kinetic measurement, and codified as model description. Where inverse dynamic analysis is applied, its basis is in Newton's laws of motion, and in solving for muscular redundancy it is necessary to invoke knowledge of central nervous motor strategy. This epistemological approach contrasts strongly with the models of machine learning, which are generally over-parameterised and largely data-driven. Even as spectacular performance has been delivered by the application of these models in a number of discrete domains of artificial intelligence, work towards general human-level intelligence has faltered, leading many to wonder if the data-driven approach is fundamentally limited, and spurring efforts to combine machine learning with knowledge-based modelling. Through a series of five studies, this thesis explores the combination of neuromusculoskeletal modelling with machine learning in order to enhance the core tasks of prediction and control. Several principles for the development of clinically useful artificially intelligent systems emerge: stability, computational efficiency and incorporation of prior knowledge. The first study concerns the use of neural network function approximators for the prediction of internal forces during human movement, an important task with many clinical applications, but one for which the standard tools of modelling are slow and cumbersome. By training on a large dataset of motions and their corresponding forces, state of the art performance is demonstrated, with many-fold increases in inference speed enabling the deployment of trained models for use in a real time biofeedback system. Neural networks trained in this way, to imitate some optimal controller, encode a mapping from high-level movement descriptors to actuator commands, and may thus be deployed in simulation as \textit{policies} to control the actions of humanoid models. Unfortunately, the high complexity of realistic simulation makes stable control a challenging task, beyond the capabilities of such naively trained models. The objective of the second study was to improve performance and stability of policy-based controllers for humanoid models in simulation. A novel technique was developed, borrowing from established unsupervised adversarial methods in computer vision. This technique enabled significant gains in performance relative to a neural network baseline, without the need for additional access to the optimal controller. For the third study, increases in the capabilities of these policy-based controllers were sought. Reinforcement learning is widely considered the most powerful means of optimising such policies, but it is computationally inefficient, and this inefficiency limits its clinical utility. To mitigate this problem, a novel framework, making use of domain-specific knowledge present in motion data, and in an inverse model of the biomechanical system, was developed. Training on simple desktop hardware, this framework enabled rapid initialisation of humanoid models that were able to move naturally through a 3-dimensional simulated environment, with 900-fold improvements in sample efficiency relative to a related technique based on pure reinforcement learning. After training with subject-specific anatomical parameters, and motion data, learned policies represent personalised models of motor control that may be further interrogated to test hypotheses about movement. For the fourth study, subject-specific controllers were taken and used as the substrate for transfer learning, by removing kinematic constraints and optimising with respect to the magnitude of the medial knee joint reaction force, an important biomechanical variable in osteoarthritis of the knee. Models learned new kinematic strategies for the reduction of this biomarker, which were subsequently validated by their use, in the real world, to construct subject-specific routines for real time gait retraining. Six out of eight subjects were able to reduce medial knee joint loading by pursuing the personalised kinematic targets found in simulation. Personalisation of assistive devices, such as limb prostheses, is another area of growing interest, and one for which computational frameworks promise cost-effective solutions. Reinforcement learning provides powerful techniques for this task but the expansion of the scope of optimisation, to include previously static elements of a prosthesis, is problematic for its complexity and resulting sample inefficiency. The fifth and final study demonstrates a new algorithm that leverages the methods described in the previous studies, and additional techniques for variance control, to surmount this problem, improving sample efficiency and simultaneously, through the use of prior knowledge encoded in motion data, providing a rational means of determining optimality in the prosthesis. Trained models were able to jointly optimise motor control and prosthesis design to enable improved performance in a walking task, and optimised designs were robust to both random seed and reward specification. This algorithm could be used to speed the design and production of real personalised prostheses, representing a potent realisation of the potential benefits of combined reinforcement learning and realistic neuromusculoskeletal modelling.Open Acces

Musculoskeletal Modeling and Control of the Human Upper Limb during Manual Wheelchair Propulsion: Application in Functional Electrical Stimulation Rehabilitation Therapy

Manual wheelchair users rely on their upper limbs for independence and daily activities. The high incidence of upper limb injuries can be attributed to the significant muscular demands imposed by propulsion as a repetitive movement. People with spinal cord injury are at high risk for upper limb injuries, including neuromusculoskeletal pathologies and nociceptive pain, as human upper limbs are poorly designed to facilitate chronic weight-bearing activities, such as manual wheelchair propulsion. Comprehending the underlying biomechanical mechanisms of motor control and developing appropriate rehabilitation tasks are essential to deal with the effects of poor motor control on the performance of manual wheelchair users and prevent long-term upper limb disability, which can interrupt electrical signals between the brain and muscles. Functional electrical stimulation utilizes low-intensity electrical signals to artificially generate body movements by stimulating the damaged peripheral nerves of patients with impaired motor control. Therefore, this study investigates the central nervous system strategy to control human movements, which can be used for task-specific functional electrical stimulation rehabilitation therapy. To this aim, two degrees of freedom musculoskeletal model of the upper limb, including six muscles, is developed, and an optimal controller consisting of two separate optimal parts is proposed to track the desired trajectories in the joint space and estimate the optimal muscle activations regarding physiological constraints. The simulation results are validated with electromyography datasets collected from twelve participants. This study's primary advantages are generating optimal joint torques, accurate trajectory tracking, and good similarities between estimated and measured muscle activations

Development of a hybrid robotic system based on an adaptive and associative assistance for rehabilitation of reaching movement after stroke

Stroke causes irreversible neurological damage. Depending on the location and the size of this brain injury, different body functions could result affected. One of the most common consequences is motor impairments. The level of motor impairment affectation varies between post-stroke subjects, but often, it hampers the execution of most activities of daily living. Consequently, the quality of life of the stroke population is severely decreased. The rehabilitation of the upper-limb motor functions has gained special attention in the scientific community due the poor reported prognosis of post-stroke patients for recovering normal upper-extremity function after standard rehabilitation therapy. Driven by the advance of technology and the design of new rehabilitation methods, the use of robot devices, functional electrical stimulation and brain-computer interfaces as a neuromodulation system is proposed as a novel and promising rehabilitation tools. Although the uses of these technologies present potential benefits with respect to standard rehabilitation methods, there still are some milestones to be addressed for the consolidation of these methods and techniques in clinical settings. Mentioned evidences reflect the motivation for this dissertation. This thesis presents the development and validation of a hybrid robotic system based on an adaptive and associative assistance for rehabilitation of reaching movements in post-stroke subjects. The hybrid concept refers the combined use of robotic devices with functional electrical stimulation. Adaptive feature states a tailored assistance according to the users’ motor residual capabilities, while the associative term denotes a precise pairing between the users’ motor intent and the peripheral hybrid assistance. The development of the hybrid platform comprised the following tasks: 1. The identification of the current challenges for hybrid robotic system, considering twofold perspectives: technological and clinical. The hybrid systems submitted in literature were critically reviewed for such purpose. These identified features will lead the subsequent development and method framed in this work. 2. The development and validation of a hybrid robotic system, combining a mechanical exoskeleton with functional electrical stimulation to assist the execution of functional reaching movements. Several subsystems are integrated within the hybrid platform, which interact each other to cooperatively complement the rehabilitation task. Complementary, the implementation of a controller based on functional electrical stimulation to dynamically adjust the level of assistance is addressed. The controller is conceived to tackle one of the main limitations when using electrical stimulation, i.e. the highly nonlinear and time-varying muscle response. An experimental procedure was conducted with healthy and post-stroke patients to corroborate the technical feasibility and the usability evaluation of the system. 3. The implementation of an associative strategy within the hybrid platform. Three different strategies based on electroencephalography and electromyography signals were analytically compared. The main idea is to provide a precise temporal association between the hybrid assistance delivered at the periphery (arm muscles) and the users’ own intention to move and to configure a feasible clinical setup to be use in real rehabilitation scenarios. 4. Carry out a comprehensive pilot clinical intervention considering a small cohort of patient with post-stroke patients to evaluate the different proposed concepts and assess the feasibility of using the hybrid system in rehabilitation settings. In summary, the works here presented prove the feasibility of using the hybrid robotic system as a rehabilitative tool with post-stroke subjects. Moreover, it is demonstrated the adaptive controller is able to adjust the level of assistance to achieve successful tracking movement with the affected arm. Remarkably, the accurate association in time between motor cortex activation, represented through the motor-related cortical potential measured with electroencephalography, and the supplied hybrid assistance during the execution of functional (multidegree of freedom) reaching movement facilitate distributed cortical plasticity. These results encourage the validation of the overall hybrid concept in a large clinical trial including an increased number of patients with a control group, in order to achieve more robust clinical results and confirm the presented herein.Programa Oficial de Doctorado en Ingeniería Eléctrica, Electrónica y AutomáticaPresidente: Ramón Ceres Ruiz.- Secretario: Luis Enrique Moreno Lorente.- Vocal: Antonio Olivier

Bio­-inspired approaches to the control and modelling of an anthropomimetic robot

Introducing robots into human environments requires them to handle settings designed specifically for human size and morphology, however, large, conventional humanoid robots with stiff, high powered joint actuators pose a significant danger to humans. By contrast, “anthropomimetic” robots mimic both human morphology and internal structure; skeleton, muscles, compliance and high redundancy. Although far safer, their resultant compliant structure presents a formidable challenge to conventional control. Here we review, and seek to address, characteristic control issues of this class of robot, whilst exploiting their biomimetic nature by drawing upon biological motor control research. We derive a novel learning controller for discovering effective reaching actions created through sustained activation of one or more muscle synergies, an approach which draws upon strong, recent evidence from animal and humans studies, but is almost unexplored to date in musculoskeletal robot literature. Since the best synergies for a given robot will be unknown, we derive a deliberately simple reinforcement learning approach intended to allow their emergence, in particular those patterns which aid linearization of control. We also draw upon optimal control theories to encourage the emergence of smoother movement by incorporating signal dependent noise and trial repetition. In addition, we argue the utility of developing a detailed dynamic model of a complete robot and present a stable, physics-­‐‑based model, of the anthropomimetic ECCERobot, running in real time with 55 muscles and 88 degrees of freedom. Using the model, we find that effective reaching actions can be learned which employ only two sequential motor co-­‐‑activation patterns, each controlled by just a single common driving signal. Factor analysis shows the emergent muscle co-­‐‑activations can be reconstructed to significant accuracy using weighted combinations of only 13 common fragments, labelled “candidate synergies”. Using these synergies as drivable units the same controller learns the same task both faster and better, however, other reaching tasks perform less well, proportional to dissimilarity; we therefore propose that modifications enabling emergence of a more generic set of synergies are required. Finally, we propose a continuous controller for the robot, based on model predictive control, incorporating our model as a predictive component for state estimation, delay-­‐‑ compensation and planning, including merging of the robot and sensed environment into a single model. We test the delay compensation mechanism by controlling a second copy of the model acting as a proxy for the real robot, finding that performance is significantly improved if a precise degree of compensation is applied and show how rapidly an un-­‐‑compensated controller fails as the model accuracy degrades

Haptic induced motor learning and the extension of its benefits to stroke patients

In this research, the Haptic Master robotic arm and virtual environments are used to induce motor learning in subjects with no known musculoskeletal or neurological disorders. It is found in this research that both perception and performance of the subject are increased through the haptic and visual feedback delivered through the Haptic Master. These system benefits may be extended to enhance therapies for patients with loss of motor skills due to neurological disease or brain injury. Force and visual feedback were manipulated within virtual environment scenarios to facilitate learning. In one force feedback condition, the subject is required to maneuver a sphere through a haptic maze or linear channel. In the second feedback condition, the subject\u27s movement was stopped when the sphere came in contact with the haptic walls. To resume movement, the force vector had to be redirected towards the optimal trajectory. To analyze the efficiency of the various scenarios, the area between the optimal and actual trajectories was used as a measure of learning. The results from this research demonstrated that within more complex environments one type of force feedback was more successful in facilitating motor learning. In a simpler environment, two out of three subjects experienced a higher degree of motor learning with the same type of force feedback. Learning is not enhanced with the presence of visual feedback. Also, in nearly all studied cases, the primary limitation to learning is shoulder and attention fatigue brought on by the experimentation

Hybrid walking therapy with fatigue management for spinal cord injured individuals

In paraplegic individuals with upper motor neuron lesions the descending path for signals from central nervous system to the muscles are lost or diminished. Motor neuroprosthesis based on electrical stimulation can be applied to induce restoration of motor function in paraplegic patients. Furthermore, electrical stimulation of such motor neuroprosthesis can be more efficiently managed and delivered if combined with powered exoskeletons that compensate the limited force in the stimulated muscles and bring additional support to the human body. Such hybrid overground gait therapy is likely to be more efficient to retrain the spinal cord in incomplete injuries than conventional, robotic or neuroprosthetic approaches. However, the control of bilateral joints is difficult due to the complexity, non-linearity and time-variance of the system involved. Also, the effects of muscle fatigue and spasticity in the stimulated muscles complicate the control task. Furthermore, a compliant joint actuation is required to allow for a cooperative control approach that is compatible with the assist-as-needed rehabilitation paradigm. These were direct motivations for this research. The overall aim was to generate the necessary knowledge to design a novel hybrid walking therapy with fatigue management for incomplete spinal cord injured subjects. Research activities were conducted towards the establishment of the required methods and (hardware and software) systems that required to proof the concept with a pilot clinical evaluation. Speciffically, a compressive analysis of the state of the art on hybrid exoskeletons revealed several challenges which were tackled by this dissertation. Firstly, assist-as-needed was implemented over the basis of a compliant control of the robotic exoskeleton and a closed-loop control of the neuroprosthesis. Both controllers are integrated within a hybrid-cooperative strategy that is able to balance the assistance of the robotic exoskeleton regarding muscle performance. This approach is supported on the monitoring of the leg-exoskeleton physical interaction. Thus the fatigue caused by neuromuscular stimulation was also subject of speciffic research. Experimental studies were conducted with paraplegic patients towards the establishment of an objective criteria for muscle fatigue estimation and management. The results of these studies were integrated in the hybrid-cooperative controller in order to detect and manage muscle fatigue while providing walking therapy. Secondly closed-loop control of the neuroprosthesis was addressed in this dissertation. The proposed control approach allowed to tailor the stimulation pattern regarding the speciffic residual motor function of the lower limb of the patient. In order to uncouple the closed-loop control from muscle performance monitoring, the hybrid-cooperative control approach implemented a sequential switch between closed-loop and open-loop control of the neuroprosthesis. Lastly, a comprehensive clinical evaluation protocol allowed to assess the impact of the hybrid walking therapy on the gait function of a sample of paraplegic patients. Results demonstrate that: 1) the hybrid controller adapts to patient residual function during walking, 2) the therapy is tolerated by patients, and 3) the walking function of patients was improved after participating in the study. In conclusion, the hybrid walking therapy holds potential for rehabilitate walking in motor incomplete paraplegic patients, guaranteeing further research on this topic. This dissertation is framed within two research projects: REHABOT (Ministerio de Ciencia e Innovación, grant DPI2008-06772-C03-02) and HYPER (Hybrid Neuroprosthetic and Neurorobotic Devices for Functional Compensation and Rehabilitation of Motor Disorders, grant CSD2009-00067 CONSOLIDER INGENIO 2010). Within these research projects, cutting-edge research is conducted in the eld of hybrid actuation and control for rehabilitation of motor disorders. This dissertation constitutes proof-of concept of the hybrid walking therapy for paraplegic individuals for these projects. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------En individuos parapléjicos con lesiones de la motoneurona superior, la conexión descendente para la transmisión de las señales del sistema nervioso central a los músculos se ve perdida o disminuida. Las neuroprótesis motoras basadas en la estimulación eléctrica pueden ser aplicadas para inducir la restauración de la función motora en pacientes con paraplejia. Además, la estimulación eléctrica de tales neuroprótesis motoras se puede gestionar y aplicar de manera más eficiente mediante la combinación con exoesqueletos robóticos que compensen la generación limitada de fuerza de los músculos estimulados, y proporcionen soporte adicional para el cuerpo. Dicha terapia de marcha ambulatoria puede ser probablemente más eficaz para la recuperación de las funciones de la médula espinal en lesiones incompletas que las terapias convencionales, robóticas o neuroprotesicas. Sin embargo, el control bilateral de las articulaciones es difícil debido a la complejidad, no-linealidad y la variación con el tiempo de las características del sistema en cuestión. Además, la fatiga muscular y la espasticidad de los músculos estimulados complican la tarea de control. Por otra parte, se requiere una actuación robótica modulable para permitir un enfoque de control cooperativo compatible con el paradigma de rehabilitación de asistencia bajo demanda. Todo lo anterior constituyó las motivaciones directas para esta investigación. El objetivo general fue generar el conocimiento necesario para diseñar un nuevo tratamiento híbrido de rehabilitación marcha con gestión de la fatiga para lesionados medulares incompletos. Se llevaron a cabo actividades de investigación para el establecimiento de los métodos necesarios y los sistemas (hardware y software) requeridos para probar el concepto mediante una evaluación clínica piloto. Específicamente, un análisis del estado de la técnica sobre exoesqueletos híbridos reveló varios retos que fueron abordados en esta tesis. En primer lugar, el paradigma de asistencia bajo demanda se implementó sobre la base de un control adaptable del exoesqueleto robótico y un control en lazo cerrado de la neuroprótesis. Ambos controladores están integrados dentro de una estrategia híbrida cooperativa que es capaz de equilibrar la asistencia del exoesqueleto robótico en relación con el rendimiento muscular. Este enfoque se soporta sobre la monitorización de la interacción física entre la pierna y el exoesqueleto. Por tanto, la fatiga causada por la estimulación neuromuscular también fue objeto de una investigación específica. Se realizaron estudios experimentales con pacientes parapléjicos para el establecimiento de un criterio objetivo para la detección y la gestión de la fatiga muscular. Los resultados de estos estudios fueron integrados en el controlador híbrido-cooperativo con el fin de detectar y gestionar la fatiga muscular mientras se realiza la terapia híbrida de rehabilitación de la marcha. En segundo lugar, el control en lazo cerrado de la neuroprótesis fue abordado en esta tesis. El método de control propuesto permite adaptar el patrón de estimulación en relación con la funcionalidad residual específica de la extremidad inferior del paciente. Sin embargo, con el n de desacoplar el control en lazo cerrado de la monitorización del rendimiento muscular, el enfoque de control híbrido-cooperativo incorpora una conmutación secuencial entre el control en lazo cerrado y en lazo abierto de la neuropr otesis. Por último, un protocolo de evaluación clínica global permitido evaluar el impacto de la terapia híbrida de la marcha en la función de la marcha de una muestra de pacientes parapléjicos. Los resultados demuestran que: 1) el controlador híbrido se adapta a la función residual del paciente durante la marcha, 2) la terapia es tolerada por los pacientes, y 3) la funci on de marcha del paciente mejora despu es de participar en el estudio. En conclusión, la terapia de híbrida de la marcha alberga un potencial para la rehabilitación de la marcha en pacientes parapléjicos incompletos motor, garantizando realizar investigación más profunda sobre este tema. Esta tesis se enmarca dentro de los dos proyectos de investigación: REHABOT (Ministerio de Ciencia e Innovación, referencia DPI2008-06772-C03-02) y HYPER (Hybrid Neuroprosthetic and Neurorobotic Devices for Functional Compensation and Rehabilitation of Motor Disorders, referencia CSD2009-00067 CONSOLIDER INGENIO 2010). Dentro de estos proyectos se lleva a cabo investigación de vanguardia en el campo de la actuación y el control híbrido de la combinación robot-neuroprótesis para la rehabilitación de trastornos motores. Esta tesis constituye la prueba de concepto de la terapia de híbrida de la marcha para individuos parapléjicos en estos proyectos.This dissertation is framed within two research projects: REHABOT (Ministerio de Ciencia e Innovación, grant DPI2008-06772-C03-02) and HYPER (Hybrid Neuroprosthetic and Neurorobotic Devices for Functional Compensation and Rehabilitation of Motor Disorders, grant CSD2009-00067 CONSOLIDER INGENIO 2010