Real-time haptic modeling and simulation for prosthetic insertion

In this work a surgical simulator is produced which enables a training otologist to conduct a virtual, real-time prosthetic insertion. The simulator provides the Ear, Nose and Throat surgeon with real-time visual and haptic responses during virtual cochlear implantation into a 3D model of the human Scala Tympani (ST). The parametric model is derived from measured data as published in the literature and accounts for human morphological variance, such as differences in cochlear shape, enabling patient-specific pre- operative assessment. Haptic modeling techniques use real physical data and insertion force measurements, to develop a force model which mimics the physical behavior of an implant as it collides with the ST walls during an insertion. Output force profiles are acquired from the insertion studies conducted in the work, to validate the haptic model. The simulator provides the user with real-time, quantitative insertion force information and associated electrode position as user inserts the virtual implant into the ST model. The information provided by this study may also be of use to implant manufacturers for design enhancements as well as for training specialists in optimal force administration, using the simulator. The paper reports on the methods for anatomical modeling and haptic algorithm development, with focus on simulator design, development, optimization and validation. The techniques may be transferrable to other medical applications that involve prosthetic device insertions where user vision is obstructed

A Biomimetic Approach to Controlling Restorative Robotics

Movement is the only way a person can interact with the world around them. When trauma to the neuromuscular systems disrupts the control of movement, quality of life suffers. To restore limb functionality, active robotic interventions and/or rehabilitation are required. Unfortunately, the primary obstacle in a person’s recovery is the limited robustness of the human-machine interfaces. Current systems rely on control approaches that rely on the person to learn how the system works instead of the system being more intuitive and working with the person naturally. My research goal is to design intuitive control mechanisms based on biological processes termed the biomimetic approach. I have applied this control scheme to problems with restorative robotics focused on the upper and lower limb control. Operating an advanced active prosthetic hand is a two-pronged problem of actuating a high-dimensional mechanism and controlling it with an intuitive interface. Our approach attempts to solve these problems by going from muscle activity, electromyography (EMG), to limb kinematics calculated through dynamic simulation of a musculoskeletal model. This control is more intuitive to the user because they attempt to move their hand naturally, and the prosthetic hand performs that movement. The key to this approach was validating simulated muscle paths using both experimental measurements and anatomical constraints where data is missing. After the validation, simulated muscle paths and forces are used to decipher the intended movement. After we have calculated the intended movement, we can move a prosthetic hand to match. This approach required minimal training to give an amputee the ability to control prosthetic hand movements, such as grasping. A more intuitive controller has the potential to improve how people interact and use their prosthetic hands. Similarly, the rehabilitation of the locomotor system in people with damaged motor pathways or missing limbs require appropriate interventions. The problem of decoding human motor intent in a treadmill walking task can be solved with a biomimetic approach. Estimated limb speed is essential for this approach according to the theoretical input-output computation performed by spinal central pattern generators (CPGs), which represents neural circuitry responsible for autonomous control of stepping. The system used the locomotor phases, swing and stance, to estimate leg speeds and enable self-paced walking as well as steering in virtual reality with congruent visual flow. The unique advantage of this system over the previous state-of-art is the independent leg speed control, which is required for multidirectional movement in VR. This system has the potential to contribute to VR gait rehab techniques. Creating biologically-inspired controllers has the potential to improve restorative robotics and allow people a better opportunity to recover lost functionality post-injury. Movement is the only way a person can interact with the world around them. When trauma to the neuromuscular systems disrupts the control of movement, quality of life suffers. To restore limb functionality, active robotic interventions and/or rehabilitation are required. Unfortunately, the primary obstacle in a person’s recovery is the limited robustness of the human-machine interfaces. Current systems rely on control approaches that rely on the person to learn how the system works instead of the system being more intuitive and working with the person naturally. My research goal is to design intuitive control mechanisms based on biological processes termed the biomimetic approach. I have applied this control scheme to problems with restorative robotics focused on the upper and lower limb control.Operating an advanced active prosthetic hand is a two-pronged problem of actuating a high-dimensional mechanism and controlling it with an intuitive interface. Our approach attempts to solve these problems by going from muscle activity, electromyography (EMG), to limb kinematics calculated through dynamic simulation of a musculoskeletal model. This control is more intuitive to the user because they attempt to move their hand naturally, and the prosthetic hand performs that movement. The key to this approach was validating simulated muscle paths using both experimental measurements and anatomical constraints where data is missing. After the validation, simulated muscle paths and forces are used to decipher the intended movement. After we have calculated the intended movement, we can move a prosthetic hand to match. This approach required minimal training to give an amputee the ability to control prosthetic hand movements, such as grasping. A more intuitive controller has the potential to improve how people interact and use their prosthetic hands.Similarly, the rehabilitation of the locomotor system in people with damaged motor pathways or missing limbs require appropriate interventions. The problem of decoding human motor intent in a treadmill walking task can be solved with a biomimetic approach. Estimated limb speed is essential for this approach according to the theoretical input-output computation performed by spinal central pattern generators (CPGs), which represents neural circuitry responsible for autonomous control of stepping. The system used the locomotor phases, swing and stance, to estimate leg speeds and enable self-paced walking as well as steering in virtual reality with congruent visual flow. The unique advantage of this system over the previous state-of-art is the independent leg speed control, which is required for multidirectional movement in VR. This system has the potential to contribute to VR gait rehab techniques.Creating biologically-inspired controllers has the potential to improve restorative robotics and allow people a better opportunity to recover lost functionality post-injury

A Generalized Method for Predictive Simulation-Based Lower Limb Prosthesis Design

Lower limb prostheses are designed to replace the functions and form of the missing biological anatomy. These functions are hypothesized to improve user outcome measures which are negatively affected by receiving an amputation – such as metabolic cost of transport, preferred walking speed, and perceived discomfort during walking. However, the effect of these design functions on the targeted outcome measures is highly variable, suggesting that these relationships are not fully understood. Biomechanics simulation and modeling tools are increasingly capable of analyzing the effects of a design on the resulting user gait. In this work, prothesis-aided gait is optimized in simulation to reduce both muscle effort and peak loads on the residual limb using a generalized prosthesis model. Compared to a traditional revolute powered ankle joint model, a two degree-of freedom generalized model reduced muscle activations by 50% and peak loads by 15%. Simulated prosthesis behaviors corresponding to the optimal gait patterns were translated into a two degree-of-freedom ankle-foot prosthesis design with powered bidirectional linear translation and plantarflexion. The prototype is capable of delivering up to 171 N-m of plantarflexion torque and 499 N of translation force, with 15° dorsi-/35° plantarflexion and 10 cm translation range of motion. The mass and height of the ankle-foot are 2.29 kg and 19.5 cm, respectively. The mass of the entire system including the wearable offboard system is 8.58 kg. This platform is designed to emulate the behavior of the simulated prosthesis, as well as be configurable to emulate alternate behaviors obtained from simulations with different optimization objectives. The prototype is controlled to replicate simulated walking patterns using a high level finite state controller, mid-level stiffness controller, and low level load controller. Closed loop load control has bandwidth of 15 Hz in translation and 7.2 Hz in flexion. Load tracking during walking with a single able-bodied human subject ranges from 93 to 159 N in translation and 4.6 to 21.3 N-m in flexion. The contribution of this work is to provide a framework for predictive simulation-based prosthesis design, evidence of its practical implementation, and the experimental tools to validate future predictive simulation studies

Determination Of Optimal Counter-Mass Location In Active Prostheses For Transfemoral Amputees To Replicate Normal Swing

Transfemoral amputees suffer the loss of the knee and ankle joints, as well as partial or complete loss of many of the lower extremity muscle groups involved in ambulation. Recent advances in lower limb prostheses have involved the design of active, powered prosthetic knee and ankle-foot components capable of generating knee and ankle torques similar to that of normal gait. The associated onboard motors, conditioning/processing, and battery units of these active components result in increased mass of the respective prosthesis. While not an issue during stance, this increased mass of the prosthesis affects swing. The goal of this study is to develop and validate mathematical models of the transfemoral residual limb and prosthesis, expand these models to include an active ankle-foot, and investigate counter-mass magnitude(s) and location(s) via model optimization that might improve kinematic symmetry during swing. Single- (thigh only, shank only) and multi-segment (combined thigh and shank) optimization of counter-mass magnitudes and locations indicated that a 2.0 kg counter-mass added 8 cm distal and 10 cm posterior to the distal end of knee unit within the shank segment approximated knee kinematics of able-bodied subjects. This location, however, induced artificial hip torques that reduced hip flexion during swing. While such a counter-mass location and magnitude demonstrated theoretical potential, this location is not clinically realistic; mass can only be added within the prosthesis, distal to the residual limb. Clinically realistic counter-masses must also keep the total prosthetic mass to less than 5 kg; greater mass requires supplemental prosthetic suspension, would likely increase energy expenditure during ambulation, and contribute to increased likelihood of fatigue even with active prosthetic components. The ability to simulate the effects of active prosthetic components inclusive of varying placement of battery and signal conditioning units may advance the design of active prostheses that will minimize kinematic asymmetry and result in greater patient acceptance

Neuromuscular Reflex Control for Prostheses and Exoskeletons

Recent powered lower-limb prosthetic and orthotic (P/O) devices aim to restore legged mobility for persons with an amputation or spinal cord injury. Though various control strategies have been proposed for these devices, specifically finite-state impedance controllers, natural gait mechanics are not usually achieved. The goal of this project was to invent a biologically-inspired controller for powered P/O devices. We hypothesize that a more muscle-like actuation system, including spinal reflexes and vestibular feedback, can achieve able-bodied walking and also respond to outside perturbations. The outputs of the Virtual Muscle Reflex (VMR) controller are joint torque commands, sent to the electric motors of a P/O device. We identified the controller parameters through optimizations using human experimental data of perturbed walking, in which we minimized the error between the torque produced by our controller and the standard torque trajectories observed in the able-bodied experiments. In simulations, we then compare the VMR controller to a four-phase impedance controller. For both controllers the coefficient of determination R^2 and root-mean-square (RMS) error were calculated as a function of the gait cycle. When simulating the hip, knee, and ankle joints, the RMS error and R^2 across all joints and all trials is 15.65 Nm and 0.28 for the impedance controller, respectively, and for the VMR controller, these values are 15.15 Nm and 0.29, respectively. With similar performance, it was concluded that the VMR controller can reproduce characteristics of human walking in response to perturbations as effectively as an impedance controller. We then implemented the VMR controller on the Parker Hannifin powered exoskeleton and performed standard isokinetic and isometric knee rehabilitation exercises to observe the behavior of the virtual muscle model. In the isometric results, RMS error between the measured and commanded extension and flexion torques are 3.28 Nm and 1.25 Nm, respectively. In the isokinetic trials, we receive RMS error between the measured and commanded extension and flexion torques of 0.73 Nm and 0.24 Nm. Since the onboard virtual muscles demonstrate similar muscle force-length and force-velocity relationships observed in humans, we conclude the model is capable of the same stabilizing capabilities as observed in an impedance controller

Neuromuscular Reflex Control for Prostheses and Exoskeletons

Recent powered lower-limb prosthetic and orthotic (P/O) devices aim to restore legged mobility for persons with an amputation or spinal cord injury. Though various control strategies have been proposed for these devices, specifically finite-state impedance controllers, natural gait mechanics are not usually achieved. The goal of this project was to invent a biologically-inspired controller for powered P/O devices. We hypothesize that a more muscle-like actuation system, including spinal reflexes and vestibular feedback, can achieve able-bodied walking and also respond to outside perturbations. The outputs of the Virtual Muscle Reflex (VMR) controller are joint torque commands, sent to the electric motors of a P/O device. We identified the controller parameters through optimizations using human experimental data of perturbed walking, in which we minimized the error between the torque produced by our controller and the standard torque trajectories observed in the able-bodied experiments. In simulations, we then compare the VMR controller to a four-phase impedance controller. For both controllers the coefficient of determination R^2 and root-mean-square (RMS) error were calculated as a function of the gait cycle. When simulating the hip, knee, and ankle joints, the RMS error and R^2 across all joints and all trials is 15.65 Nm and 0.28 for the impedance controller, respectively, and for the VMR controller, these values are 15.15 Nm and 0.29, respectively. With similar performance, it was concluded that the VMR controller can reproduce characteristics of human walking in response to perturbations as effectively as an impedance controller. We then implemented the VMR controller on the Parker Hannifin powered exoskeleton and performed standard isokinetic and isometric knee rehabilitation exercises to observe the behavior of the virtual muscle model. In the isometric results, RMS error between the measured and commanded extension and flexion torques are 3.28 Nm and 1.25 Nm, respectively. In the isokinetic trials, we receive RMS error between the measured and commanded extension and flexion torques of 0.73 Nm and 0.24 Nm. Since the onboard virtual muscles demonstrate similar muscle force-length and force-velocity relationships observed in humans, we conclude the model is capable of the same stabilizing capabilities as observed in an impedance controller

PREDICTIVE SIMULATION OF HUMAN MOVEMENT AND APPLICATIONS TO ASSISTIVE DEVICE DESIGN AND CONTROL

Predictive simulation based on dynamic optimization using musculoskeletal models is a powerful approach for studying biomechanics of human gait. Predictive simulation can be used for a variety of applications from designing assistive devices to testing theories of motor controls. However, one of the challenges in formulating the predictive dynamic optimization problem is that the cost function, which represents the underlying goal of the walking task (e.g., minimal energy consumption) is generally unknown and is assumed a priori. While different studies used different cost functions, the qualities of the gaits with those cost functions were often not provided. Therefore, this dissertation evaluates and examines different cost function forms for dynamic simulation of human walking. The problem of the walking cost function determination was cast as a bilevel optimization, which was solved using a nested evolutionary approach. The results showed cost functions based on a weighted combination of muscle-based performance criteria (e.g., metabolic cost, muscle fatigue), gait smoothness, and gait stability led to better walking solutions compared to any cost functions only based on muscle performance criteria. Further evaluations of the walking cost functions were done in the simulation cases of human walking augmented with assistive devices such as prosthesis and exoskeleton. The simulations of augmented walking were comparable to the experimental results, which suggests the potential of using the simulation approach to address problems of finding assistive device design and control

Neuromuscular Reflex Control for Prostheses and Exoskeletons

Recent powered lower-limb prosthetic and orthotic (P/O) devices aim to restore legged mobility for persons with an amputation or spinal cord injury. Though various control strategies have been proposed for these devices, specifically finite-state impedance controllers, natural gait mechanics are not usually achieved. The goal of this project was to invent a biologically-inspired controller for powered P/O devices. We hypothesize that a more muscle-like actuation system, including spinal reflexes and vestibular feedback, can achieve able-bodied walking and also respond to outside perturbations. The outputs of the Virtual Muscle Reflex (VMR) controller are joint torque commands, sent to the electric motors of a P/O device. We identified the controller parameters through optimizations using human experimental data of perturbed walking, in which we minimized the error between the torque produced by our controller and the standard torque trajectories observed in the able-bodied experiments. In simulations, we then compare the VMR controller to a four-phase impedance controller. For both controllers the coefficient of determination R^2 and root-mean-square (RMS) error were calculated as a function of the gait cycle. When simulating the hip, knee, and ankle joints, the RMS error and R^2 across all joints and all trials is 15.65 Nm and 0.28 for the impedance controller, respectively, and for the VMR controller, these values are 15.15 Nm and 0.29, respectively. With similar performance, it was concluded that the VMR controller can reproduce characteristics of human walking in response to perturbations as effectively as an impedance controller. We then implemented the VMR controller on the Parker Hannifin powered exoskeleton and performed standard isokinetic and isometric knee rehabilitation exercises to observe the behavior of the virtual muscle model. In the isometric results, RMS error between the measured and commanded extension and flexion torques are 3.28 Nm and 1.25 Nm, respectively. In the isokinetic trials, we receive RMS error between the measured and commanded extension and flexion torques of 0.73 Nm and 0.24 Nm. Since the onboard virtual muscles demonstrate similar muscle force-length and force-velocity relationships observed in humans, we conclude the model is capable of the same stabilizing capabilities as observed in an impedance controller

Feedback Control of an Exoskeleton for Paraplegics: Toward Robustly Stable Hands-free Dynamic Walking

This manuscript presents control of a high-DOF fully actuated lower-limb exoskeleton for paraplegic individuals. The key novelty is the ability for the user to walk without the use of crutches or other external means of stabilization. We harness the power of modern optimization techniques and supervised machine learning to develop a smooth feedback control policy that provides robust velocity regulation and perturbation rejection. Preliminary evaluation of the stability and robustness of the proposed approach is demonstrated through the Gazebo simulation environment. In addition, preliminary experimental results with (complete) paraplegic individuals are included for the previous version of the controller.Comment: Submitted to IEEE Control System Magazine. This version addresses reviewers' concerns about the robustness of the algorithm and the motivation for using such exoskeleton