Control Design and Implementation of an Active Transtibial Prosthesis

Prior work at Marquette University developed the Marquette Prosthesis, an active transtibial prosthesis that utilized a torsional spring and a four-bar mechanism. The controls for the Marquette Prosthesis implemented a finite state control algorithm to determine the state of gait of the amputee along with two lower level controllers, a PI moment controller to control the moment during stance and a PID position controller to control the position during stance. The Marquette Prosthesis was successful in mimicking the gait profile presented by Winter. However, after completing human subject testing, the Marquette Prosthesis was insufficient in trying to match the gait profile of those who varied from this textbook stride. Active transtibial prostheses typically apply finite state control algorithms that struggle with cadence and gait variability of the amputee. Recent work in artificial neural networks (ANN) have shown the possibility to predict the user\u27s intent which can be used as an input signal in an improved controller. The Marquette Prosthesis II was developed that uses a stiffness controller to control the relationship between the position and torque of the ankle. A model of the improved Marquette Prosthesis II was developed in Simulink to ensure that the stiffness controller was robust enough and that this type of control was possible with the limitations of the Marquette Prosthesis, i.e., the link lengths, torsional spring and motor. The mechanical system of the Marquette Prosthesis was then changed such that the spring was in series between the motor and four-bar mechanism to establish a relationship between the motor position, torque of the spring and four-bar mechanism. The control hardware was selected and the stiffness controller was implemented on the Marquette Prosthesis II. The Marquette Prosthesis II control algorithm was tested and validated to show that this approach is feasible

Energy Regeneration and Environment Sensing for Robotic Leg Prostheses and Exoskeletons

Robotic leg prostheses and exoskeletons can provide powered locomotor assistance to older adults and/or persons with physical disabilities. However, limitations in automated control and energy-efficient actuation have impeded their transition from research laboratories to real-world environments. With regards to control, the current automated locomotion mode recognition systems being developed rely on mechanical, inertial, and/or neuromuscular sensors, which inherently have limited prediction horizons (i.e., analogous to walking blindfolded). Inspired by the human vision-locomotor control system, here a multi-generation environment sensing and classification system powered by computer vision and deep learning was developed to predict the oncoming walking environments prior to physical interaction, therein allowing for more accurate and robust high-level control decisions. To support this initiative, the “ExoNet” database was developed – the largest and most diverse open-source dataset of wearable camera images of indoor and outdoor real-world walking environments, which were annotated using a novel hierarchical labelling architecture. Over a dozen state-of-the-art deep convolutional neural networks were trained and tested on ExoNet for large-scale image classification and automatic feature engineering. The benchmarked CNN architectures and their environment classification predictions were then quantitatively evaluated and compared using an operational metric called “NetScore”, which balances the classification accuracy with the architectural and computational complexities (i.e., important for onboard real-time inference with mobile computing devices). Of the benchmarked CNN architectures, the EfficientNetB0 network achieved the highest test accuracy; VGG16 the fastest inference time; and MobileNetV2 the best NetScore. These comparative results can inform the optimal architecture design or selection depending on the desired performance of an environment classification system. With regards to energetics, backdriveable actuators with energy regeneration can improve the energy efficiency and extend the battery-powered operating durations by converting some of the otherwise dissipated energy during negative mechanical work into electrical energy. However, the evaluation and control of these regenerative actuators has focused on steady-state level-ground walking. To encompass real-world community mobility more broadly, here an energy regeneration system, featuring mathematical and computational models of human and wearable robotic systems, was developed to simulate energy regeneration and storage during other locomotor activities of daily living, specifically stand-to-sit movements. Parameter identification and inverse dynamic simulations of subject-specific optimized biomechanical models were used to calculate the negative joint mechanical work and power while sitting down (i.e., the mechanical energy theoretically available for electrical energy regeneration). These joint mechanical energetics were then used to simulate a robotic exoskeleton being backdriven and regenerating energy. An empirical characterization of an exoskeleton was carried out using a joint dynamometer system and an electromechanical motor model to calculate the actuator efficiency and to simulate energy regeneration and storage with the exoskeleton parameters. The performance calculations showed that regenerating electrical energy during stand-to-sit movements provide small improvements in energy efficiency and battery-powered operating durations. In summary, this research involved the development and evaluation of environment classification and energy regeneration systems to improve the automated control and energy-efficient actuation of next-generation robotic leg prostheses and exoskeletons for real-world locomotor assistance

Novel Bidirectional Body - Machine Interface to Control Upper Limb Prosthesis

Objective. The journey of a bionic prosthetic user is characterized by the opportunities and limitations involved in adopting a device (the prosthesis) that should enable activities of daily living (ADL). Within this context, experiencing a bionic hand as a functional (and, possibly, embodied) limb constitutes the premise for mitigating the risk of its abandonment through the continuous use of the device. To achieve such a result, different aspects must be considered for making the artificial limb an effective support for carrying out ADLs. Among them, intuitive and robust control is fundamental to improving amputees’ quality of life using upper limb prostheses. Still, as artificial proprioception is essential to perceive the prosthesis movement without constant visual attention, a good control framework may not be enough to restore practical functionality to the limb. To overcome this, bidirectional communication between the user and the prosthesis has been recently introduced and is a requirement of utmost importance in developing prosthetic hands. Indeed, closing the control loop between the user and a prosthesis by providing artificial sensory feedback is a fundamental step towards the complete restoration of the lost sensory-motor functions. Within my PhD work, I proposed the development of a more controllable and sensitive human-like hand prosthesis, i.e., the Hannes prosthetic hand, to improve its usability and effectiveness. Approach. To achieve the objectives of this thesis work, I developed a modular and scalable software and firmware architecture to control the Hannes prosthetic multi-Degree of Freedom (DoF) system and to fit all users’ needs (hand aperture, wrist rotation, and wrist flexion in different combinations). On top of this, I developed several Pattern Recognition (PR) algorithms to translate electromyographic (EMG) activity into complex movements. However, stability and repeatability were still unmet requirements in multi-DoF upper limb systems; hence, I started by investigating different strategies to produce a more robust control. To do this, EMG signals were collected from trans-radial amputees using an array of up to six sensors placed over the skin. Secondly, I developed a vibrotactile system to implement haptic feedback to restore proprioception and create a bidirectional connection between the user and the prosthesis. Similarly, I implemented an object stiffness detection to restore tactile sensation able to connect the user with the external word. This closed-loop control between EMG and vibration feedback is essential to implementing a Bidirectional Body - Machine Interface to impact amputees’ daily life strongly. For each of these three activities: (i) implementation of robust pattern recognition control algorithms, (ii) restoration of proprioception, and (iii) restoration of the feeling of the grasped object's stiffness, I performed a study where data from healthy subjects and amputees was collected, in order to demonstrate the efficacy and usability of my implementations. In each study, I evaluated both the algorithms and the subjects’ ability to use the prosthesis by means of the F1Score parameter (offline) and the Target Achievement Control test-TAC (online). With this test, I analyzed the error rate, path efficiency, and time efficiency in completing different tasks. Main results. Among the several tested methods for Pattern Recognition, the Non-Linear Logistic Regression (NLR) resulted to be the best algorithm in terms of F1Score (99%, robustness), whereas the minimum number of electrodes needed for its functioning was determined to be 4 in the conducted offline analyses. Further, I demonstrated that its low computational burden allowed its implementation and integration on a microcontroller running at a sampling frequency of 300Hz (efficiency). Finally, the online implementation allowed the subject to simultaneously control the Hannes prosthesis DoFs, in a bioinspired and human-like way. In addition, I performed further tests with the same NLR-based control by endowing it with closed-loop proprioceptive feedback. In this scenario, the results achieved during the TAC test obtained an error rate of 15% and a path efficiency of 60% in experiments where no sources of information were available (no visual and no audio feedback). Such results demonstrated an improvement in the controllability of the system with an impact on user experience. Significance. The obtained results confirmed the hypothesis of improving robustness and efficiency of a prosthetic control thanks to of the implemented closed-loop approach. The bidirectional communication between the user and the prosthesis is capable to restore the loss of sensory functionality, with promising implications on direct translation in the clinical practice

Towards Natural Control of Artificial Limbs

The use of implantable electrodes has been long thought as the solution for a more natural control of artificial limbs, as these offer access to long-term stable and physiologically appropriate sources of control, as well as the possibility to elicit appropriate sensory feedback via neurostimulation. Although these ideas have been explored since the 1960’s, the lack of a long-term stable human-machine interface has prevented the utilization of even the simplest implanted electrodes in clinically viable limb prostheses.In this thesis, a novel human-machine interface for bidirectional communication between implanted electrodes and the artificial limb was developed and clinically implemented. The long-term stability was achieved via osseointegration, which has been shown to provide stable skeletal attachment. By enhancing this technology as a communication gateway, the longest clinical implementation of prosthetic control sourced by implanted electrodes has been achieved, as well as the first in modern times. The first recipient has used it uninterruptedly in daily and professional activities for over one year. Prosthetic control was found to improve in resolution while requiring less muscular effort, as well as to be resilient to motion artifacts, limb position, and environmental conditions.In order to support this work, the literature was reviewed in search of reliable and safe neuromuscular electrodes that could be immediately used in humans. Additional work was conducted to improve the signal-to-noise ratio and increase the amount of information retrievable from extraneural recordings. Different signal processing and pattern recognition algorithms were investigated and further developed towards real-time and simultaneous prediction of limb movements. These algorithms were used to demonstrate that higher functionality could be restored by intuitive control of distal joints, and that such control remains viable over time when using epimysial electrodes. Lastly, the long-term viability of direct nerve stimulation to produce intuitive sensory feedback was also demonstrated.The possibility to permanently and reliably access implanted electrodes, thus making them viable for prosthetic control, is potentially the main contribution of this work. Furthermore, the opportunity to chronically record and stimulate the neuromuscular system offers new venues for the prediction of complex limb motions and increased understanding of somatosensory perception. Therefore, the technology developed here, combining stable attachment with permanent and reliable human-machine communication, is considered by the author as a critical step towards more functional artificial limbs

Restoring Fine Motor Skills through Neural Interface Technology.

Loss of motor function in the upper-limb, whether through paralysis or through loss of the limb itself, is a profound disability which affects a large population worldwide. Lifelike, fully-articulated prosthetic hands exist and are commercially available; however, there is currently no satisfactory method of controlling all of the available degrees of freedom. In order to generate better control signals for this technology, and help restore normal movement, it is necessary to interface directly with the nervous system. This thesis is intended to address several of the limitations of current neural interfaces and enable the long-term extraction of control signals for fine movements of the hand and fingers. The first study addresses the problems of low signal amplitudes and short implant lifetimes in peripheral nerve interfaces. In two rhesus macaques, we demonstrate the successful implantation of regenerative peripheral nerve interfaces (RPNI), which allowed us to record high amplitude, functionally-selective signals from peripheral nerves up to 20 months post-implantation. These signals could be accurately decoded into intended movement, and used to enable monkeys to control a virtual hand prosthesis. The second study presents a novel experimental paradigm for intracortical neural interfaces, which enables detailed investigation of fine motor information contained in primary motor cortex. We used this paradigm to demonstrate accurate decoding of continuous fingertip position and enable a monkey to control a virtual hand in closed-loop. This is the first demonstration of volitional control of fine motor skill enabled by a cortical neural interface. The final study presents the design and testing of a wireless implantable neural recording system. By extracting signal power in a single, configurable frequency band onboard the device, this system achieves low power consumption while maintaining decode performance, and is applicable to cortical, peripheral, and myoelectric signals. Taken together, these results represent a significant step towards clinical reality for neural interfaces, and towards restoration of full and dexterous movement for people with severe disabilities.PhDBiomedical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/120648/1/irwinz_1.pd

Powered Transtibial Prosthetic Device Control System Design, Implementation and Testing

A powered lower limb prosthesis, which consists of a four bar mechanism, a torsional spring and a brushed DC motor, was previously designed and fabricated. To regulate the motor power input, a two level controller was proposed and built. The control algorithm includes a higher level finite state controller and lower level PID controllers. To implement the control system, a digital signal processor (DSP) control board and MATLAB Simulink were used to realize the higher level control and a DC motor controller was used to realize the lower level PID control. Sensors were selected to provide the required feedback. The entire control system was implemented on a convenient to carry backpack. Amputee subject testing was performed to obtain some experimental verification of the design. The results showed that the control system performed consistently with the designed control algorithm and did assist in the amputee’s walking. Compared to a currently available powered prosthesis, this control is simple in structure and able to mimic the nonlinear behavior of the ankle closely

Invasive intraneural interfaces: foreign body reaction issues

Intraneural interfaces are stimulation/registration devices designed to couple the peripheral nervous system (PNS) with the environment. Over the last years, their use has increased in a wide range of applications, such as the control of a new generation of neural-interfaced prostheses. At present, the success of this technology is limited by an electrical impedance increase, due to an inflammatory response called foreign body reaction (FBR), which leads to the formation of a fibrotic tissue around the interface, eventually causing an inefficient transduction of the electrical signal. Based on recent developments in biomaterials and inflammatory/fibrotic pathologies, we explore and select the biological solutions that might be adopted in the neural interfaces FBR context: modifications of the interface surface, such as organic and synthetic coatings; the use of specific drugs or molecular biology tools to target the microenvironment around the interface; the development of bio-engineered-scaffold to reduce immune response and promote interface-tissue integration. By linking what we believe are the major crucial steps of the FBR process with related solutions, we point out the main issues that future research has to focus on: biocompatibility without losing signal conduction properties, good reproducible in vitro/in vivo models, drugs exhaustion and undesired side effects. The underlined pros and cons of proposed solutions show clearly the importance of a better understanding of all the molecular and cellular pathways involved and the need of a multi-target action based on a bio-engineered combination approach

Feasibility study of a permanently implanted prosthetic hand

The feasibility of a permanently implanted prosthetic hand was evaluated from both an internal biocompatibility and exterior mechanics point of view. A literature review of the issues involved in permanent implantation of a percutanious device was performed in the areas of bone interaction and fixation and neural interface control. A theoretical implant was designed for a 90th percentile male, using an HA-G-Ti composite material to provide a permanent base to which the hand could attach. Using a radial implant length of 1.87 inches and an ulna implant length of 1.32 inches, the simulated implant could withstand a push out force of 10,260 pounds. Using nerve guidance channels and micro-electrode arrays, a Regenerative Neural Interface was postulated to control the implant. The use of Laminin-5 was suggested as a method of preventing the lack of wound closure observed in percutaneous devices. The exterior portion of a permanent artificial hand was analyzed by the construction of a robotic hand optimized for weight, size, grip force and wrist torque, power consumption and range of motion. Using a novel dual drive system, each finger was equipped with both joint position servos as well as a tendon. Fine grip shape was formed using the servos, while the tendon was pulled taunt when grasping an object. Control of the prosthetic was performed using a distributed network of micro-controllers. Each finger\u27s behavior was governed by a master/slave system where input from a control glove was processed by a master controller with joint servo and tendon instructions passed to lower-level controllers for management of hand actuators. The final weight of the prototype was 3.85 pounds and was approximately 25% larger than the 90th percentile male hand it was based on. Grip force was between 1.25 and 2 pounds per finger, depending on amount of finger flexion with a wrist lifting capacity of 1.2 pounds at the center of the palm. The device had an average current draw of 3 amps in both normal operation and tight grasping. Range of motion was similar to that of the human model. Overall feasibility is examined and factors involved in industrial implementation are also discussed

On improving control and efficiency of a portable pneumatically powered ankle-foot orthosis

Ankle foot orthoses (AFOs) are widely used as assistive and/or rehabilitation devices to correct gait of people with lower leg neuromuscular dysfunction and muscle weakness. An AFO is an external device worn on the lower leg and foot that provides mechanical assistance at the ankle joint. Active AFOs are powered devices that provide assistive torque at the ankle joint. We have previously developed the Portable Powered Ankle-Foot Orthosis (PPAFO), which uses pneumatic power via compressed CO2 to provide untethered ankle torque assistance. My dissertation work focused on the development of control strategies for the PPAFO that are robust, applicable to different gait patterns, functional in different gait modes, and energy efficient. Three studies addressing these topics are presented in this dissertation: (1) estimation of the system state during the gait cycle for actuation control; (2) gait mode recognition and control (e.g., stair and ramp descent/ascent); and (3) system analysis and improvement of pneumatic energy efficiency. Study 1 presents the work on estimating the gait state for powered AFO control. The proposed scheme is a state estimator that reliably detects gait events while using only a limited array of sensor data (ankle angle and contact forces at the toe and heel). Our approach uses cross-correlation between a window of past measurements and a learned model to estimate the configuration of the human walker, and detects gait events based on this estimate. The proposed state estimator was experimentally validated on five healthy subjects and with one subject that had neuromuscular impairment. The results highlight that this new approach reduced the root-mean-square error by up to 40% for the impaired subject and up to 49% for the healthy subjects compared to a simplistic direct event controller. Moreover, this approach was robust to perturbations due to changes in walking speed and control actuation. Study 2 proposed a gait mode recognition and control solution to identify a change in walking environment such as stair and ramp ascent/descent. Since portability is a key to the success of the PPAFO as a gait assist device, it is critical to recognize and control for multiple gait modes (i.e., level walking, stair ascent/descent and ramp ascent/descent). While manual mode switching is implemented on most devices, we propose an automatic gait mode recognition scheme by tracking the 3D position of the PPAFO from an inertial measurement unit (IMU). Experimental results indicate that the controller was able to identify the position, orientation and gait mode in real time, and properly control the actuation. The overall recognition success rate was over 97%. Study 3 addressed improving operational runtime by analyzing the system efficiency and proposing an energy harvesting and recycling scheme to save fuel. Through a systematic analysis, the overall system efficiency was determined by deriving both the system operational efficiency and the system component efficiency. An improved pneumatic operation utilized an accumulator to harvest and then recycle the exhaust energy from a previous actuation to power the subsequent actuation. The overall system efficiency was improved from 20.5% to 29.7%, a fuel savings of 31%. Work losses across pneumatic components and solutions to improve them were quantified and discussed. Future work including reducing delay in recognition, exploring faulty recognition, additional options for harvesting human energy, and learning control were proposed