Does Reinforcement Learning outperform PID in the control of FES-induced elbow flex-extension?

Functional electrical stimulation (FES) is an effective technology in post-stroke rehabilitation of the upper limbs. Because of the complexity of the system, traditional linear controllers are still far to drive accurate and natural movements. In this work, we apply reinforcement learning (RL) to design a nonlinear controller for an upper limb FES system combined with a passive exoskeleton. RL methods learn by interacting with the environment and, to efficiently use the collected data, we simulated large numbers of experience episodes through artificial neural network (ANN) models of the electrically stimulated arm muscles. The performance of the novel control solution was compared to a PID controller on five healthy subjects during planar reaching tasks. Both controllers correctly drove the arm at the target position, with a mean absolute error < 1°. The RL control significantly outperformed the PID in terms of setting time, position accuracy and smoothness. Future trials are needed to confirm these promising results

Development of a hybrid assist-as-need hand exoskeleton for stroke rehabilitation.

Stroke is one of the leading causes of disability globally and can significantly impair a patient’s ability to function on a daily basis. Through physical rehabilitative measures a patient may regain a level of functional independence. However, required therapy dosages are often not met. Rehabilitation is typically implemented through manual one-to-one assistance with a physiotherapist, which quickly becomes labour intensive and costly. Hybrid application of functional electrical stimulation (FES) and robotic support can access the physiological benefits of direct muscle activation while providing controlled and repeatable motion assistance. Furthermore, patient engagement can be heightened through the integration of a volitional intent measure, such as electromyography (EMG). Current hybrid hand-exoskeletons have demonstrated that a balanced hybrid support profile can alleviate FES intensity and motor torque requirements, whilst improving reference tracking errors. However, these support profiles remain fixed and patient fatigue is not addressed. The aim of this thesis was to develop a proof-of-concept assist-as-need hybrid exoskeleton for post-stroke hand rehabilitation, with fatigue monitoring to guide the balance of support modalities. The device required the development and integration of a constant current (CC) stimulator, stimulus-resistant EMG device, and hand-exoskeleton. The hand exoskeleton in this work was formed from a parametric Watt I linkage model that adapts to different finger sizes. Each linkage was optimised with respect to angular precision and compactness using Differential Evolution (DE). The exoskeleton’s output trajectory was shown to be sensitive to parameter variation, potentially caused by finger measurement error and shifts in coupler placement. However, in a set of cylindrical grasping trials it was observed that a range of movement strategies could be employed towards a successful grasp. As there are many possible trajectories that result in a successful grasp, it was deduced that the exoskeleton can still provide functional assistance despite its sensitivity to parameter variation. The CC stimulator developed in this work has a part cost of USD $145 and allows flexible adjustment of waveform parameters through an on-board micro-controller. The device is designed to output current up to ±30mA given a voltage compliance of ±50V. When applied across a 2kΩ load, the device exhibited a linear output transfer function, with a maximum ramp tracking error of 5 The stimulus-resistant EMG device builds on current designs by using a novel Schmitt trigger based artefact detection channel to adaptively blank stimulation artefacts without stimulator synchronisation. The design has a part cost of USD$150 and has been made open-source. The device demonstrated its ability to record EMG over its predominant energy spectrum during stimulation, through the stimulation electrodes or through separate electrodes. Pearson’s correlation coefficients greater than 0.84 were identified be- tween the normalised spectra of volitional EMG (vEMG) estimates during stimulation and of stimulation-free EMG recordings. This spectral similarity permits future research into applications such as spectral-based monitoring of fatigue and muscle coherence, posing an advantage over current same-electrode stimulation and recording systems, which can- not sample the lower end of the EMG spectrum due to elevated high-pass filter cut-off frequencies. The stimulus-resistant EMG device was used to investigate elicited EMG (eEMG)-based fatigue metrics during vEMG-controlled stimulation and hybrid support profiles. During intermittent vEMG-controlled stimulation, the eEMG peak-to-peak amplitude (PTP) index was the median frequency (MDF) had a negative correlation for all subjects with R > 0:62 during stimulation-induced wrist flexion and R > 0:55 during stimulation-induced finger flexion. During hybrid FES-robotic support trials, a 40% reduction in stimulus intensity resulted in an average 21% reduction in MDF gradient magnitudes. This reflects lower levels of fatigue during the hybrid support profile and indicates that the MDF gradient can provide useful information on the progression of muscle fatigue. A hybrid exoskeleton system was formed through the integration of the CC stimulator, stimulus-resistant EMG device, and the hand exoskeleton developed in this work. The system provided assist-as-need functional grasp assistance through stimulation and robotic components, governed by the user’s vEMG. The hybrid support profile demonstrated consistent motion assistance with lowered stimulation intensities, which in-turn lowered the subjects’ perceived levels of fatigue

TEMPORAL NEUROMUSCULAR ALTERATIONS OF THE QUADRICEPS AFTER UNILATERAL ANTERIOR CRUCIATE LIGAMENT RECONSTRUCTION

Objective: The primary aim of this research was to examine the temporal pattern of neuromuscular quadriceps deficits in both the involved and uninvolved limbs of patients assigned to the control group after anterior cruciate ligament reconstruction (ACLr), by assessing quadriceps strength, voluntary activation, and corticomotor excitability prior to surgery (baseline), three months after ACLr, and six months after ACLr. A secondary aim of this research was to determine whether quadriceps strength, voluntary activation, and/or corticomotor excitability assessed in patients prior to ACLr and/or at three months after surgery, is predictive of lower extremity postural control and/or self-reported function at six months after ACLr. Lastly, a tertiary aim of this research was to determine if a 12-week home-based neuromuscular electrical stimulation (Home-NMES) program elicits greater bilateral improvements in quadriceps strength, voluntary activation, and corticomotor excitability of patients at three and six months after ACLr compared to a 12-week standard home-exercise program (control group). Participants: Fifty patients scheduled to undergo unilateral ACLr were randomly allocated to the home-NMES group (19 Female, 6 Male; age: 18.9 ± 5.4 years; height: 170.8 ± 9.7 cm; weight: 74.6 ± 18.5 kg; 28.0±20.0 days-post-injury) or control group (14 Female, 11 Male; age: 19.4 ± 4.5 years; height: 171.1 ± 11.5 cm; weight: 70.7 ± 11.9 kg). Methods: A randomized clinical trial design was used in this study. Prior to ACLr, isometric quadriceps strength and voluntary quadriceps activation were assessed in both limbs of patients, and corticomotor excitability was assessed in the involved limb. Three days after ACLr, both groups were instructed to begin their allocated interventions. The Home-NMES group administered NMES to their involved limb’s quadriceps three sessions a day for 15 minutes, and five days a week for 12 weeks using a portable NMES device. The control group was treated according to the current standard-of-care, but they were also instructed to perform volitional isometric quadriceps contractions for the same duration and frequency as the Home-Based NMES protocol. The outcomes measures were reassessed in both groups at three and six months post-ACLr. Main Outcome Measures: Quadriceps strength and voluntary activation were assessed using maximal voluntary isometric contractions and the superimposed burst technique, respectively. Normalized peak knee extension torque and central activation ratio were used to quantify isometric quadriceps strength and activation, respectively. Corticomotor excitability was evaluated with transcranial magnetic stimulation, and quantified with active motor threshold). The Y-balance test anterior reach (YBT-A) and Knee Injury and Osteoarthritis Outcome Score (KOOS) were used to assess the patients lower extremity knee function at six months post-ACLr. Statistical Analyses: Specific Aim 1: A 2x3 (limb x time) mixed model, ANOVA with repeated measures was performed in the control group to assess differences between the involved limb and the uninvolved limb for isometric quadriceps strength, and voluntary quadriceps activation over time. A one-way mixed model, ANOVA with repeated measures was performed in the control group to assess differences in corticomotor excitability over time. Post-hoc comparisons were performed when appropriate. Specific Aim 2: Separate, mixed model, linear regression analyses were performed in the control group (involved limb) to determine the effect that the neuromuscular quadriceps outcome measures assessed at baseline and 3 months post-ACLr, had on lower extremity knee functional outcome measures assessed at 6 months post-ACLr. Specific Aim 3: A 2x2x3 (group x limb x time) mixed model, ANOVA with repeated measures was performed to assess group differences between the involved limb and the uninvolved limb in isometric quadriceps strength, and voluntary quadriceps activation over time. A 2x3 (group x time) mixed model, ANOVA with repeated measures was performed to assess group differences in corticomotor excitability over time. Post-hoc comparisons were performed when appropriate. Results: Aim 1: Patients demonstrated lower quadriceps strength on their involved limb compared to their uninvolved limb at baseline, three months post-ACLr, and six months post-ACLr. Quadriceps strength progressively decreased in the involved limb of patients from baseline to 3 months post-ACLr, baseline to 6 months post-ACLr, and increased from 3 months to 6 months post-ACLr. Quadriceps strength was also decreased in the uninvolved limb of patients from baseline to 6 months post-ACLr. ). Irrespective of when it was assessed, voluntary quadriceps activation was higher in the involved limb of patients compared to their uninvolved limb. There were no changes in corticomotor excitability of the involved limb over time. Specific Aim 2: The quadriceps strength of patients at three months post-ACLr had a significant positive effect on their 6-month YBT-A performance KOOS score. ). Neither voluntary quadriceps activation or corticomotor excitability or AMT (at baseline or 3-month post-ACLr) had a significant effect on any of the 6-month lower extremity functional outcome measures. Specific Aim 3: Irrespective of limb or when it was assessed, quadriceps strength was higher in the control group compared to the Home-NMES group. Both groups demonstrated lower quadriceps strength on their involved limbs compared to their uninvolved limbs at baseline, three months post-ACLr, and six months post-ACLr. Quadriceps progressively decreased in the involved limbs of both groups from baseline to three months post-ACLr and baseline to six months post-ACLr, and increased from three months to six months post- ACLr. At baseline, voluntary quadriceps activation was higher in the involved limbs of both groups compared to their uninvolved limbs. There were no group differences or changes over time observed in the involved limb of both groups with corticomotor excitability. Conclusion: Although quadriceps weakness is more apparent in the involved limb of patients after ACLr, the quadriceps strength of their uninvolved limb was also affected. Clinicians are encouraged to not rely on a quadriceps strength limb symmetry index when making return-sport-decisions for their patients after recovering from ACLr. The quadriceps in the uninvolved limb of patients demonstrated more inhibition, which may explain the quadriceps strength deficits observed in the uninvolved limb of patients following ACLr. To reduce the risk of subsequent injury upon return-to-sport and protect against the development of knee OA, we recommend that clinicians incorporate bilateral interventions aimed at restoring quadriceps strength and disinhibiting the quadriceps. Intensive quadriceps strengthening should be performed in the early stages of ACLr rehabilitation, so that lower extremity function can be improved in patients later on. Lastly, the effectiveness of home-based NMES as a modality for restoring quadriceps strength and activation in patients after ACLr is inconclusive. Home-based NMES provides patients with the ability to receive higher doses of NMES to the quadriceps; but its effectiveness may be limited by low contraction intensities and poor treatment compliance in patients