Human Preference-Based Learning for High-dimensional Optimization of Exoskeleton Walking Gaits

Optimizing lower-body exoskeleton walking gaits for user comfort requires understanding users’ preferences over a high-dimensional gait parameter space. However, existing preference-based learning methods have only explored low-dimensional domains due to computational limitations. To learn user preferences in high dimensions, this work presents LINECOSPAR, a human-in-the-loop preference-based framework that enables optimization over many parameters by iteratively exploring one-dimensional subspaces. Additionally, this work identifies gait attributes that characterize broader preferences across users. In simulations and human trials, we empirically verify that LINECOSPAR is a sample-efficient approach for high-dimensional preference optimization. Our analysis of the experimental data reveals a correspondence between human preferences and objective measures of dynamicity, while also highlighting differences in the utility functions underlying individual users’ gait preferences. This result has implications for exoskeleton gait synthesis, an active field with applications to clinical use and patient rehabilitation

Preference-Based Learning for Exoskeleton Gait Optimization

This paper presents a personalized gait optimization framework for lower-body exoskeletons. Rather than optimizing numerical objectives such as the mechanical cost of transport, our approach directly learns from user preferences, e.g., for comfort. Building upon work in preference-based interactive learning, we present the CoSpar algorithm. CoSpar prompts the user to give pairwise preferences between trials and suggest improvements; as exoskeleton walking is a non-intuitive behavior, users can provide preferences more easily and reliably than numerical feedback. We show that CoSpar performs competitively in simulation and demonstrate a prototype implementation of CoSpar on a lower-body exoskeleton to optimize human walking trajectory features. In the experiments, CoSpar consistently found user-preferred parameters of the exoskeleton’s walking gait, which suggests that it is a promising starting point for adapting and personalizing exoskeletons (or other assistive devices) to individual users

Human Preference-Based Learning for High-dimensional Optimization of Exoskeleton Walking Gaits

Understanding users' gait preferences of a lower-body exoskeleton requires optimizing over the high-dimensional gait parameter space. However, existing preference-based learning methods have only explored low-dimensional domains due to computational limitations. To learn user preferences in high dimensions, this work presents LineCoSpar, a human-in-the-loop preference-based framework that enables optimization over many parameters by iteratively exploring one-dimensional subspaces. Additionally, this work identifies gait attributes that characterize broader preferences across users. In simulations and human trials, we empirically verify that LineCoSpar is a sample-efficient approach for high-dimensional preference optimization. Our analysis of the experimental data reveals a correspondence between human preferences and objective measures of dynamic stability, while also highlighting inconsistencies in the utility functions underlying different users' gait preferences. This has implications for exoskeleton gait synthesis, an active field with applications to clinical use and patient rehabilitation

"Body-In-The-Loop": Optimizing Device Parameters Using Measures of Instantaneous Energetic Cost

This paper demonstrates methods for the online optimization of assistive robotic devices such as powered prostheses, orthoses and exoskeletons. Our algorithms estimate the value of a physiological objective in real-time (with a body “in-the-loop”) and use this information to identify optimal device parameters. To handle sensor data that are noisy and dynamically delayed, we rely on a combination of dynamic estimation and response surface identification. We evaluated three algorithms (Steady-State Cost Mapping, Instantaneous Cost Mapping, and Instantaneous Cost Gradient Search) with eight healthy human subjects. Steady-State Cost Mapping is an established technique that fits a cubic polynomial to averages of steady-state measures at different parameter settings. The optimal parameter value is determined from the polynomial fit. Using a continuous sweep over a range of parameters and taking into account measurement dynamics, Instantaneous Cost Mapping identifies a cubic polynomial more quickly. Instantaneous Cost Gradient Search uses a similar technique to iteratively approach the optimal parameter value using estimates of the local gradient. To evaluate these methods in a simple and repeatable way, we prescribed step frequency via a metronome and optimized this frequency to minimize metabolic energetic cost. This use of step frequency allows a comparison of our results to established techniques and enables others to replicate our methods. Our results show that all three methods achieve similar accuracy in estimating optimal step frequency. For all methods, the average error between the predicted minima and the subjects’ preferred step frequencies was less than 1% with a standard deviation between 4% and 5%. Using Instantaneous Cost Mapping, we were able to reduce subject walking-time from over an hour to less than 10 minutes. While, for a single parameter, the Instantaneous Cost Gradient Search is not much faster than Steady-State Cost Mapping, the Instantaneous Cost Gradient Search extends favorably to multi-dimensional parameter spaces

Adaptive Controllers for Assistive Robotic Devices

Lower extremity assistive robotic devices, such as exoskeletons and prostheses, have the potential to improve mobility for millions of individuals, both healthy and disabled. These devices are designed to work in conjunction with the user to enhance or replace lost functionality of a limb. Given the large variability in walking dynamics from person to person, it is still an open research question of how to optimally control such devices to maximize their benefit for each individual user. In this context, it is becoming more and more evident that there exists no "one size fits all" solution, but that each device needs to be tuned on a subject-specific basis to best account for each user's unique gait characteristics. However, the controllers that run in the background of these devices to dictate when and what type of actuation to deliver often have up to a hundred different parameters that can be tuned on a subject-specific basis. To hand tune each parameter can be an extremely tedious and time consuming process. Additionally, current tuning practices often rely on subjective measures to inform the fitting process. To address the current obstacles associated with device control and tuning, I have developed novel tools that overcome some of these issues through the design of control architectures that autonomously adapt to the user based upon real-time physiological measures. This approach frames the tuning process of a device as a real-time optimization and allows for the device to co-adapt with the wearer during use. As an outcome of these approaches, I have been able to investigate what qualities of a device controller are beneficial to users through the analysis of whole body kinematics, dynamics, and energetics. The framework of my research has been broken down into four major projects. First, I investigated how current standards of processing and analyzing physiological measures could be improved upon. Specifically, I focused on how to analyze non-steady-state measures of metabolic work rate in real time and how the noise content of theses measures can inform confidence analyses. Second, I applied the techniques I developed for analyzing non-steady-state measures of metabolic work rate to conduct a real-time optimization of powered bilateral ankle exoskeletons. For this study I employed a gradient descent optimization to tune and optimize an actuation timing parameter of these simple exoskeletons on a subject-specific basis. Third, I investigated how users may use an adaptive controller where they had a more direct impact on the adaptation via their own muscle recruitment. In this study, I designed and tested an adaptive gain proportional myoelectric controller with healthy subjects walking in bilateral ankle exoskeletons. Through this work I showed that subjects adapted to using increased levels of total ankle power compared to unpowered walking in the devices. As a result, subjects decreased power at their hip and were able to achieve large decreases in their metabolic work rate compared to unpowered walking. Fourth, I compared how subjects may use a controller driven by neural signals differently than one driven by mechanically intrinsic signals. The results of this project suggest that control based on neural signals may be better suited for therapeutic rehabilitation than control based on mechanically intrinsic signals. Together, these four projects have drastically improved upon subject-specific control of assistive devices in both a research and clinical setting.PHDMechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/144029/1/jrkoller_1.pd

ROIAL: Region of Interest Active Learning for Characterizing Exoskeleton Gait Preference Landscapes

Characterizing what types of exoskeleton gaits are comfortable for users, and understanding the science of walking more generally, require recovering a user's utility landscape. Learning these landscapes is challenging, as walking trajectories are defined by numerous gait parameters, data collection from human trials is expensive, and user safety and comfort must be ensured. This work proposes the Region of Interest Active Learning (ROIAL) framework, which actively learns each user's underlying utility function over a region of interest that ensures safety and comfort. ROIAL learns from ordinal and preference feedback, which are more reliable feedback mechanisms than absolute numerical scores. The algorithm's performance is evaluated both in simulation and experimentally for three able-bodied subjects walking inside of a lower-body exoskeleton. ROIAL learns Bayesian posteriors that predict each exoskeleton user's utility landscape across four exoskeleton gait parameters. The algorithm discovers both commonalities and discrepancies across users' gait preferences and identifies the gait parameters that most influenced user feedback. These results demonstrate the feasibility of recovering gait utility landscapes from limited human trials

Preference-Based Learning for Exoskeleton Gait Optimization

This paper presents a personalized gait optimization framework for lower-body exoskeletons. Rather than optimizing numerical objectives such as the mechanical cost of transport, our approach directly learns from user preferences, e.g., for comfort. Building upon work in preference-based interactive learning, we present the CoSpar algorithm. CoSpar prompts the user to give pairwise preferences between trials and suggest improvements; as exoskeleton walking is a non-intuitive behavior, users can provide preferences more easily and reliably than numerical feedback. We show that CoSpar performs competitively in simulation and demonstrate a prototype implementation of CoSpar on a lower-body exoskeleton to optimize human walking trajectory features. In the experiments, CoSpar consistently found user-preferred parameters of the exoskeleton’s walking gait, which suggests that it is a promising starting point for adapting and personalizing exoskeletons (or other assistive devices) to individual users

Development and Biomechanical Analysis toward a Mechanically Passive Wearable Shoulder Exoskeleton

Shoulder disability is a prevalent health issue associated with various orthopedic and neurological conditions, like rotator cuff tear and peripheral nerve injury. Many individuals with shoulder disability experience mild to moderate impairment and struggle with elevating the shoulder or holding the arm against gravity. To address this clinical need, I have focused my research on developing wearable passive exoskeletons that provide continuous at-home movement assistance. Through a combination of experiments and computational tools, I aim to optimize the design of these exoskeletons. In pursuit of this goal, I have designed, fabricated, and preliminarily evaluated a wearable, passive, cam-driven shoulder exoskeleton prototype. Notably, the exoskeleton features a modular spring-cam-wheel module, allowing customizable assistive force to compensate for different proportions of the shoulder elevation moment due to gravity. The results of my research demonstrated that this exoskeleton, providing modest one-fourth gravity moment compensation at the shoulder, can effectively reduce muscle activity, including deltoid and rotator cuff muscles. One crucial aspect of passive shoulder exoskeleton design is determining the optimal anti-gravity assistance level. I have addressed this challenge using computational tools and found that an assistance level within the range of 20-30% of the maximum gravity torque at the shoulder joint yields superior performance for specific shoulder functional tasks. When facing a new task dynamic, such as wearing a passive shoulder exoskeleton, the human neuro-musculoskeletal system adapts and modulates limb impedance at the end-limb (i.e., hand) to enhance task stability. I have presented development and validation of a realistic neuromusculoskeletal model of the upper limb that can predict stiffness modulation and motor adaptation in response to newly introduced environments and force fields. Future studies will explore the model\u27s applicability in predicting stiffness modulation for 3D movements in novel environments, such as passive assistive devices\u27 force fields