Pose Estimation and Segmentation for Rehabilitation

The global population is getting older and the aging demographic is increasing demands on health-care industry. This will drive the demand for post stroke, joint replacement, and chronic disease management rehabilitation. Currently physiotherapists rely on mostly subjective and observational tools for patient assessment and progress tracking. This thesis proposes methods to enable the use of non-intrusive, small, wearable, wireless sensors to estimate the pose of the lower body during rehabilitation and extract objective performance measures useful for therapists. Two different kinematic models of the human lower body are introduced. The first approach expresses the body position and orientation in the world frame using three prismatic and revolute joints, while the second switches the model's base between the right and the left ankle during gait. An Extended Kalman Filter (EKF) is set up to estimate the joint angles, velocities, and accelerations of the models using measurements from inertial measurement units. The state update model assumes constant joint acceleration and is linear. Measurement prediction, relating the joint positions, velocities and accelerations to the measured angular velocity and linear acceleration at each IMU, is done using forward kinematics, using one of the two proposed kinematic models. The approach is validated on healthy participant gait using motion capture studio data for ground truth comparison. The prismatic and revolute model achieves better Cartesian position accuracy in the swing leg due to a shorter kinematic chain, while the switching base model improves the stance leg Cartesian estimate and does not allow measurement noise to accumulate as drift in global position, knee joint angle root mean squared errors (RMSE) of 6.1 and 5.6 degrees are attained respectively by the models. Next the Rhythmic Extended Kalman Filter (R-EKF) algorithm is developed to improve pose estimation. It learns a model of rhythmic movement over time based on harmonic Fourier series and removes the constant acceleration assumption. The estimated phase and frequency of the motion also allow the proposed approach to segment the motion into repetitions and extract useful features such as gait symmetry, step length, and mean joint movement and variance. The algorithm is shown to outperform the EKF in simulation, on healthy participant data, and stroke patient monthly assessments. For the healthy participant marching dataset, the R-EKF improves joint acceleration and velocity estimates over regular EKF by 40% and 37% respectively, estimates joint angles with 2.4 degree RMSE, and segments the motion into repetitions with 96% accuracy. While the proposed R-EKF effectively segments rhythmic rehabilitation movement such as gait, not all rehabilitation motions are rhythmic or may have uneven delays between repetitions by regimen design or due to fatigue. For such motions a time-series segmentation as data point classification algorithm is proposed. Common dimensionality reduction and classification techniques are applied to estimated joint angle data to classify each time-step as a segment or non-segment point. The algorithm is tested on five common rehabilitation exercises performed by healthy participants and achieves a segmentation accuracy of 82%

Automated Pose Estimation for the Assessment of Dynamic Knee Valgus and Risk of Knee Injury during the Single Leg Squat

Many clinical assessment protocols rely on the evaluation of functional movement tests such as the Single Leg Squat (SLS), which are often assessed visually. Visual assessment is subjective and depends on the experience of the clinician. Developing a reliable automatic human motion tracking and assessment system can improve the accuracy of SLS clinical assessments and provide objective results that can be tracked and monitored over time to guide rehabilitation and determine an individual's response to an intervention. In this study, an Inertial Measurement Unit (IMU) based method for automated assessment of squat quality is proposed to provide clinicians with a quantitative measure of SLS performance. First, an automated pose estimation method is applied to SLS motion data. A set of three IMUs is used to estimate the joint angles, velocities and accelerations of the squatting leg. To tackle noisy sensor measurements and gyro drift, a 7 degree of freedom (DOF) kinematic model of the lower leg was applied together with a constant acceleration assumption to approximate the angular velocity and linear acceleration at each sensor location. The kinematic model predictions of the angular velocity and linear acceleration and sensor measurements were fused via an Extended Kalman Filter (EKF). The position, velocity, and acceleration of each DOF were defined as the states to be estimated by the EKF. The pose estimation results showed successful extraction of joint angles with an average RMS error of 3.2 degrees, 5.5 degrees, 7 degrees compared to joint angles estimated from motion capture for the ankle, knee, and hip joints, respectively. For this estimation, the required parameters for the kinematic model, including information about the sensor placement and orientation as well as the kinematic link lengths, were extracted from the marker data. However, in clinical applications of the proposed method, when marker data is not available, these parameters need to be measured. Measuring these parameters is time consuming in the clinical setting, which limits application of IMUs for clinical purposes. With the motivation to make this procedure easier and faster, a method for approximating the parameters using placement assumptions and body measures was described. A sensitivity analysis was performed to detect those parameters which most affect pose estimation accuracy. The sensitivity analysis results revealed that sensor orientation is the most critical factor for accurate pose estimation. In this thesis, a simple and easy to use method is proposed for sensor orientation calibration, based on a systematic placement of sensors and using gyroscope information for orientation estimation. This protocol was evaluated experimentally and pose estimation error with approximated parameters before and after applying the calibration protocol were compared. The comparison results showed that the estimate of the sensor orientation increases the pose estimation accuracy by 6.5 degrees for the knee joint angle and with an average of 1.8 degrees for other joints without the need for time consuming calibration. In the second part of the thesis, an algorithm for automated assessment of the SLS in terms of dynamic knee valgus and risk of knee injury is developed. After applying the pose estimation algorithm to IMU data of SLS motions, the estimated time series data of joint angles, velocities and accelerations for consecutive squats were segmented into individual squat repetitions. Statistical time domain features were generated from each repetition. The most informative features were selected using a combination of 18 feature selection techniques. Six common classifiers in including SVM, Linear Multinomial Logistic Regression, Decision Tree, Naive Bayes, K Nearest Neighborhood, and Random Forests were applied to the full dimensional data, the subset of selected features, and extracted features by supervised principal component analysis. The proposed approach was evaluated in two trials. First, a pilot study was conducted on a small dataset, followed by analysis on a larger clinical data set, collected by our clinical collaborator. For the clinical study, a dataset of SLS performed by healthy participants was collected and labelled by three expert clinical raters using two different labeling criteria: "observed amount of knee valgus" and "overall risk of injury". Labels included "good", "moderate", and "poor" squat quality or "high risk", "mild to moderate risk", and "no risk" of injury. Feature selection results showed that both flexion at the hip and knee, as well as hip and ankle internal rotation are discriminative features, and that participants with "poor" squats bend the hip and knee less than those with better squat performance. Furthermore, improved classifi cation performance was achieved by training separate classifi ers strati ed by gender. Classifi cation results showed excellent accuracy, 93.1% for classifying squat quality as "poor" or "good" and 95.3% for differentiating between high and no risk of injury

Human motion estimation and controller learning

Humans are capable of complex manipulation and locomotion tasks. They are able to achieve energy-efficient gait, reject disturbances, handle changing loads, and adapt to environmental constraints. Using inspiration from the human body, robotics researchers aim to develop systems with similar capabilities. Research suggests that humans minimize a task specific cost function when performing movements. In order to learn this cost function from demonstrations and incorporate it into a controller, it is first imperative to accurately estimate the expert motion. The captured motions can then be analyzed to extract the objective function the expert was minimizing. We propose a framework for human motion estimation from wearable sensors. Human body joints are modeled by matrix Lie groups, using special orthogonal groups SO(2) and SO(3) for joint pose and special Euclidean group SE(3) for base link pose representation. To estimate the human joint pose, velocity and acceleration, we provide the equations for employing the extended Kalman Filter on Lie Groups, thus explicitly accounting for the non-Euclidean geometry of the state space. Incorporating interaction constraints with respect to the environment or within the participant allows us to track global body position without an absolute reference and ensure viable pose estimate. The algorithms are extensively validated in both simulation and real-world experiments. Next, to learn underlying expert control strategies from the expert demonstrations we present a novel fast approximate multi-variate Gaussian Process regression. The method estimates the underlying cost function, without making assumptions on its structure. The computational efficiency of the approach allows for real time forward horizon prediction. Using a linear model predictive control framework we then reproduce the demonstrated movements on a robot. The learned cost function captures the variability in expert motion as well as the correlations between states, leading to a controller that both produces motions and reacts to disturbances in a human-like manner. The model predictive control formulation allows the controller to satisfy task and joint space constraints avoiding obstacles and self collisions, as well as torque constraints, ensuring operational feasibility. The approach is validated on the Franka Emika robot using real human motion exemplars