A Locality-based Neural Solver for Optical Motion Capture

We present a novel locality-based learning method for cleaning and solving optical motion capture data. Given noisy marker data, we propose a new heterogeneous graph neural network which treats markers and joints as different types of nodes, and uses graph convolution operations to extract the local features of markers and joints and transform them to clean motions. To deal with anomaly markers (e.g. occluded or with big tracking errors), the key insight is that a marker's motion shows strong correlations with the motions of its immediate neighboring markers but less so with other markers, a.k.a. locality, which enables us to efficiently fill missing markers (e.g. due to occlusion). Additionally, we also identify marker outliers due to tracking errors by investigating their acceleration profiles. Finally, we propose a training regime based on representation learning and data augmentation, by training the model on data with masking. The masking schemes aim to mimic the occluded and noisy markers often observed in the real data. Finally, we show that our method achieves high accuracy on multiple metrics across various datasets. Extensive comparison shows our method outperforms state-of-the-art methods in terms of prediction accuracy of occluded marker position error by approximately 20%, which leads to a further error reduction on the reconstructed joint rotations and positions by 30%. The code and data for this paper are available at https://github.com/non-void/LocalMoCap.Comment: Siggraph Asia 2023 Conference Pape

7-degree-of-freedom hybrid-manipulator exoskeleton for lower-limb motion capture

Lower-limb exoskeletons are wearable robotic systems with a kinematic structure closely matching that of the human leg. In part, this technology can be used to provide clinical assessment and improved independent-walking competency for people living with the effects of stroke, spinal cord injury, Parkinson’s disease, multiple sclerosis, and sarcopenia. Individually, these demographics represent approximately: 405 thousand, 100 thousand, 67.5 thousand, 100 thousand, and 5.9 million Canadians, respectively. Key shortcomings in the current state-of-the-art are: restriction on several of the human leg’s primary joint movements, coaxial joint alignments at the exoskeleton-human interface, and exclusion of well-suited parallel manipulator components. A novel exoskeleton design is thus formulated to address these issues while maintaining large ranges of joint motion. Ultimately, a single-leg unactuated prototype is constructed for seven degree-of-freedom joint angle measurements; it achieves an extent of motion-capture accuracy comparable to a commercial inertial-based system during three levels of human mobility testing

Attention-Based Recurrent Autoencoder for Motion Capture Denoising

To resolve the problem of massive loss of MoCap data from optical motion capture, we propose a novel network architecture based on attention mechanism and recurrent network. Its advantage is that the use of encoder-decoder enables automatic human motion manifold learning, capturing the hidden spatial-temporal relationships in motion sequences. In addition, by using the multi-head attention mechanism, it is possible to identify the most relevant corrupted frames with specific position information to recovery the missing markers, which can lead to more accurate motion reconstruction. Simulation experiments demonstrate that the network model we proposed can effectively handle the large-scale missing markers problem with better robustness, smaller errors and more natural recovered motion sequence compared to the reference method

Intelligent Sensors for Human Motion Analysis

The book, "Intelligent Sensors for Human Motion Analysis," contains 17 articles published in the Special Issue of the Sensors journal. These articles deal with many aspects related to the analysis of human movement. New techniques and methods for pose estimation, gait recognition, and fall detection have been proposed and verified. Some of them will trigger further research, and some may become the backbone of commercial systems

Smart Technology for Telerehabilitation: A Smart Device Inertial-sensing Method for Gait Analysis

The aim of this work was to develop and validate an iPod Touch (4th generation) as a potential ambulatory monitoring system for clinical and non-clinical gait analysis. This thesis comprises four interrelated studies, the first overviews the current available literature on wearable accelerometry-based technology (AT) able to assess mobility-related functional activities in subjects with neurological conditions in home and community settings. The second study focuses on the detection of time-accurate and robust gait features from a single inertial measurement unit (IMU) on the lower back, establishing a reference framework in the process. The third study presents a simple step length algorithm for straight-line walking and the fourth and final study addresses the accuracy of an iPod’s inertial-sensing capabilities, more specifically, the validity of an inertial-sensing method (integrated in an iPod) to obtain time-accurate vertical lower trunk displacement measures. The systematic review revealed that present research primarily focuses on the development of accurate methods able to identify and distinguish different functional activities. While these are important aims, much of the conducted work remains in laboratory environments, with relatively little research moving from the “bench to the bedside.” This review only identified a few studies that explored AT’s potential outside of laboratory settings, indicating that clinical and real-world research significantly lags behind its engineering counterpart. In addition, AT methods are largely based on machine-learning algorithms that rely on a feature selection process. However, extracted features depend on the signal output being measured, which is seldom described. It is, therefore, difficult to determine the accuracy of AT methods without characterizing gait signals first. Furthermore, much variability exists among approaches (including the numbers of body-fixed sensors and sensor locations) to obtain useful data to analyze human movement. From an end-user’s perspective, reducing the amount of sensors to one instrument that is attached to a single location on the body would greatly simplify the design and use of the system. With this in mind, the accuracy of formerly identified or gait events from a single IMU attached to the lower trunk was explored. The study’s analysis of the trunk’s vertical and anterior-posterior acceleration pattern (and of their integrands) demonstrates, that a combination of both signals may provide more nuanced information regarding a person’s gait cycle, ultimately permitting more clinically relevant gait features to be extracted. Going one step further, a modified step length algorithm based on a pendulum model of the swing leg was proposed. By incorporating the trunk’s anterior-posterior displacement, more accurate predictions of mean step length can be made in healthy subjects at self-selected walking speeds. Experimental results indicate that the proposed algorithm estimates step length with errors less than 3% (mean error of 0.80 ± 2.01cm). The performance of this algorithm, however, still needs to be verified for those suffering from gait disturbances. Having established a referential framework for the extraction of temporal gait parameters as well as an algorithm for step length estimations from one instrument attached to the lower trunk, the fourth and final study explored the inertial-sensing capabilities of an iPod Touch. With the help of Dr. Ian Sheret and Oxford Brookes’ spin-off company ‘Wildknowledge’, a smart application for the iPod Touch was developed. The study results demonstrate that the proposed inertial-sensing method can reliably derive lower trunk vertical displacement (intraclass correlations ranging from .80 to .96) with similar agreement measurement levels to those gathered by a conventional inertial sensor (small systematic error of 2.2mm and a typical error of 3mm). By incorporating the aforementioned methods, an iPod Touch can potentially serve as a novel ambulatory monitor system capable of assessing gait in clinical and non-clinical environments

Motor Control Insights on Walking Planner and its Stability

The application of biomechanic and motor control models in the control of bidedal robots (humanoids, and exoskeletons) has revealed limitations of our understanding of human locomotion. A recently proposed model uses the potential energy for bipedal structures to model the bipedal dynamics, and it allows to predict the system dynamics from its kinematics. This work proposes a task-space planner for human-like straight locomotion that target application of in rehabilitation robotics and computational neuroscience. The proposed architecture is based on the potential energy model and employs locomotor strategies from human data as a reference for human behaviour. The model generates Centre of Mass (CoM) trajectories, foot swing trajectories and the Base of Support (BoS) over time. The data show that the proposed architecture can generate behaviour in line with human walking strategies for both the CoM and the foot swing. Despite the CoM vertical trajectory being not as smooth as a human trajectory, yet the proposed model significantly reduces the error in the estimation of the CoM vertical trajectory compared to the inverted pendulum models. The proposed model is also able to asses the stability based on the body kinematics embedding in currently used in the clinical practice. However, the model also implies a shift in the interpretation of the spatiotemporal parameters of the gait, which are now determined by the conditions for the equilibrium and not \textit{vice versa}. In other words, locomotion is a dynamic reaching where the motor primitives are also determined by gravity

Patient Movement Monitoring Based on IMU and Deep Learning

Osteoarthritis (OA) is the leading cause of disability among the aging population in the United States and is frequently treated by replacing deteriorated joints with metal and plastic components. Developing better quantitative measures of movement quality to track patients longitudinally in their own homes would enable personalized treatment plans and hasten the advancement of promising new interventions. Wearable sensors and machine learning used to quantify patient movement could revolutionize the diagnosis and treatment of movement disorders. The purpose of this dissertation was to overcome technical challenges associated with the use of wearable sensors, specifically Inertial Measurement Units (IMUs), as a diagnostic tool for osteoarthritic (OA) and total knee replacement patients (TKR) through a detailed biomechanical assessment and development of machine learning algorithms. Specifically, the first study developed a relevant dataset consisting of IMU and associated biomechanical parameters of OA and TKR patients performing various activities, created a machine learning-based framework to accurately estimate spatiotemporal movement characteristics from IMU during level ground walking, and defined optimum sensor configuration associated with the patient population and activity. The second study designed a framework to generate synthetic kinematic and associated IMU data as well as investigated the influence of adding synthetic data into training-measured data on deep learning model performance. The third study investigated the kinematic variation between two patient’s population across various activities: stair ascent, stair descent, and gait using principle component analysis PCA. Additionally, PCA-based autoencoders were developed to generate synthetic kinematics data for each patient population and activity. The fourth study investigated the potential use of a universal deep learning model for the estimation of lower extremities’ kinematics across various activities. Therefore, this model can be used as a global model for transfer learning methods in future research. This line of study resulted in a machine-learning framework that can be used to estimate biomechanical movements based on a stream of signals emitted from low-cost and portable IMUs. Eventually, this could lead to a simple clinical tool for tracking patients\u27 movements in their own homes and translating those movements into diagnostic metrics that clinicians will be able to use to tailor treatment to each patient\u27s needs in the future

Wearable Movement Sensors for Rehabilitation: From Technology to Clinical Practice

This Special Issue shows a range of potential opportunities for the application of wearable movement sensors in motor rehabilitation. However, the papers surely do not cover the whole field of physical behavior monitoring in motor rehabilitation. Most studies in this Special Issue focused on the technical validation of wearable sensors and the development of algorithms. Clinical validation studies, studies applying wearable sensors for the monitoring of physical behavior in daily life conditions, and papers about the implementation of wearable sensors in motor rehabilitation are under-represented in this Special Issue. Studies investigating the usability and feasibility of wearable movement sensors in clinical populations were lacking. We encourage researchers to investigate the usability, acceptance, feasibility, reliability, and clinical validity of wearable sensors in clinical populations to facilitate the application of wearable movement sensors in motor rehabilitation