A comparative study of pose representation and dynamics modelling for online motion quality assessment

© 2015 The Authors. Published by Elsevier Inc. Quantitative assessment of the quality of motion is increasingly in demand by clinicians in healthcare and rehabilitation monitoring of patients. We study and compare the performances of different pose representations and HMM models of dynamics of movement for online quality assessment of human motion. In a general sense, our assessment framework builds a model of normal human motion from skeleton-based samples of healthy individuals. It encapsulates the dynamics of human body pose using robust manifold representation and a first-order Markovian assumption. We then assess deviations from it via a continuous online measure. We compare different feature representations, reduced dimensionality spaces, and HMM models on motions typically tested in clinical settings, such as gait on stairs and flat surfaces, and transitions between sitting and standing. Our dataset is manually labelled by a qualified physiotherapist. The continuous-state HMM, combined with pose representation based on body-joints' location, outperforms standard discrete-state HMM approaches and other skeleton-based features in detecting gait abnormalities, as well as assessing deviations from the motion model on a frame-by-frame basis

Non-Intrusive Gait Recognition Employing Ultra Wideband Signal Detection

A self-regulating and non-contact impulse radio ultra wideband (IR-UWB) based 3D human gait analysis prototype has been modeled and developed with the help of supervised machine learning (SML) for this application for the first time. The work intends to provide a rewarding assistive biomedical application which would help doctors and clinicians monitor human gait trait and abnormalities with less human intervention in the fields of physiological examinations, physiotherapy, home assistance, rehabilitation success determination and health diagnostics, etc. The research comprises IR-UWB data gathered from a number of male and female participants in both anechoic chamber and multi-path environments. In total twenty four individuals have been recruited, where twenty individuals were said to have normal gait and four persons complained of knee pain that resulted in compensated spastic walking patterns. A 3D postural model of human movements has been created from the backscattering property of the radar pulses employing understanding of spherical trigonometry and vector fields. This subjective data (height of the body areas from the ground) of an individual have been recorded and implemented to extract the gait trait from associated biomechanical activity and differentiates the lower limb movement patterns from other body areas. Initially, a 2D postural model of human gait is presented from IR-UWB sensing phenomena employing spherical co-ordinate and trigonometry where only two dimensions such as, distance from radar and height of reflection have been determined. There are five pivotal gait parameters; step frequency, cadence, step length, walking speed, total covered distance, and body orientation which have all been measured employing radar principles and short term Fourier transformation (STFT). Subsequently, the proposed gait identification and parameter characterization has been analysed, tested and validated against popularly accepted smartphone applications with resulting variations of less than 5%. Subsequently, the spherical trigonometric model has been elevated to a 3D postural model where the prototype can determine width of motion, distance from radar, and height of reflection. Vector algebra has been incorporated with this 3D model to measure knee angles and hip angles from the extension and flexion of lower limbs to understand the gait behavior throughout the entire range of bipedal locomotion. Simultaneously, the Microsoft Kinect Xbox One has been employed during the experiment to assist in the validation process. The same vector mathematics have been implemented to the skeleton data obtained from Kinect to determine both the hip and knee angles. The outcomes have been compared by statistical graphical approach Bland and Altman (B&A) analysis. Further, the changes of knee angles obtained from the normal gaits have been used to train popular SMLs such as, k-nearest neighbour (kNN) and support vector machines (SVM). The trained model has subsequently been tested with the new data (knee angles extracted from both normal and abnormal gait) to assess the prediction ability of gait abnormality recognition. The outcomes have been validated through standard and wellknown statistical performance metrics with promising results found. The outcomes prove the acceptability of the proposed non-contact IR-UWB gait recognition to detect gait

Markerless Gait Classification Employing 3D IR-UWB Physiological Motion Sensing

Human gait refers to the propulsion achieved by the effort of human limbs, a reflex progression resulting from the rhythmic reciprocal bursts of flexor and extensor activity. Several quantitative models are followed by health professionals to diagnose gait abnormality. Marker-based gait quantification is considered a gold standard by the research and health communities. It reconstructs motion in 3D and provides parameters to measure gait. But, it is an expensive and intrusive technique, limited to soft tissue artefact, prone to incorrect marker positioning, and skin sensitivity problems. Hence, markerless, swiftly deployable, non-intrusive, camera-less prototypes would be a game changing possibility, and an example is proposed here. This paper illustrates a 3D gait motion analyser employing impulse radio ultra-wide band (IR-UWB) wireless technology. The prototype can measure 3D motion and determine quantitative parameters considering anatomical reference planes. Knee angles have been calculated from the gait by applying vector algebra. Simultaneously, the model has been corroborated with the popular markerless camera based 3D motion capturing system, the Kinect sensor. Bland and Altman (B&A) statistics has been applied to the proposed prototype and Kinect sensor results to verify the measurement agreement. Finally, the proposed prototype has been incorporated with popular supervised machine learning such as, k-nearest neighbour (kNN), support vector machine (SVM) and the deep learning technique deep neural multilayer perceptron (DMLP) network to automatically recognize gait abnormalities, with promising results presented

A review of computer vision-based approaches for physical rehabilitation and assessment

The computer vision community has extensively researched the area of human motion analysis, which primarily focuses on pose estimation, activity recognition, pose or gesture recognition and so on. However for many applications, like monitoring of functional rehabilitation of patients with musculo skeletal or physical impairments, the requirement is to comparatively evaluate human motion. In this survey, we capture important literature on vision-based monitoring and physical rehabilitation that focuses on comparative evaluation of human motion during the past two decades and discuss the state of current research in this area. Unlike other reviews in this area, which are written from a clinical objective, this article presents research in this area from a computer vision application perspective. We propose our own taxonomy of computer vision-based rehabilitation and assessment research which are further divided into sub-categories to capture novelties of each research. The review discusses the challenges of this domain due to the wide ranging human motion abnormalities and difficulty in automatically assessing those abnormalities. Finally, suggestions on the future direction of research are offered

Analysis of 3D human gait reconstructed with a depth camera and mirrors

L'évaluation de la démarche humaine est l'une des composantes essentielles dans les soins de santé. Les systèmes à base de marqueurs avec plusieurs caméras sont largement utilisés pour faire cette analyse. Cependant, ces systèmes nécessitent généralement des équipements spécifiques à prix élevé et/ou des moyens de calcul intensif. Afin de réduire le coût de ces dispositifs, nous nous concentrons sur un système d'analyse de la marche qui utilise une seule caméra de profondeur. Le principe de notre travail est similaire aux systèmes multi-caméras, mais l'ensemble de caméras est remplacé par un seul capteur de profondeur et des miroirs. Chaque miroir dans notre configuration joue le rôle d'une caméra qui capture la scène sous un point de vue différent. Puisque nous n'utilisons qu'une seule caméra, il est ainsi possible d'éviter l'étape de synchronisation et également de réduire le coût de l'appareillage. Notre thèse peut être divisée en deux sections: reconstruction 3D et analyse de la marche. Le résultat de la première section est utilisé comme entrée de la seconde. Notre système pour la reconstruction 3D est constitué d'une caméra de profondeur et deux miroirs. Deux types de capteurs de profondeur, qui se distinguent sur la base du mécanisme d'estimation de profondeur, ont été utilisés dans nos travaux. Avec la technique de lumière structurée (SL) intégrée dans le capteur Kinect 1, nous effectuons la reconstruction 3D à partir des principes de l'optique géométrique. Pour augmenter le niveau des détails du modèle reconstruit en 3D, la Kinect 2 qui estime la profondeur par temps de vol (ToF), est ensuite utilisée pour l'acquisition d'images. Cependant, en raison de réflections multiples sur les miroirs, il se produit une distorsion de la profondeur dans notre système. Nous proposons donc une approche simple pour réduire cette distorsion avant d'appliquer les techniques d'optique géométrique pour reconstruire un nuage de points de l'objet 3D. Pour l'analyse de la démarche, nous proposons diverses alternatives centrées sur la normalité de la marche et la mesure de sa symétrie. Cela devrait être utile lors de traitements cliniques pour évaluer, par exemple, la récupération du patient après une intervention chirurgicale. Ces méthodes se composent d'approches avec ou sans modèle qui ont des inconvénients et avantages différents. Dans cette thèse, nous présentons 3 méthodes qui traitent directement les nuages de points reconstruits dans la section précédente. La première utilise la corrélation croisée des demi-corps gauche et droit pour évaluer la symétrie de la démarche, tandis que les deux autres methodes utilisent des autoencodeurs issus de l'apprentissage profond pour mesurer la normalité de la démarche.The problem of assessing human gaits has received a great attention in the literature since gait analysis is one of key components in healthcare. Marker-based and multi-camera systems are widely employed to deal with this problem. However, such systems usually require specific equipments with high price and/or high computational cost. In order to reduce the cost of devices, we focus on a system of gait analysis which employs only one depth sensor. The principle of our work is similar to multi-camera systems, but the collection of cameras is replaced by one depth sensor and mirrors. Each mirror in our setup plays the role of a camera which captures the scene at a different viewpoint. Since we use only one camera, the step of synchronization can thus be avoided and the cost of devices is also reduced. Our studies can be separated into two categories: 3D reconstruction and gait analysis. The result of the former category is used as the input of the latter one. Our system for 3D reconstruction is built with a depth camera and two mirrors. Two types of depth sensor, which are distinguished based on the scheme of depth estimation, have been employed in our works. With the structured light (SL) technique integrated into the Kinect 1, we perform the 3D reconstruction based on geometrical optics. In order to increase the level of details of the 3D reconstructed model, the Kinect 2 with time-of-flight (ToF) depth measurement is used for image acquisition instead of the previous generation. However, due to multiple reflections on the mirrors, depth distortion occurs in our setup. We thus propose a simple approach for reducing such distortion before applying geometrical optics to reconstruct a point cloud of the 3D object. For the task of gait analysis, we propose various alternative approaches focusing on the problem of gait normality/symmetry measurement. They are expected to be useful for clinical treatments such as monitoring patient's recovery after surgery. These methods consist of model-free and model-based approaches that have different cons and pros. In this dissertation, we present 3 methods that directly process point clouds reconstructed from the previous work. The first one uses cross-correlation of left and right half-bodies to assess gait symmetry while the other ones employ deep auto-encoders to measure gait normality