Human Motion Analysis: From Gait Modeling to Shape Representation and Pose Estimation

This dissertation presents a series of fundamental approaches to the human motion analysis from three perspectives, i.e., manifold learning-based gait motion modeling, articulated shape representation and efficient pose estimation. Firstly, a new joint gait-pose manifold (JGPM) learning algorithm is proposed to jointly optimize the gait and pose variables simultaneously. To enhance the representability and flexibility for complex motion modeling, we also propose a multi-layer JGPM that is capable of dealing with a variety of walking styles and various strides. We resort to a topologically-constrained Gaussian process latent variable model (GPLVM) to learn the multi-layer JGPM where two new techniques are introduced to facilitate model learning. First is training data diversification that creates a set of simulated motion data with different strides under limited data. Second is the topology-aware local learning that is to speed up model learning by taking advantage of the local topological structure. We demonstrate the effectiveness of our approach by synthesizing the high-quality motions from the multi-layer model. The experimental results show that the multi-layer JGPM outperforms several existing GPLVM-based models in terms of the overall performance of motion modeling.On the other hand, to achieve efficient human pose estimation from a single depth sensor, we develop a generalized Gaussian kernel correlation (GKC)-based framework which supports not only body shape modeling, but also articulated pose tracking. We first generalize GKC from the univariate Gaussian to the multivariate one and derive a unified GKC function that provides a continuous and differentiable similarity measure between a template and an observation, both of which are represented by a collection of univariate and/or multivariate Gaussian kernels. Then, to facilitate the data matching and accommodate articulated body deformation, we embed a quaternion-based articulated skeleton into a collection of multivariate Gaussians-based template model and develop an articulated GKC (AGKC) which supports subject-specific shape modeling and articulated pose tracking for both the full-body and hand. Our tracking algorithm is simple yet effective and computationally efficient. We evaluate our algorithm on two benchmark depth datasets. The experimental results are promising and competitive when compared with state-of-the-art algorithms.Electrical Engineerin

Cascaded 3D Full-body Pose Regression from Single Depth Image at 100 FPS

There are increasing real-time live applications in virtual reality, where it plays an important role in capturing and retargetting 3D human pose. But it is still challenging to estimate accurate 3D pose from consumer imaging devices such as depth camera. This paper presents a novel cascaded 3D full-body pose regression method to estimate accurate pose from a single depth image at 100 fps. The key idea is to train cascaded regressors based on Gradient Boosting algorithm from pre-recorded human motion capture database. By incorporating hierarchical kinematics model of human pose into the learning procedure, we can directly estimate accurate 3D joint angles instead of joint positions. The biggest advantage of this model is that the bone length can be preserved during the whole 3D pose estimation procedure, which leads to more effective features and higher pose estimation accuracy. Our method can be used as an initialization procedure when combining with tracking methods. We demonstrate the power of our method on a wide range of synthesized human motion data from CMU mocap database, Human3.6M dataset and real human movements data captured in real time. In our comparison against previous 3D pose estimation methods and commercial system such as Kinect 2017, we achieve the state-of-the-art accuracy

Learning to Transform Time Series with a Few Examples

We describe a semi-supervised regression algorithm that learns to transform one time series into another time series given examples of the transformation. This algorithm is applied to tracking, where a time series of observations from sensors is transformed to a time series describing the pose of a target. Instead of defining and implementing such transformations for each tracking task separately, our algorithm learns a memoryless transformation of time series from a few example input-output mappings. The algorithm searches for a smooth function that fits the training examples and, when applied to the input time series, produces a time series that evolves according to assumed dynamics. The learning procedure is fast and lends itself to a closed-form solution. It is closely related to nonlinear system identification and manifold learning techniques. We demonstrate our algorithm on the tasks of tracking RFID tags from signal strength measurements, recovering the pose of rigid objects, deformable bodies, and articulated bodies from video sequences. For these tasks, this algorithm requires significantly fewer examples compared to fully-supervised regression algorithms or semi-supervised learning algorithms that do not take the dynamics of the output time series into account

Dual Quaternions as Constraints in 4D-DPM Models for Pose Estimation

This work was partially financed by Plan Nacional de Investigacion y Desarrollo (I+D), Comision Interministerial de Ciencia y Tecnologia (FEDER-CICYT) under the project DPI2013-44227-R.Martínez Bertí, E.; Sánchez Salmerón, AJ.; Ricolfe Viala, C. (2017). Dual Quaternions as Constraints in 4D-DPM Models for Pose Estimation. Sensors. 17 (8)(1913):1-16. https://doi.org/10.3390/s17081913S11617 (8)191

Articulated Statistical Shape Modelling of the Shoulder Joint

The shoulder joint is the most mobile and unstable joint in the human body. This makes it vulnerable to soft tissue pathologies and dislocation. Insight into the kinematics of the joint may enable improved diagnosis and treatment of different shoulder pathologies. Shoulder joint kinematics can be influenced by the articular geometry of the joint. The aim of this project was to develop an analysis framework for shoulder joint kinematics via the use of articulated statistical shape models (ASSMs). Articulated statistical shape models extend conventional statistical shape models by combining the shape variability of anatomical objects collected from different subjects (statistical shape models), with the physical variation of pose between the same objects (articulation). The developed pipeline involved manual annotation of anatomical landmarks selected on 3D surface meshes of scapulae and humeri and establishing dense surface correspondence across these data through a registration process. The registration was performed using a Gaussian process morphable model fitting approach. In order to register two objects separately, while keeping their shape and kinematics relationship intact, one of the objects (scapula) was fixed leaving the other (humerus) to be mobile. All the pairs of registered humeri and scapulae were brought back to their native imaged position using the inverse of the associated registration transformation. The glenohumeral rotational center and local anatomic coordinate system of the humeri and scapulae were determined using the definitions suggested by the International Society of Biomechanics. Three motions (flexion, abduction, and internal rotation) were generated using Euler angle sequences. The ASSM of the model was built using principal component analysis and validated. The validation results show that the model adequately estimated the shape and pose encoded in the training data. Developing ASSM of the shoulder joint helps to define the statistical shape and pose parameters of the gleno humeral articulating surfaces. An ASSM of the shoulder joint has potential applications in the analysis and investigation of population-wide joint posture variation and kinematics. Such analyses may include determining and quantifying abnormal articulation of the joint based on the range of motion; understanding of detailed glenohumeral joint function and internal joint measurement; and diagnosis of shoulder pathologies. Future work will involve developing a protocol for encoding the shoulder ASSM with real, rather than handcrafted, pose variation