Human metrology for person classification and recognition

Human metrological features generally refers to geometric measurements extracted from humans, such as height, chest circumference or foot length. Human metrology provides an important soft biometric that can be used in challenging situations, such as person classification and recognition at a distance, where hard biometric traits such as fingerprints and iris information cannot easily be acquired. In this work, we first study the question of predictability and correlation in human metrology. We show that partial or available measurements can be used to predict other missing measurements. We then investigate the use of human metrology for the prediction of other soft biometrics, viz. gender and weight. The experimental results based on our proposed copula-based model suggest that human body metrology contains enough information for reliable prediction of gender and weight. Also, the proposed copula-based technique is observed to reduce the impact of noise on prediction performance. We then study the question of whether face metrology can be exploited for reliable gender prediction. A new method based solely on metrological information from facial landmarks is developed. The performance of the proposed metrology-based method is compared with that of a state-of-the-art appearance-based method for gender classification. Results on several face databases show that the metrology-based approach resulted in comparable accuracy to that of the appearance-based method. Furthermore, we study the question of person recognition (classification and identification) via whole body metrology. Using CAESAR 1D database as baseline, we simulate intra-class variation with various noise models. The experimental results indicate that given enough number of features, our metrology-based recognition system can have promising performance that is comparable to several recent state-of-the-art recognition systems. We propose a non-parametric feature selection methodology, called adapted k-nearest neighbor estimator, which does not rely on intra-class distribution of the query set. This leads to improved results over other nearest neighbor estimators (as feature selection criteria) for moderate number of features. Finally we quantify the discrimination capability of human metrology, from both individuality and capacity perspectives. Generally, a biometric-based recognition technique relies on an assumption that the given biometric is unique to an individual. However, the validity of this assumption is not yet generally confirmed for most soft biometrics, such as human metrology. In this work, we first develop two schemes that can be used to quantify the individuality of a given soft-biometric system. Then, a Poisson channel model is proposed to analyze the recognition capacity of human metrology. Our study suggests that the performance of such a system depends more on the accuracy of the ground truth or training set

Facial Image Analysis for Body Mass Index, Makeup and Identity

The principal aim of facial image analysis in computer vision is to extract valuable information(e.g., age, gender, ethnicity, and identity) by interpreting perceived electronic signals from face images. In this dissertation, we develop facial image analysis systems for body mass index (BMI) prediction, makeup detection, as well as facial identity with makeup changes and BMI variations.;BMI is a commonly used measure of body fatness. In the first part of this thesis, we study BMI related topics. At first, we develop a computational method to predict BMI information from face images automatically. We formulate the BMI prediction from facial features as a machine vision problem. Three regression methods, including least square estimation, Gaussian processes for regression, and support vector regression are employed to predict the BMI value. Our preliminary results show that it is feasible to develop a computational system for BMI prediction from face images. Secondly, we address the influence of BMI changes on face identity. Both synthesized and real face images are assembled as the databases to facilitate our study. Empirically, we found that large BMI alterations can significantly reduce the matching accuracy of the face recognition system. Then we study if the influence of BMI changes can be reduced to improve the face recognition performance. The partial least squares (PLS) method is applied for this purpose. Experimental results show the feasibility to develop algorithms to address the influence of facial adiposity variations on face recognition, caused by BMI changes.;Makeup can affect facial appearance obviously. In the second part of this thesis, we deal with makeup influence on face identity. It is principal to perform makeup detection at first to address makeup influence. Four categories of features are proposed to characterize facial makeup cues in our study, including skin color tone, skin smoothness, texture, and highlight. A patch selection scheme and discriminative mapping are presented to enhance the performance of makeup detection. Secondly, we study dual attributes from makeup and non-makeup faces separately to reflect facial appearance changes caused by makeup in a semantic level. Cross-makeup attribute classification and accuracy change analysis is operated to divide dual attributes into four categories according to different makeup effects. To develop a face recognition system that is robust to facial makeup, PLS method is proposed on features extracted from local patches. We also propose a dual-attributes based method for face verification. Shared dual attributes can be used to measure facial similarity, rather than a direct matching with low-level features. Experimental results demonstrate the feasibility to eliminate the influence of makeup on face recognition.;In summary, contributions of this dissertation center in developing facial image analysis systems to deal with newly emerged topics effectively, i.e., BMI prediction, makeup detection, and the rcognition of face identity with makeup and BMI changes. In particular,to the best of our knowledge, BMI related topics, i.e., BMI prediction; the influence of BMI changes on face recognition; and face recognition robust to BMI changes are first explorations to the biometrics society

On Body Mass Index Analysis from Human Visual Appearance

In the past few decades, overweight and obesity are spreading widely like an epidemic. Generally, a person is considered overweight by body mass index (BMI). In addition to a body fat measurement, BMI is also a risk factor for many diseases, such as cardiovascular diseases, cancers and diabetes, etc. Therefore, BMI is important for personal health monitoring and medical research. Currently, BMI is measured in person with special devices. It is an urgent demand to explore conveniently preventive tools. This work investigates the feasibility of analyzing BMI from human visual appearances, including 2-dimensional (2D)/3-dimensional (3D) body and face data. Motivated by health science studies which have shown that anthropometric measures, such as waist-hip ratio, waist circumference, etc., are indicators for obesity, we analyze body weight from frontal view human body images. A framework is developed for body weight analysis from body images, along with the computation methods of five anthropometric features for body weight characterization. Then, we study BMI estimation from the 3D data by measuring the correlation between the estimated body volume and BMIs, and develop an efficient BMI computation method which consists of body weight and height estimation from normally dressed people in 3D space. We also intensively study BMI estimation from frontal view face images via two key aspects: facial representation extracting and BMI estimator learning. First, we investigate the visual BMI estimation problem from the aspect of the characteristics and performance of different facial representation extracting methods by three designed experiments. Then we study visual BMI estimation from facial images by a two-stage learning framework. BMI related facial features are learned in the first stage. To address the ambiguity of BMI labels, a label distribution based BMI estimator is proposed for the second stage. The experimental results show that this framework improves the performance step by step. Finally, to address the challenges caused by BMI data and labels, we integrate feature learning and estimator learning in one convolutional neural network (CNN). A label assignment matching scheme is proposed which successfully achieves an improvement in BMI estimation from face images

Machine Learning Approaches to Human Body Shape Analysis

Soft biometrics, biomedical sciences, and many other fields of study pay particular attention to the study of the geometric description of the human body, and its variations. Although multiple contributions, the interest is particularly high given the non-rigid nature of the human body, capable of assuming different poses, and numerous shapes due to variable body composition. Unfortunately, a well-known costly requirement in data-driven machine learning, and particularly in the human-based analysis, is the availability of data, in the form of geometric information (body measurements) with related vision information (natural images, 3D mesh, etc.). We introduce a computer graphics framework able to generate thousands of synthetic human body meshes, representing a population of individuals with stratified information: gender, Body Fat Percentage (BFP), anthropometric measurements, and pose. This contribution permits an extensive analysis of different bodies in different poses, avoiding the demanding, and expensive acquisition process. We design a virtual environment able to take advantage of the generated bodies, to infer the body surface area (BSA) from a single view. The framework permits to simulate the acquisition process of newly introduced RGB-D devices disentangling different noise components (sensor noise, optical distortion, body part occlusions). Common geometric descriptors in soft biometric, as well as in biomedical sciences, are based on body measurements. Unfortunately, as we prove, these descriptors are not pose invariant, constraining the usability in controlled scenarios. We introduce a differential geometry approach assuming body pose variations as isometric transformations of the body surface, and body composition changes covariant to the body surface area. This setting permits the use of the Laplace-Beltrami operator on the 2D body manifold, describing the body with a compact, efficient, and pose invariant representation. We design a neural network architecture able to infer important body semantics from spectral descriptors, closing the gap between abstract spectral features, and traditional measurement-based indices. Studying the manifold of body shapes, we propose an innovative generative adversarial model able to learn the body shapes. The method permits to generate new bodies with unseen geometries as a walk on the latent space, constituting a significant advantage over traditional generative methods