SCULPTOR: Skeleton-Consistent Face Creation Using a Learned Parametric Generator

Recent years have seen growing interest in 3D human faces modelling due to its wide applications in digital human, character generation and animation. Existing approaches overwhelmingly emphasized on modeling the exterior shapes, textures and skin properties of faces, ignoring the inherent correlation between inner skeletal structures and appearance. In this paper, we present SCULPTOR, 3D face creations with Skeleton Consistency Using a Learned Parametric facial generaTOR, aiming to facilitate easy creation of both anatomically correct and visually convincing face models via a hybrid parametric-physical representation. At the core of SCULPTOR is LUCY, the first large-scale shape-skeleton face dataset in collaboration with plastic surgeons. Named after the fossils of one of the oldest known human ancestors, our LUCY dataset contains high-quality Computed Tomography (CT) scans of the complete human head before and after orthognathic surgeries, critical for evaluating surgery results. LUCY consists of 144 scans of 72 subjects (31 male and 41 female) where each subject has two CT scans taken pre- and post-orthognathic operations. Based on our LUCY dataset, we learn a novel skeleton consistent parametric facial generator, SCULPTOR, which can create the unique and nuanced facial features that help define a character and at the same time maintain physiological soundness. Our SCULPTOR jointly models the skull, face geometry and face appearance under a unified data-driven framework, by separating the depiction of a 3D face into shape blend shape, pose blend shape and facial expression blend shape. SCULPTOR preserves both anatomic correctness and visual realism in facial generation tasks compared with existing methods. Finally, we showcase the robustness and effectiveness of SCULPTOR in various fancy applications unseen before.Comment: 16 page, 13 fig

Data Driven Dense 3D Facial Reconstruction From 3D Skull Shape

Indiana University-Purdue University Indianapolis (IUPUI)This thesis explores a data driven machine learning based solution for Facial reconstruction from three dimensional (3D) skull shape for recognizing or identifying unknown subjects during forensic investigation. With over 8000 unidentified bodies during the past 3 decades, facial reconstruction of disintegrated bodies in helping with identification has been a critical issue for forensic practitioners. Historically, clay modelling has been used for facial reconstruction that not only requires an expert in the field but also demands a substantial amount of time for modelling, even after acquiring the skull model. Such manual reconstruction typically takes from a month to over 3 months of time and effort. The solution presented in this thesis uses 3D Cone Beam Computed Tomography (CBCT) data collected from many people to build a model of the relationship of facial skin to skull bone over a dense set of locations on the face. It then uses this skin-to-bone relationship model learned from the data to reconstruct the predicted face model from a skull shape of an unknown subject. The thesis also extends the algorithm in a way that could help modify the reconstructed face model interactively to account for the effects of age or weight. This uses the predicted face model as a starting point and creates different hypotheses of the facial appearances for different physical attributes. Attributes like age and body mass index (BMI) are used to show the physical facial appearance changes with the help of a tool we constructed. This could improve the identification process. The thesis also presents a methods designed for testing and validating the facial reconstruction algorithm

A machine learning approach to statistical shape models with applications to medical image analysis

Statistical shape models have become an indispensable tool for image analysis. The use of shape models is especially popular in computer vision and medical image analysis, where they were incorporated as a prior into a wide range of different algorithms. In spite of their big success, the study of statistical shape models has not received much attention in recent years. Shape models are often seen as an isolated technique, which merely consists of applying Principal Component Analysis to a set of example data sets. In this thesis we revisit statistical shape models and discuss their construction and applications from the perspective of machine learning and kernel methods. The shapes that belong to an object class are modeled as a Gaussian Process whose parameters are estimated from example data. This formulation puts statistical shape models in a much wider context and makes the powerful inference tools from learning theory applicable to shape modeling. Furthermore, the formulation is continuous and thus helps to avoid discretization issues, which often arise with discrete models. An important step in building statistical shape models is to establish surface correspondence. We discuss an approach which is based on kernel methods. This formulation allows us to integrate the statistical shape model as an additional prior. It thus unifies the methods of registration and shape model fitting. Using Gaussian Process regression we can integrate shape constraints in our model. These constraints can be used to enforce landmark matching in the fitting or correspondence problem. The same technique also leads directly to a new solution for shape reconstruction from partial data. In addition to experiments on synthetic 2D data sets, we show the applicability of our methods on real 3D medical data of the human head. In particular, we build a 3D model of the human skull, and present its applications for the planning of cranio-facial surgeries

Effective 3D Geometric Matching for Data Restoration and Its Forensic Application

3D geometric matching is the technique to detect the similar patterns among multiple objects. It is an important and fundamental problem and can facilitate many tasks in computer graphics and vision, including shape comparison and retrieval, data fusion, scene understanding and object recognition, and data restoration. For example, 3D scans of an object from different angles are matched and stitched together to form the complete geometry. In medical image analysis, the motion of deforming organs is modeled and predicted by matching a series of CT images. This problem is challenging and remains unsolved, especially when the similar patterns are 1) small and lack geometric saliency; 2) incomplete due to the occlusion of the scanning and damage of the data. We study the reliable matching algorithm that can tackle the above difficulties and its application in data restoration. Data restoration is the problem to restore the fragmented or damaged model to its original complete state. It is a new area and has direct applications in many scientific fields such as Forensics and Archeology. In this dissertation, we study novel effective geometric matching algorithms, including curve matching, surface matching, pairwise matching, multi-piece matching and template matching. We demonstrate its applications in an integrated digital pipeline of skull reassembly, skull completion, and facial reconstruction, which is developed to facilitate the state-of-the-art forensic skull/facial reconstruction processing pipeline in law enforcement

Virtual Morphology and Evolutionary Morphometrics in the new millenium.

EDITED VOLUME; abstract N/

Latent Disentanglement in Mesh Variational Autoencoders Improves the Diagnosis of Craniofacial Syndromes and Aids Surgical Planning

The use of deep learning to undertake shape analysis of the complexities of the human head holds great promise. However, there have traditionally been a number of barriers to accurate modelling, especially when operating on both a global and local level. In this work, we will discuss the application of the Swap Disentangled Variational Autoencoder (SD-VAE) with relevance to Crouzon, Apert and Muenke syndromes. Although syndrome classification is performed on the entire mesh, it is also possible, for the first time, to analyse the influence of each region of the head on the syndromic phenotype. By manipulating specific parameters of the generative model, and producing procedure-specific new shapes, it is also possible to simulate the outcome of a range of craniofacial surgical procedures. This opens new avenues to advance diagnosis, aids surgical planning and allows for the objective evaluation of surgical outcomes

A man from San Juan: facial approximation within anthropology

Dentro de la antropología y la identificación forense, la aproximación facial (también conocida como "reconstrucción facial") se presenta frecuentemente como un hecho consumado, con evidencia mínima de las investigaciones y los métodos usados para alcanzar los resultados. Este trabajo presenta un panorama detallado de la investigación y los métodos usados para realizar una aproximación facial grafica en 2D de un hombre prehistórico del valle de Jachal, provincia de San Juan, Argentina. Se entiende que corresponde a un miembro del extinto grupo Huarpe. El cráneo del individuo muestra numerosos rasgos morfológicos que son consistentes con lo que se conoce acerca de este grupo de agricultores tempranos americanos. Debido a que muchos de los métodos utilizados en esta reconstrucción facial aun no han sido verificados y aquellos que han sido debidamente probados se basan en promedios estadísticos de la variación humana, los resultados presentados deben ser vistos como indicativos de la posible apariencia facial del individuo dado el nivel actual de conocimiento, más que como un resultado definitivo.Within both anthropology and forensic identification, a facial approximation (also known as "facial reconstruction") is often presented as an accomplished fact, with minimal, or no evidence of the research and methods used to achieve the result. This paper presents a detailed overview of the research and methods used for a 2D computer graphic facial approximation of a prehistoric man unearthed in the Jachal Valley, San Juan Province, Argentina. Understood to be a member of the extinct Huarpe, this individual's skull displays many of the morphological features that are consistent with what is known about this group of early Amerindian farmers. Because many of the recommended methods that inform this facial approximation have yet to be verified, and those that have been appropriately tested are based on statistical averages of human variation, the results need to be viewed as indicative of this individual's possible facial appearance using current levels of knowledge, rather than as a definitive result.Asociación de Antropología Biológica de la República Argentin

A man from San Juan: facial approximation within anthropology

Dentro de la antropología y la identificación forense, la aproximación facial (también conocida como "reconstrucción facial") se presenta frecuentemente como un hecho consumado, con evidencia mínima de las investigaciones y los métodos usados para alcanzar los resultados. Este trabajo presenta un panorama detallado de la investigación y los métodos usados para realizar una aproximación facial grafica en 2D de un hombre prehistórico del valle de Jachal, provincia de San Juan, Argentina. Se entiende que corresponde a un miembro del extinto grupo Huarpe. El cráneo del individuo muestra numerosos rasgos morfológicos que son consistentes con lo que se conoce acerca de este grupo de agricultores tempranos americanos. Debido a que muchos de los métodos utilizados en esta reconstrucción facial aun no han sido verificados y aquellos que han sido debidamente probados se basan en promedios estadísticos de la variación humana, los resultados presentados deben ser vistos como indicativos de la posible apariencia facial del individuo dado el nivel actual de conocimiento, más que como un resultado definitivo.Within both anthropology and forensic identification, a facial approximation (also known as "facial reconstruction") is often presented as an accomplished fact, with minimal, or no evidence of the research and methods used to achieve the result. This paper presents a detailed overview of the research and methods used for a 2D computer graphic facial approximation of a prehistoric man unearthed in the Jachal Valley, San Juan Province, Argentina. Understood to be a member of the extinct Huarpe, this individual's skull displays many of the morphological features that are consistent with what is known about this group of early Amerindian farmers. Because many of the recommended methods that inform this facial approximation have yet to be verified, and those that have been appropriately tested are based on statistical averages of human variation, the results need to be viewed as indicative of this individual's possible facial appearance using current levels of knowledge, rather than as a definitive result.Asociación de Antropología Biológica de la República Argentin