Recent years have seen growing interest in 3D human faces modelling due to
its wide applications in digital human, character generation and animation.
Existing approaches overwhelmingly emphasized on modeling the exterior shapes,
textures and skin properties of faces, ignoring the inherent correlation
between inner skeletal structures and appearance. In this paper, we present
SCULPTOR, 3D face creations with Skeleton Consistency Using a Learned
Parametric facial generaTOR, aiming to facilitate easy creation of both
anatomically correct and visually convincing face models via a hybrid
parametric-physical representation. At the core of SCULPTOR is LUCY, the first
large-scale shape-skeleton face dataset in collaboration with plastic surgeons.
Named after the fossils of one of the oldest known human ancestors, our LUCY
dataset contains high-quality Computed Tomography (CT) scans of the complete
human head before and after orthognathic surgeries, critical for evaluating
surgery results. LUCY consists of 144 scans of 72 subjects (31 male and 41
female) where each subject has two CT scans taken pre- and post-orthognathic
operations. Based on our LUCY dataset, we learn a novel skeleton consistent
parametric facial generator, SCULPTOR, which can create the unique and nuanced
facial features that help define a character and at the same time maintain
physiological soundness. Our SCULPTOR jointly models the skull, face geometry
and face appearance under a unified data-driven framework, by separating the
depiction of a 3D face into shape blend shape, pose blend shape and facial
expression blend shape. SCULPTOR preserves both anatomic correctness and visual
realism in facial generation tasks compared with existing methods. Finally, we
showcase the robustness and effectiveness of SCULPTOR in various fancy
applications unseen before.Comment: 16 page, 13 fig