Mean value coordinates–based caricature and expression synthesis

We present a novel method for caricature synthesis based on mean value coordinates (MVC). Our method can be applied to any single frontal face image to learn a specified caricature face pair for frontal and 3D caricature synthesis. This technique only requires one or a small number of exemplar pairs and a natural frontal face image training set, while the system can transfer the style of the exemplar pair across individuals. Further exaggeration can be fulfilled in a controllable way. Our method is further applied to facial expression transfer, interpolation, and exaggeration, which are applications of expression editing. Additionally, we have extended our approach to 3D caricature synthesis based on the 3D version of MVC. With experiments we demonstrate that the transferred expressions are credible and the resulting caricatures can be characterized and recognized

Cryo-EM map interpretation and protein model-building using iterative map segmentation.

A procedure for building protein chains into maps produced by single-particle electron cryo-microscopy (cryo-EM) is described. The procedure is similar to the way an experienced structural biologist might analyze a map, focusing first on secondary structure elements such as helices and sheets, then varying the contour level to identify connections between these elements. Since the high density in a map typically follows the main-chain of the protein, the main-chain connection between secondary structure elements can often be identified as the unbranched path between them with the highest minimum value along the path. This chain-tracing procedure is then combined with finding side-chain positions based on the presence of density extending away from the main path of the chain, allowing generation of a Cα model. The Cα model is converted to an all-atom model and is refined against the map. We show that this procedure is as effective as other existing methods for interpretation of cryo-EM maps and that it is considerably faster and produces models with fewer chain breaks than our previous methods that were based on approaches developed for crystallographic maps

Automatic generation of subject-specific finite element models of the spine from magnetic resonance images

The generation of subject-specific finite element models of the spine is generally a time-consuming process based on computed tomography (CT) images, where scanning exposes subjects to harmful radiation. In this study, a method is presented for the automatic generation of spine finite element models using images from a single magnetic resonance (MR) sequence. The thoracic and lumbar spine of eight adult volunteers was imaged using a 3D multi-echogradient-echo sagittal MR sequence. A deep-learning method was used to generate synthetic CT images from the MR images. A pre-trained deeplearning network was used for the automatic segmentation of vertebrae from the synthetic CT images. Another deep-learning network was trained for the automatic segmentation of intervertebral discs from the MR images. The automatic segmentations were validated against manual segmentations for two subjects, one with scoliosis, and another with a spine implant. A template mesh of the spine was registered to the segmentations in three steps using a Bayesian coherent point drift algorithm. First, rigid registration was applied on the complete spine. Second, non-rigid registration was used for the individual discs and vertebrae. Third, the complete spine was non-rigidly registered to the individually registered discs and vertebrae. Comparison of the automatic and manual segmentations led to dice-scores of 0.93–0.96 for all vertebrae and discs. The lowest dice-score was in the disc at the height of the implant where artifacts led to under-segmentation. The mean distance between the morphed meshes and the segmentations was below 1 mm. In conclusion, the presented method can be used to automatically generate accurate subject-specific spine models

Handbook of Digital Face Manipulation and Detection

This open access book provides the first comprehensive collection of studies dealing with the hot topic of digital face manipulation such as DeepFakes, Face Morphing, or Reenactment. It combines the research fields of biometrics and media forensics including contributions from academia and industry. Appealing to a broad readership, introductory chapters provide a comprehensive overview of the topic, which address readers wishing to gain a brief overview of the state-of-the-art. Subsequent chapters, which delve deeper into various research challenges, are oriented towards advanced readers. Moreover, the book provides a good starting point for young researchers as well as a reference guide pointing at further literature. Hence, the primary readership is academic institutions and industry currently involved in digital face manipulation and detection. The book could easily be used as a recommended text for courses in image processing, machine learning, media forensics, biometrics, and the general security area

Automatically Controlled Morphing of 2D Shapes with Textures

This paper deals with 2D image transformations from a perspective of a 3D heterogeneous shape modeling and computer animation. Shape and image morphing techniques have attracted a lot of attention in artistic design, computer animation, and interactive and streaming applications. We present a novel method for morphing between two topologically arbitrary 2D shapes with sophisticated textures (raster color attributes) using a metamorphosis technique called space-time blending (STB) coupled with space-time transfinite interpolation. The method allows for a smooth transition between source and target objects by generating in-between shapes and associated textures without setting any correspondences between boundary points or features. The method requires no preprocessing and can be applied in 2D animation when position and topology of source and target objects are significantly different. With the conversion of given 2D shapes to signed distance fields, we have detected a number of problems with directly applying STB to them. We propose a set of novel and mathematically substantiated techniques, providing automatic control of the morphing process with STB and an algorithm of applying those techniques in combination. We illustrate our method with applications in 2D animation and interactive applications

The Development And Application Of A Statistical Shape Model Of The Human Craniofacial Skeleton

Biomechanical investigations involving the characterization of biomaterials or improvement of implant design often employ finite element (FE) analysis. However, the contemporary method of developing a FE mesh from computed tomography scans involves much manual intervention and can be a tedious process. Researchers will often focus their efforts on creating a single highly validated FE model at the expense of incorporating variability of anatomical geometry and material properties, thus limiting the applicability of their findings. The goal of this thesis was to address this issue through the use of a statistical shape model (SSM). A SSM is a probabilistic description of the variation in the shape of a given class of object. (Additional scalar data, such as an elastic constant, can also be incorporated into the model.) By discretizing a sample (i.e. training set) of unique objects of the same class using a set of corresponding nodes, the main modes of shape variation within that shape class are discovered via principal component analysis. By combining the principal components using different linear combinations, new shape instances are created, each with its own unique geometry while retaining the characteristics of its shape class. In this thesis, FE models of the human craniofacial skeleton (CFS) were first validated to establish their viability. A mesh morphing procedure was then developed to map one mesh onto the geometry of 22 other CFS models forming a training set for a SSM of the CFS. After verifying that FE results derived from morphed meshes were no different from those obtained using meshes created with contemporary methods, a SSM of the human CFS was created, and 1000 CFS FE meshes produced. It was found that these meshes accurately described the geometric variation in human population, and were used in a Monte Carlo analysis of facial fracture, finding past studies attempting to characterize the fracture probability of the zygomatic bone are overly conservative