174 research outputs found
DeepSketchHair: Deep Sketch-based 3D Hair Modeling
We present sketchhair, a deep learning based tool for interactive modeling of
3D hair from 2D sketches. Given a 3D bust model as reference, our sketching
system takes as input a user-drawn sketch (consisting of hair contour and a few
strokes indicating the hair growing direction within a hair region), and
automatically generates a 3D hair model, which matches the input sketch both
globally and locally. The key enablers of our system are two carefully designed
neural networks, namely, S2ONet, which converts an input sketch to a dense 2D
hair orientation field; and O2VNet, which maps the 2D orientation field to a 3D
vector field. Our system also supports hair editing with additional sketches in
new views. This is enabled by another deep neural network, V2VNet, which
updates the 3D vector field with respect to the new sketches. All the three
networks are trained with synthetic data generated from a 3D hairstyle
database. We demonstrate the effectiveness and expressiveness of our tool using
a variety of hairstyles and also compare our method with prior art
Tag-based annotation creates better avatars
Avatar creation from human images allows users to customize their digital
figures in different styles. Existing rendering systems like Bitmoji,
MetaHuman, and Google Cartoonset provide expressive rendering systems that
serve as excellent design tools for users. However, twenty-plus parameters,
some including hundreds of options, must be tuned to achieve ideal results.
Thus it is challenging for users to create the perfect avatar. A machine
learning model could be trained to predict avatars from images, however the
annotators who label pairwise training data have the same difficulty as users,
causing high label noise. In addition, each new rendering system or version
update requires thousands of new training pairs. In this paper, we propose a
Tag-based annotation method for avatar creation. Compared to direct annotation
of labels, the proposed method: produces higher annotator agreements, causes
machine learning to generates more consistent predictions, and only requires a
marginal cost to add new rendering systems.Comment: 15 pages, 7 figures, 4 table
GroomGen: A High-Quality Generative Hair Model Using Hierarchical Latent Representations
Despite recent successes in hair acquisition that fits a high-dimensional
hair model to a specific input subject, generative hair models, which establish
general embedding spaces for encoding, editing, and sampling diverse
hairstyles, are way less explored. In this paper, we present GroomGen, the
first generative model designed for hair geometry composed of highly-detailed
dense strands. Our approach is motivated by two key ideas. First, we construct
hair latent spaces covering both individual strands and hairstyles. The latent
spaces are compact, expressive, and well-constrained for high-quality and
diverse sampling. Second, we adopt a hierarchical hair representation that
parameterizes a complete hair model to three levels: single strands, sparse
guide hairs, and complete dense hairs. This representation is critical to the
compactness of latent spaces, the robustness of training, and the efficiency of
inference. Based on this hierarchical latent representation, our proposed
pipeline consists of a strand-VAE and a hairstyle-VAE that encode an individual
strand and a set of guide hairs to their respective latent spaces, and a hybrid
densification step that populates sparse guide hairs to a dense hair model.
GroomGen not only enables novel hairstyle sampling and plausible hairstyle
interpolation, but also supports interactive editing of complex hairstyles, or
can serve as strong data-driven prior for hairstyle reconstruction from images.
We demonstrate the superiority of our approach with qualitative examples of
diverse sampled hairstyles and quantitative evaluation of generation quality
regarding every single component and the entire pipeline.Comment: SIGGRAPH Asia 202
Image-Based Approaches to Hair Modeling
Hair is a relevant characteristic of virtual characters, therefore the modeling of plausible facial hair and hairstyles is an essential step in the generation of computer generated (CG) avatars. However, the inherent geometric complexity of hair together with the huge number of filaments of an average human head make the task of modeling hairstyles a very challenging one. To date this is commonly a manual process which requires artist skills or very specialized and costly acquisition software. In this work we present an image-based approach to model facial hair (beard and eyebrows) and (head) hairstyles. Since facial hair is usually much shorter than the average head hair two different methods are resented, adapted to the characteristics of the hair to be modeled. Facial hair is modeled using data extracted from facial texture images and missing information is inferred by means of a database-driven prior model. Our hairstyle reconstruction technique employs images of the hair to be modeled taken with a thermal camera. The major advantage of our thermal image-based method over conventional image-based techniques lies on the fact that during data capture the hairstyle is "lit from the inside": the thermal camera captures heat irradiated by the head and actively re-emitted by the hair filaments almost isotropically. Following this approach we can avoid several issues of conventional image-based techniques, like shadowing or anisotropy in reflectance. The presented technique requires minimal user interaction and a simple acquisition setup. Several challenging examples demonstrate the potential of the proposed approach
Automatic Animation of Hair Blowing in Still Portrait Photos
We propose a novel approach to animate human hair in a still portrait photo.
Existing work has largely studied the animation of fluid elements such as water
and fire. However, hair animation for a real image remains underexplored, which
is a challenging problem, due to the high complexity of hair structure and
dynamics. Considering the complexity of hair structure, we innovatively treat
hair wisp extraction as an instance segmentation problem, where a hair wisp is
referred to as an instance. With advanced instance segmentation networks, our
method extracts meaningful and natural hair wisps. Furthermore, we propose a
wisp-aware animation module that animates hair wisps with pleasing motions
without noticeable artifacts. The extensive experiments show the superiority of
our method. Our method provides the most pleasing and compelling viewing
experience in the qualitative experiments and outperforms state-of-the-art
still-image animation methods by a large margin in the quantitative evaluation.
Project url: \url{https://nevergiveu.github.io/AutomaticHairBlowing/}Comment: Accepted to ICCV 202
Computer-assisted animation creation techniques for hair animation and shade, highlight, and shadow
制度:新 ; 報告番号:甲3062号 ; 学位の種類:博士(工学) ; 授与年月日:2010/2/25 ; 早大学位記番号:新532
InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
This paper describes InfoGAN, an information-theoretic extension to the
Generative Adversarial Network that is able to learn disentangled
representations in a completely unsupervised manner. InfoGAN is a generative
adversarial network that also maximizes the mutual information between a small
subset of the latent variables and the observation. We derive a lower bound to
the mutual information objective that can be optimized efficiently, and show
that our training procedure can be interpreted as a variation of the Wake-Sleep
algorithm. Specifically, InfoGAN successfully disentangles writing styles from
digit shapes on the MNIST dataset, pose from lighting of 3D rendered images,
and background digits from the central digit on the SVHN dataset. It also
discovers visual concepts that include hair styles, presence/absence of
eyeglasses, and emotions on the CelebA face dataset. Experiments show that
InfoGAN learns interpretable representations that are competitive with
representations learned by existing fully supervised methods
TeCH: Text-guided Reconstruction of Lifelike Clothed Humans
Despite recent research advancements in reconstructing clothed humans from a
single image, accurately restoring the "unseen regions" with high-level details
remains an unsolved challenge that lacks attention. Existing methods often
generate overly smooth back-side surfaces with a blurry texture. But how to
effectively capture all visual attributes of an individual from a single image,
which are sufficient to reconstruct unseen areas (e.g., the back view)?
Motivated by the power of foundation models, TeCH reconstructs the 3D human by
leveraging 1) descriptive text prompts (e.g., garments, colors, hairstyles)
which are automatically generated via a garment parsing model and Visual
Question Answering (VQA), 2) a personalized fine-tuned Text-to-Image diffusion
model (T2I) which learns the "indescribable" appearance. To represent
high-resolution 3D clothed humans at an affordable cost, we propose a hybrid 3D
representation based on DMTet, which consists of an explicit body shape grid
and an implicit distance field. Guided by the descriptive prompts +
personalized T2I diffusion model, the geometry and texture of the 3D humans are
optimized through multi-view Score Distillation Sampling (SDS) and
reconstruction losses based on the original observation. TeCH produces
high-fidelity 3D clothed humans with consistent & delicate texture, and
detailed full-body geometry. Quantitative and qualitative experiments
demonstrate that TeCH outperforms the state-of-the-art methods in terms of
reconstruction accuracy and rendering quality. The code will be publicly
available for research purposes at https://huangyangyi.github.io/TeCHComment: Project: https://huangyangyi.github.io/TeCH, Code:
https://github.com/huangyangyi/TeC
- …