45 research outputs found
Seeing a Rose in Five Thousand Ways
What is a rose, visually? A rose comprises its intrinsics, including the
distribution of geometry, texture, and material specific to its object
category. With knowledge of these intrinsic properties, we may render roses of
different sizes and shapes, in different poses, and under different lighting
conditions. In this work, we build a generative model that learns to capture
such object intrinsics from a single image, such as a photo of a bouquet. Such
an image includes multiple instances of an object type. These instances all
share the same intrinsics, but appear different due to a combination of
variance within these intrinsics and differences in extrinsic factors, such as
pose and illumination. Experiments show that our model successfully learns
object intrinsics (distribution of geometry, texture, and material) for a wide
range of objects, each from a single Internet image. Our method achieves
superior results on multiple downstream tasks, including intrinsic image
decomposition, shape and image generation, view synthesis, and relighting.Comment: Project page: https://cs.stanford.edu/~yzzhang/projects/rose
Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields
Capitalizing on the recent advances in image generation models, existing
controllable face image synthesis methods are able to generate high-fidelity
images with some levels of controllability, e.g., controlling the shapes,
expressions, textures, and poses of the generated face images. However, these
methods focus on 2D image generative models, which are prone to producing
inconsistent face images under large expression and pose changes. In this
paper, we propose a new NeRF-based conditional 3D face synthesis framework,
which enables 3D controllability over the generated face images by imposing
explicit 3D conditions from 3D face priors. At its core is a conditional
Generative Occupancy Field (cGOF) that effectively enforces the shape of the
generated face to commit to a given 3D Morphable Model (3DMM) mesh. To achieve
accurate control over fine-grained 3D face shapes of the synthesized image, we
additionally incorporate a 3D landmark loss as well as a volume warping loss
into our synthesis algorithm. Experiments validate the effectiveness of the
proposed method, which is able to generate high-fidelity face images and shows
more precise 3D controllability than state-of-the-art 2D-based controllable
face synthesis methods. Find code and demo at
https://keqiangsun.github.io/projects/cgof
CGOF++: Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields
Capitalizing on the recent advances in image generation models, existing
controllable face image synthesis methods are able to generate high-fidelity
images with some levels of controllability, e.g., controlling the shapes,
expressions, textures, and poses of the generated face images. However,
previous methods focus on controllable 2D image generative models, which are
prone to producing inconsistent face images under large expression and pose
changes. In this paper, we propose a new NeRF-based conditional 3D face
synthesis framework, which enables 3D controllability over the generated face
images by imposing explicit 3D conditions from 3D face priors. At its core is a
conditional Generative Occupancy Field (cGOF++) that effectively enforces the
shape of the generated face to conform to a given 3D Morphable Model (3DMM)
mesh, built on top of EG3D [1], a recent tri-plane-based generative model. To
achieve accurate control over fine-grained 3D face shapes of the synthesized
images, we additionally incorporate a 3D landmark loss as well as a volume
warping loss into our synthesis framework. Experiments validate the
effectiveness of the proposed method, which is able to generate high-fidelity
face images and shows more precise 3D controllability than state-of-the-art
2D-based controllable face synthesis methods.Comment: This article is an extension of the NeurIPS'22 paper arXiv:2206.0836
Stanford-ORB: A Real-World 3D Object Inverse Rendering Benchmark
We introduce Stanford-ORB, a new real-world 3D Object inverse Rendering
Benchmark. Recent advances in inverse rendering have enabled a wide range of
real-world applications in 3D content generation, moving rapidly from research
and commercial use cases to consumer devices. While the results continue to
improve, there is no real-world benchmark that can quantitatively assess and
compare the performance of various inverse rendering methods. Existing
real-world datasets typically only consist of the shape and multi-view images
of objects, which are not sufficient for evaluating the quality of material
recovery and object relighting. Methods capable of recovering material and
lighting often resort to synthetic data for quantitative evaluation, which on
the other hand does not guarantee generalization to complex real-world
environments. We introduce a new dataset of real-world objects captured under a
variety of natural scenes with ground-truth 3D scans, multi-view images, and
environment lighting. Using this dataset, we establish the first comprehensive
real-world evaluation benchmark for object inverse rendering tasks from
in-the-wild scenes, and compare the performance of various existing methods.Comment: NeurIPS 2023 Datasets and Benchmarks Track. The first two authors
contributed equally to this work. Project page:
https://stanfordorb.github.io
Evolutionary origin of a tetraploid allium species in the Qinghai-Tibet Plateau
Extinct taxa may be detectable if they were ancestors to extant hybrid species, which retain their genetic signature. In this study, we combined phylogenomics, population genetics and fluorescence in situ hybridization (GISH and FISH) analyses to trace the origin of the alpine tetraploid Allium tetraploideum (2n = 4x = 32), one of the five known members in the subgenus Cyathophora. We found that A. tetraploideum was an obvious allotetrapoploid derived from ancestors including at least two closely related diploid species, A. farreri and A. cyathophorum, from which it differs by multiple ecological and genomic attributes. However, these two species cannot account for the full genome of A. tetraploideum, indicating that at least one extinct diploid is also involved in its ancestry. Furthermore, A. tetraploideum appears to have arisen via homoploid hybrid speciation (HHS) from two extinct allotetraploid parents, which derived in turn from the aforementioned diploids. Other modes of origin were possible, but all were even more complex and involved additional extinct ancestors. Our study together highlights how some polyploid species might have very complex origins, involving both HHS and polyploid speciation and also extinct ancestors.</p
The third international hackathon for applying insights into large-scale genomic composition to use cases in a wide range of organisms
publishedVersio