31,756 research outputs found
3inGAN: Learning a 3D Generative Model from Images of a Self-similar Scene
We introduce 3inGAN, an unconditional 3D generative model trained from 2D
images of a single self-similar 3D scene. Such a model can be used to produce
3D "remixes" of a given scene, by mapping spatial latent codes into a 3D
volumetric representation, which can subsequently be rendered from arbitrary
views using physically based volume rendering. By construction, the generated
scenes remain view-consistent across arbitrary camera configurations, without
any flickering or spatio-temporal artifacts. During training, we employ a
combination of 2D, obtained through differentiable volume tracing, and 3D
Generative Adversarial Network (GAN) losses, across multiple scales, enforcing
realism on both its 3D structure and the 2D renderings. We show results on
semi-stochastic scenes of varying scale and complexity, obtained from real and
synthetic sources. We demonstrate, for the first time, the feasibility of
learning plausible view-consistent 3D scene variations from a single exemplar
scene and provide qualitative and quantitative comparisons against recent
related methods.Comment: Conference accept at 3DV 202
Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatars
3D-aware generative adversarial networks (GANs) synthesize high-fidelity and
multi-view-consistent facial images using only collections of single-view 2D
imagery. Towards fine-grained control over facial attributes, recent efforts
incorporate 3D Morphable Face Model (3DMM) to describe deformation in
generative radiance fields either explicitly or implicitly. Explicit methods
provide fine-grained expression control but cannot handle topological changes
caused by hair and accessories, while implicit ones can model varied topologies
but have limited generalization caused by the unconstrained deformation fields.
We propose a novel 3D GAN framework for unsupervised learning of generative,
high-quality and 3D-consistent facial avatars from unstructured 2D images. To
achieve both deformation accuracy and topological flexibility, we propose a 3D
representation called Generative Texture-Rasterized Tri-planes. The proposed
representation learns Generative Neural Textures on top of parametric mesh
templates and then projects them into three orthogonal-viewed feature planes
through rasterization, forming a tri-plane feature representation for volume
rendering. In this way, we combine both fine-grained expression control of
mesh-guided explicit deformation and the flexibility of implicit volumetric
representation. We further propose specific modules for modeling mouth interior
which is not taken into account by 3DMM. Our method demonstrates
state-of-the-art 3D-aware synthesis quality and animation ability through
extensive experiments. Furthermore, serving as 3D prior, our animatable 3D
representation boosts multiple applications including one-shot facial avatars
and 3D-aware stylization.Comment: Project page: https://mrtornado24.github.io/Next3D
EVA3D: Compositional 3D Human Generation from 2D Image Collections
Inverse graphics aims to recover 3D models from 2D observations. Utilizing
differentiable rendering, recent 3D-aware generative models have shown
impressive results of rigid object generation using 2D images. However, it
remains challenging to generate articulated objects, like human bodies, due to
their complexity and diversity in poses and appearances. In this work, we
propose, EVA3D, an unconditional 3D human generative model learned from 2D
image collections only. EVA3D can sample 3D humans with detailed geometry and
render high-quality images (up to 512x256) without bells and whistles (e.g.
super resolution). At the core of EVA3D is a compositional human NeRF
representation, which divides the human body into local parts. Each part is
represented by an individual volume. This compositional representation enables
1) inherent human priors, 2) adaptive allocation of network parameters, 3)
efficient training and rendering. Moreover, to accommodate for the
characteristics of sparse 2D human image collections (e.g. imbalanced pose
distribution), we propose a pose-guided sampling strategy for better GAN
learning. Extensive experiments validate that EVA3D achieves state-of-the-art
3D human generation performance regarding both geometry and texture quality.
Notably, EVA3D demonstrates great potential and scalability to
"inverse-graphics" diverse human bodies with a clean framework.Comment: Project Page at https://hongfz16.github.io/projects/EVA3D.htm
- …