8 research outputs found
DSM-Net: Disentangled Structured Mesh Net for Controllable Generation of Fine Geometry
3D shape generation is a fundamental operation in computer graphics. While
significant progress has been made, especially with recent deep generative
models, it remains a challenge to synthesize high-quality geometric shapes with
rich detail and complex structure, in a controllable manner. To tackle this, we
introduce DSM-Net, a deep neural network that learns a disentangled structured
mesh representation for 3D shapes, where two key aspects of shapes, geometry
and structure, are encoded in a synergistic manner to ensure plausibility of
the generated shapes, while also being disentangled as much as possible. This
supports a range of novel shape generation applications with intuitive control,
such as interpolation of structure (geometry) while keeping geometry
(structure) unchanged. To achieve this, we simultaneously learn structure and
geometry through variational autoencoders (VAEs) in a hierarchical manner for
both, with bijective mappings at each level. In this manner we effectively
encode geometry and structure in separate latent spaces, while ensuring their
compatibility: the structure is used to guide the geometry and vice versa. At
the leaf level, the part geometry is represented using a conditional part VAE,
to encode high-quality geometric details, guided by the structure context as
the condition. Our method not only supports controllable generation
applications, but also produces high-quality synthesized shapes, outperforming
state-of-the-art methods
DSG-Net: Learning disentangled structure and geometry for 3D shape generation
3D shape generation is a fundamental operation in computer graphics. While significant progress has been made, especially with recent deep generative models, it remains a challenge to synthesize high-quality shapes with rich geometric details and complex structures, in a controllable manner. To tackle this, we introduce DSG-Net, a deep neural network that learns a disentangled structured & geometric mesh representation for 3D shapes, where two key aspects of shapes, geometry and structure, are encoded in a synergistic manner to ensure plausibility of the generated shapes, while also being disentangled as much as possible. This supports a range of novel shape generation applications with disentangled control, such as interpolation of structure (geometry) while keeping geometry (structure) unchanged. To achieve this, we simultaneously learn structure and geometry through variational autoencoders (VAEs) in a hierarchical manner for both, with bijective mappings at each level. In this manner, we effectively encode geometry and structure in separate latent spaces, while ensuring their compatibility: the structure is used to guide the geometry and vice versa. At the leaf level, the part geometry is represented using a conditional part VAE, to encode high-quality geometric details, guided by the structure context as the condition. Our method not only supports controllable generation applications, but also produces high-quality synthesized shapes, outperforming state-of-the-art methods
StructureNet: Hierarchical Graph Networks for 3D Shape Generation
The ability to generate novel, diverse, and realistic 3D shapes along with
associated part semantics and structure is central to many applications
requiring high-quality 3D assets or large volumes of realistic training data. A
key challenge towards this goal is how to accommodate diverse shape variations,
including both continuous deformations of parts as well as structural or
discrete alterations which add to, remove from, or modify the shape
constituents and compositional structure. Such object structure can typically
be organized into a hierarchy of constituent object parts and relationships,
represented as a hierarchy of n-ary graphs. We introduce StructureNet, a
hierarchical graph network which (i) can directly encode shapes represented as
such n-ary graphs; (ii) can be robustly trained on large and complex shape
families; and (iii) can be used to generate a great diversity of realistic
structured shape geometries. Technically, we accomplish this by drawing
inspiration from recent advances in graph neural networks to propose an
order-invariant encoding of n-ary graphs, considering jointly both part
geometry and inter-part relations during network training. We extensively
evaluate the quality of the learned latent spaces for various shape families
and show significant advantages over baseline and competing methods. The
learned latent spaces enable several structure-aware geometry processing
applications, including shape generation and interpolation, shape editing, or
shape structure discovery directly from un-annotated images, point clouds, or
partial scans.Comment: Conditionally Accepted to Siggraph Asia 201