288 research outputs found
CSGNet: Neural Shape Parser for Constructive Solid Geometry
We present a neural architecture that takes as input a 2D or 3D shape and
outputs a program that generates the shape. The instructions in our program are
based on constructive solid geometry principles, i.e., a set of boolean
operations on shape primitives defined recursively. Bottom-up techniques for
this shape parsing task rely on primitive detection and are inherently slow
since the search space over possible primitive combinations is large. In
contrast, our model uses a recurrent neural network that parses the input shape
in a top-down manner, which is significantly faster and yields a compact and
easy-to-interpret sequence of modeling instructions. Our model is also more
effective as a shape detector compared to existing state-of-the-art detection
techniques. We finally demonstrate that our network can be trained on novel
datasets without ground-truth program annotations through policy gradient
techniques.Comment: Accepted at CVPR-201
Recommended from our members
Representation Learning for Shape Decomposition, By Shape Decomposition
The ability to parse 3D objects into their constituent parts is essential for humans to understand and interact with the surrounding world. Imparting this skill in machines is important for various computer graphics, computer vision, and robotics tasks. Machines endowed with this skill can better interact with its surroundings, perform shape editing, texturing, recomposing, tracking, and animation. In this thesis, we ask two questions. First, how can machines decompose 3D shapes into their fundamental parts? Second, does the ability to decompose the 3D shape into these parts help learn useful 3D shape representations?
In this thesis, we focus on parsing the shape into compact representations, such as parametric surface patches and Constructive Solid Geometry (CSG) primitives, which are also widely used representations in 3D modeling in computer graphics. Inspired by the advances in neural networks for 3D shape processing, we develop neural network approaches to tackle shape decomposition. First, we present CSGNet, a network architecture to parse shapes into CSG programs, which is trained using combination of supervised and reinforcement learning. Second, we present ParSeNet, a network architecture to decompose a shape into parametric surface patches (B-Spline) and geometric primitives (plane, cone, cylinder and sphere), trained on a large set of CAD models using supervised learning.
The training of deep neural network architectures for 3D recognition and generation tasks requires a large amount of labeled datasets. We explore ways to alleviate this problem by relying on shape decomposition methods to guide the learning process. Towards that end, we first study the use of freely available metadata, albeit inconsistent, from shape repositories to learn 3D shape features. Later we show that learning to decompose a 3D shape into geometric primitives also helps in learning shape representations useful for semantic segmentation tasks. Finally, since most 3D shapes encountered in real life are textured, consisting of several fine-grained semantic parts, we propose a method to learn fine-grained representations for textured 3D shapes in a self-supervised manner by incorporating 3D geometric priors
A Survey of Methods for Converting Unstructured Data to CSG Models
The goal of this document is to survey existing methods for recovering CSG
representations from unstructured data such as 3D point-clouds or polygon
meshes. We review and discuss related topics such as the segmentation and
fitting of the input data. We cover techniques from solid modeling and CAD for
polyhedron to CSG and B-rep to CSG conversion. We look at approaches coming
from program synthesis, evolutionary techniques (such as genetic programming or
genetic algorithm), and deep learning methods. Finally, we conclude with a
discussion of techniques for the generation of computer programs representing
solids (not just CSG models) and higher-level representations (such as, for
example, the ones based on sketch and extrusion or feature based operations).Comment: 29 page
From Perception to Programs: Regularize, Overparameterize, and Amortize
Toward combining inductive reasoning with perception abilities, we develop
techniques for neurosymbolic program synthesis where perceptual input is first
parsed by neural nets into a low-dimensional interpretable representation,
which is then processed by a synthesized program. We explore several techniques
for relaxing the problem and jointly learning all modules end-to-end with
gradient descent: multitask learning; amortized inference;
overparameterization; and a differentiable strategy for penalizing lengthy
programs. Collectedly this toolbox improves the stability of gradient-guided
program search, and suggests ways of learning both how to perceive input as
discrete abstractions, and how to symbolically process those abstractions as
programs.Comment: ICML 202
Recommended from our members
Controllable Neural Synthesis for Natural Images and Vector Art
Neural image synthesis approaches have become increasingly popular over the last years due to their ability to generate photorealistic images useful for several applications, such as digital entertainment, mixed reality, synthetic dataset creation, computer art, to name a few. Despite the progress over the last years, current approaches lack two important aspects: (a) they often fail to capture long-range interactions in the image, and as a result, they fail to generate scenes with complex dependencies between their different objects or parts. (b) they often ignore the underlying 3D geometry of the shape/scene in the image, and as a result, they frequently lose coherency and details.My thesis proposes novel solutions to the above problems. First, I propose a neural transformer architecture that captures long-range interactions and context for image synthesis at high resolutions, leading to synthesizing interesting phenomena in scenes, such as reflections of landscapes onto water or flora consistent with the rest of the landscape, that was not possible to generate reliably with previous ConvNet- and other transformer-based approaches. The key idea of the architecture is to sparsify the transformer\u27s attention matrix at high resolutions, guided by dense attention extracted at lower image resolution. I present qualitative and quantitative results, along with user studies, demonstrating the effectiveness of the method, and its superiority compared to the state-of-the-art. Second, I propose a method that generates artistic images with the guidance of input 3D shapes. In contrast to previous methods, the use of a geometric representation of 3D shape enables the synthesis of more precise stylized drawings with fewer artifacts. My method outputs the synthesized images in a vector representation, enabling richer downstream analysis or editing in interactive applications. I also show that the method produces substantially better results than existing image-based methods, in terms of predicting artists’ drawings and in user evaluation of results
QuadricsNet: Learning Concise Representation for Geometric Primitives in Point Clouds
This paper presents a novel framework to learn a concise geometric primitive
representation for 3D point clouds. Different from representing each type of
primitive individually, we focus on the challenging problem of how to achieve a
concise and uniform representation robustly. We employ quadrics to represent
diverse primitives with only 10 parameters and propose the first end-to-end
learning-based framework, namely QuadricsNet, to parse quadrics in point
clouds. The relationships between quadrics mathematical formulation and
geometric attributes, including the type, scale and pose, are insightfully
integrated for effective supervision of QuaidricsNet. Besides, a novel
pattern-comprehensive dataset with quadrics segments and objects is collected
for training and evaluation. Experiments demonstrate the effectiveness of our
concise representation and the robustness of QuadricsNet. Our code is available
at \url{https://github.com/MichaelWu99-lab/QuadricsNet}Comment: Submitted to ICRA 2024. 7 page
Intelligent techniques for automatic feature recognition in CAD models
The solutions suggested in this research are implemented in a prototype AFR system and its performance verified on commonly used benchmarking parts that are composed of machining features.EThOS - Electronic Theses Online ServiceGBUnited Kingdo
- …