We target a 3D generative model for general natural scenes that are typically
unique and intricate. Lacking the necessary volumes of training data, along
with the difficulties of having ad hoc designs in presence of varying scene
characteristics, renders existing setups intractable. Inspired by classical
patch-based image models, we advocate for synthesizing 3D scenes at the patch
level, given a single example. At the core of this work lies important
algorithmic designs w.r.t the scene representation and generative patch
nearest-neighbor module, that address unique challenges arising from lifting
classical 2D patch-based framework to 3D generation. These design choices, on a
collective level, contribute to a robust, effective, and efficient model that
can generate high-quality general natural scenes with both realistic geometric
structure and visual appearance, in large quantities and varieties, as
demonstrated upon a variety of exemplar scenes.Comment: 23 pages, 26 figures, accepted by CVPR 2023. Project page:
http://weiyuli.xyz/Sin3DGen