GRASS: Generative Recursive Autoencoders for Shape Structures

We introduce a novel neural network architecture for encoding and synthesis of 3D shapes, particularly their structures. Our key insight is that 3D shapes are effectively characterized by their hierarchical organization of parts, which reflects fundamental intra-shape relationships such as adjacency and symmetry. We develop a recursive neural net (RvNN) based autoencoder to map a flat, unlabeled, arbitrary part layout to a compact code. The code effectively captures hierarchical structures of man-made 3D objects of varying structural complexities despite being fixed-dimensional: an associated decoder maps a code back to a full hierarchy. The learned bidirectional mapping is further tuned using an adversarial setup to yield a generative model of plausible structures, from which novel structures can be sampled. Finally, our structure synthesis framework is augmented by a second trained module that produces fine-grained part geometry, conditioned on global and local structural context, leading to a full generative pipeline for 3D shapes. We demonstrate that without supervision, our network learns meaningful structural hierarchies adhering to perceptual grouping principles, produces compact codes which enable applications such as shape classification and partial matching, and supports shape synthesis and interpolation with significant variations in topology and geometry.Comment: Corresponding author: Kai Xu ([email protected]

Procedural Modeling and Physically Based Rendering for Synthetic Data Generation in Automotive Applications

We present an overview and evaluation of a new, systematic approach for generation of highly realistic, annotated synthetic data for training of deep neural networks in computer vision tasks. The main contribution is a procedural world modeling approach enabling high variability coupled with physically accurate image synthesis, and is a departure from the hand-modeled virtual worlds and approximate image synthesis methods used in real-time applications. The benefits of our approach include flexible, physically accurate and scalable image synthesis, implicit wide coverage of classes and features, and complete data introspection for annotations, which all contribute to quality and cost efficiency. To evaluate our approach and the efficacy of the resulting data, we use semantic segmentation for autonomous vehicles and robotic navigation as the main application, and we train multiple deep learning architectures using synthetic data with and without fine tuning on organic (i.e. real-world) data. The evaluation shows that our approach improves the neural network's performance and that even modest implementation efforts produce state-of-the-art results.Comment: The project web page at http://vcl.itn.liu.se/publications/2017/TKWU17/ contains a version of the paper with high-resolution images as well as additional materia

A Gravitational Theory of the Quantum

The synthesis of quantum and gravitational physics is sought through a finite, realistic, locally causal theory where gravity plays a vital role not only during decoherent measurement but also during non-decoherent unitary evolution. Invariant set theory is built on geometric properties of a compact fractal-like subset $I_U$ of cosmological state space on which the universe is assumed to evolve and from which the laws of physics are assumed to derive. Consistent with the primacy of $I_U$, a non-Euclidean (and hence non-classical) state-space metric $g_p$ is defined, related to the $p$-adic metric of number theory where $p$ is a large but finite Pythagorean prime. Uncertain states on $I_U$ are described using complex Hilbert states, but only if their squared amplitudes are rational and corresponding complex phase angles are rational multiples of $2 \pi$. Such Hilbert states are necessarily $g_p$-distant from states with either irrational squared amplitudes or irrational phase angles. The gappy fractal nature of $I_U$ accounts for quantum complementarity and is characterised numerically by a generic number-theoretic incommensurateness between rational angles and rational cosines of angles. The Bell inequality, whose violation would be inconsistent with local realism, is shown to be $g_p$-distant from all forms of the inequality that are violated in any finite-precision experiment. The delayed-choice paradox is resolved through the computational irreducibility of $I_U$. The Schr\"odinger and Dirac equations describe evolution on $I_U$ in the singular limit at $p=\infty$. By contrast, an extension of the Einstein field equations on $I_U$ is proposed which reduces smoothly to general relativity as $p \rightarrow \infty$. Novel proposals for the dark universe and the elimination of classical space-time singularities are given and experimental implications outlined