3D Shape Understanding and Generation

In recent years, Machine Learning techniques have revolutionized solutions to longstanding image-based problems, like image classification, generation, semantic segmentation, object detection and many others. However, if we want to be able to build agents that can successfully interact with the real world, those techniques need to be capable of reasoning about the world as it truly is: a tridimensional space. There are two main challenges while handling 3D information in machine learning models. First, it is not clear what is the best 3D representation. For images, convolutional neural networks (CNNs) operating on raster images yield the best results in virtually all image-based benchmarks. For 3D data, the best combination of model and representation is still an open question. Second, 3D data is not available on the same scale as images – taking pictures is a common procedure in our daily lives, whereas capturing 3D content is an activity usually restricted to specialized professionals. This thesis is focused on addressing both of these issues. Which model and representation should we use for generating and recognizing 3D data? What are efficient ways of learning 3D representations from a few examples? Is it possible to leverage image data to build models capable of reasoning about the world in 3D? Our research findings show that it is possible to build models that efficiently generate 3D shapes as irregularly structured representations. Those models require significantly less memory while generating higher quality shapes than the ones based on voxels and multi-view representations. We start by developing techniques to generate shapes represented as point clouds. This class of models leads to high quality reconstructions and better unsupervised feature learning. However, since point clouds are not amenable to editing and human manipulation, we also present models capable of generating shapes as sets of shape handles -- simpler primitives that summarize complex 3D shapes and were specifically designed for high-level tasks and user interaction. Despite their effectiveness, those approaches require some form of 3D supervision, which is scarce. We present multiple alternatives to this problem. First, we investigate how approximate convex decomposition techniques can be used as self-supervision to improve recognition models when only a limited number of labels are available. Second, we study how neural network architectures induce shape priors that can be used in multiple reconstruction tasks -- using both volumetric and manifold representations. In this regime, reconstruction is performed from a single example -- either a sparse point cloud or multiple silhouettes. Finally, we demonstrate how to train generative models of 3D shapes without using any 3D supervision by combining differentiable rendering techniques and Generative Adversarial Networks

Direct volume rendering of unstructured grids

This paper investigates three categories of algorithms for direct volume rendering of unstructured grids, which are image-space, object-space, and hybrid methods. We propose three new algorithms. Cell Projection algorithm, which falls into object-space category, is capable of rendering non-convex meshes through a simple yet efficient sorting schema that exploits both image and object space coherencies. Existing hybrid methods use object-then-image traversal order that enforces the processing of each cell. Thus, these algorithms perform redundant operations and do not support early ray termination. We propose a hybrid method, called Span-Buffer Ray Casting (SBRC), that can support early ray termination discarding redundant operations by employing image-then-object traversal order. Another hybrid method, called Koyamada-SBRC (K-SBRC), is proposed with the motivation of refining image-space and hybrid methods to extract the best features of them. This method is developed by blending SBRC approach with Koyamada's algorithm, which is an efficient image-space algorithm. All proposed algorithms are capable of handling acyclic non-convex meshes and generating images of acceptable quality. SBRC and K-SBRC algorithms have the additional capabilities of rendering cyclic meshes and supporting early ray termination. The proposed algorithms and Koyamada's algorithm are implemented and experimented in a common framework for analyzing their relative performance. © 2003 Elsevier Science Ltd. All rights reserved

Exploiting coherence in time-varying voxel data

We encode time-varying voxel data for efficient storage and streaming. We store the equivalent of a separate sparse voxel octree for each frame, but utilize both spatial and temporal coherence to reduce the amount of memory needed. We represent the time-varying voxel data in a single directed acyclic graph with one root per time step. In this graph, we avoid storing identical regions by keeping one unique instance and pointing to that from several parents. We further reduce the memory consumption of the graph by minimizing the number of bits per pointer and encoding the result into a dense bitstream

Ray Tracing in Non-Euclidean Spaces

This dissertation describes a method for modeling, simulating and real-time rendering piecewise linear approximations of generic non-Euclidean 3D spaces. The 3D rendering pipeline most commonly used, where one multiplies each vertex coordinate by a 4x4 matrix to project it on the screen does not work for all cases where the space does not obey Euclid’s postulates (non-Euclidean space). Furthermore, while other non-Euclidean rendering tools only work for a limited type of spaces, our approach allows us to model, simulate, and render any isometrically embeddable non-Euclidean space and eventual objects lying therein. We envision at least two main applications for our approach. The first for helping mathematicians get a better understanding of what arbitrary spaces look like (e.g., hyperconical space, hyper-spherical space, and so forth). The second for assisting physicists to visualize and simulate the effects of bent space (e.g., black holes, wormholes, Alcubierre drive, and so forth) on light, and on physical objectsEsta dissertação descreve um método para modelar, simular e renderizar aproximações lineares de espaços não Euclideanos de forma genérica e em tempo real. A técnica de renderização 3D mais comum, que multiplica a matriz de projeção 4 x 4 por cada vértice para determinar as coordenadas do respetivo pixel no ecrã, nem sempre funciona quando o espaço não obedece a um postulado de Euclides (espaço não-Euclideano). Além disso, enquanto outras ferramentas para renderizar espaços não-Euclideanos só funcionam para certos tipos de espaços, a nossa técnica permite modelar, simular e renderizar qualquer espaço não-Euclideano embebível isometricamente, bem como eventuais objetos nele existentes. Antevemos pelo menos dois usos para a nossa técnica. A primeira para ajudar matemáticos a compreender melhor o aspeto de espaços arbitrários (e.g., espaço hiper-cónico, espaço hiper-esférico, etc.). A segunda para ajudar físicos a visualizar e simular os efeitos de espaço curvo (e.g., buracos negros, buracos de minhoca, deformações Alcubierra drive, etc.) em luz e objetos físicos circundantes

A GROWTH-BASED APPROACH TO THE AUTOMATIC GENERATION OF NAVIGATION MESHES

Providing an understanding of space in game and simulation environments is one of the major challenges associated with moving artificially intelligent characters through these environments. The usage of some form of navigation mesh has become the standard method to provide a representation of the walkable space in game environments to characters moving around in that environment. There is currently no standardized best method of producing a navigation mesh. In fact, producing an optimal navigation mesh has been shown to be an NP-Hard problem. Current approaches are a patchwork of divergent methods all of which have issues either in the time to create the navigation meshes (e.g., the best looking navigation meshes have traditionally been produced by hand which is time consuming), generate substandard quality navigation meshes (e.g., many of the automatic mesh production algorithms result in highly triangulated meshes that pose problems for character navigation), or yield meshes that contain gaps of areas that should be included in the mesh and are not (e.g., existing growth-based methods are unable to adapt to non-axis-aligned geometry and as such tend to provide a poor representation of the walkable space in complex environments). We introduce the Planar Adaptive Space Filling Volumes (PASFV) algorithm, Volumetric Adaptive Space Filling Volumes (VASFV) algorithm, and the Iterative Wavefront Edge Expansion Cell Decomposition (Wavefront) algorithm. These algorithms provide growth-based spatial decompositions for navigation mesh generation in either 2D (PASFV) or 3D (VASFV). These algorithms generate quick (on demand) decompositions (Wavefront), use quad/cube base spatial structures to provide more regular regions in the navigation mesh instead of triangles, and offer full coverage decompositions to avoid gaps in the navigation mesh by adapting to non-axis-aligned geometry. We have shown experimentally that the decompositions offered by PASFV and VASFV are superior both in character navigation ability, number of regions, and coverage in comparison to the existing and commonly used techniques of Space Filling Volumes, Hertel-Melhorn decomposition, Delaunay Triangulation, and Automatic Path Node Generation. Finally, we show that our Wavefront algorithm retains the superior performance of the PASFV and VASFV algorithms while providing faster decompositions that contain fewer degenerate and near degenerate regions. Unlike traditional navigation mesh generation techniques, the PASFV and VASFV algorithms have a real time extension (Dynamic Adaptive Space Filling Volumes, DASFV) which allows the navigation mesh to adapt to changes in the geometry of the environment at runtime. In addition, it is possible to use a navigation mesh for applications above and beyond character path planning and navigation. These multiple uses help to increase the return on the investment in creating a navigation mesh for a game or simulation environment. In particular, we will show how to use a navigation mesh for the acceleration of collision detection