Representation and coding of 3D video data

Livrable D4.1 du projet ANR PERSEECe rapport a été réalisé dans le cadre du projet ANR PERSEE (n° ANR-09-BLAN-0170). Exactement il correspond au livrable D4.1 du projet

3D Geometry Representation using Multiview Coding of Image Tiles

Compression of dynamic 3D geometry obtained from depth sensors is challenging, because noise and temporal inconsistency inherent in acquisition of depth data means there is no one-to-one correspondence between sets of 3D points in consecutive time instants. In this paper, instead of coding 3D points (or meshes) directly, we propose to represent an object’s 3D geometry as a collection of tile images. Specifically, we first place a set of image tiles around an object. Then, we project the object’s 3D geometry onto the tiles that are interpreted as 2D depth images, which we subsequently encode using a modified multiview image codec tuned for piecewise smooth signals. The crux of the tile image framework is the “optimal” placement of image tiles—one that yields the best tradeoff in rate and distortion. We show that if only planar and cylindrical tiles are considered, then the optimal placement problem for K tiles can be mapped to a tractable piecewise linear approximation problem. We propose an efficient dynamic programming algorithm to find an optimal solution to the piecewise linear approximation problem. Experimental results show that optimal tiling outperforms naive tiling by up to 35% in rate reduction, and graph transform can further exploit the smoothness of the tile images for coding gain

Retopology: a comprehensive study of current automation solutions from an artist’s workflow perspective

Dissertação de mestrado em Engenharia InformáticaTopology (the density, organization and flow of a 3D mesh’s connectivity) constrains the suitability of a 3D model for any given purpose, be it surface showcasing through renders, use in real-time engines, posing or animation. While some of these use cases might not have very strict topology requirements, others may demand optimized polygon counts for performance reasons, or even specific geometry distribution in order to take deformation directions into account. Many processes for creating 3D models such as sculpting try to make the user unaware of the inner workings of geometry, by providing flexible levels of surface detailing through dynamic geometry allocation. The resulting models have a dense, unorganized topology that is inefficient and unfit for most use cases, with the additional drawback of being hard to work with manually. Retopology is the process of providing a new topology to a model such as these, while maintaining the shape of its surface. It’s a technical and time-consuming process that clashes with the rest of the artist’s workflow, which is mainly composed of creative processes. While there’s abundant research in this area focusing on polygon distribution quality based on surface shape, artists are still left with no options but to resort to manual work when it comes to deformation-optimized topology. This document exposes this disconnect, along with a proposed framework that attempts to provide a more complete retopology solution for 3D artists. This framework combines traditional mesh extraction algorithms with adapting manually-made meshes in a pipeline that tries to understand the input on a higher level, in order to solve deficiencies that are present in current retopology tools. Our results are very positive, presenting an improvement over state of the art solutions, which could possibly steer discussion and research in this area to be more in line with the needs of 3D artists.A topologia (a densidade, organização e direções tomadas pela conectividade de uma mesh 3D) limita a adequação de um modelo 3D para um leque variado de usos, entre os quais, visualização da superfície através de renders, uso em motores real-time, poses ou animações. Embora muitos destes usos não possuam requerimentos de topologia muito rigorosos, outros podem exigir número de polígonos mais baixos por questões de performance, ou até distribuição de geometria específica para acomodar direções de deformação corretamente. Muitos processos de criação de modelos 3D, como escultura, permitem que o utilizador não esteja ciente do que se passa em termos de funcionamento da geometria por debaixo da utilização. Isto é conseguido oferecendo níveis de detalhe flexíveis, alocando geometria de forma dinâmica. Os modelos resultantes têm uma topologia densa e desorganizada, que é ineficiente e pouco apropriada para a maior parte dos casos de uso, com a desvantagem adicional de ser difícil de trabalhar com a mesma manualmente. A retopologia é o processo de gerar uma nova topologia para um modelo, ao mesmo tempo que se mantém a forma da superfície. É um processo técnico e demorado, que entra em conflito com o resto do fluxo de trabalho do artista, que é composto maioritariamente por processos artísticos. Apesar de haver investigação abundante nesta área focada na qualidade da distribuição de polígonos baseada na forma da superfície, os artistas continuam a ter de recorrer ao trabalho manual quando se trata de topologia otimizada para deformações. Este documento expõe esta divergência, propondo, em conjunto, uma framework que tenta oferecer uma solução mais completa para os artistas 3D. Esta framework combina algoritmos de extração de meshes tradicionais com adaptação de meshes feitas manualmente, numa pipeline que tenta compreender o input a um nível superior, resolvendo as deficiências presentes nas ferramentas de retopologia atuais. Os nossos resultados são bastante positivos, apresentando melhorias em relação a soluções de estado da arte, facto que poderá mudar o rumo da discussão e investigação neste campo, para melhor se adequar às necessidades dos artistas 3D

Diamond-based models for scientific visualization

Hierarchical spatial decompositions are a basic modeling tool in a variety of application domains including scientific visualization, finite element analysis and shape modeling and analysis. A popular class of such approaches is based on the regular simplex bisection operator, which bisects simplices (e.g. line segments, triangles, tetrahedra) along the midpoint of a predetermined edge. Regular simplex bisection produces adaptive simplicial meshes of high geometric quality, while simplifying the extraction of crack-free, or conforming, approximations to the original dataset. Efficient multiresolution representations for such models have been achieved in 2D and 3D by clustering sets of simplices sharing the same bisection edge into structures called diamonds. In this thesis, we introduce several diamond-based approaches for scientific visualization. We first formalize the notion of diamonds in arbitrary dimensions in terms of two related simplicial decompositions of hypercubes. This enables us to enumerate the vertices, simplices, parents and children of a diamond. In particular, we identify the number of simplices involved in conforming updates to be factorial in the dimension and group these into a linear number of subclusters of simplices that are generated simultaneously. The latter form the basis for a compact pointerless representation for conforming meshes generated by regular simplex bisection and for efficiently navigating the topological connectivity of these meshes. Secondly, we introduce the supercube as a high-level primitive on such nested meshes based on the atomic units within the underlying triangulation grid. We propose the use of supercubes to associate information with coherent subsets of the full hierarchy and demonstrate the effectiveness of such a representation for modeling multiresolution terrain and volumetric datasets. Next, we introduce Isodiamond Hierarchies, a general framework for spatial access structures on a hierarchy of diamonds that exploits the implicit hierarchical and geometric relationships of the diamond model. We use an isodiamond hierarchy to encode irregular updates to a multiresolution isosurface or interval volume in terms of regular updates to diamonds. Finally, we consider nested hypercubic meshes, such as quadtrees, octrees and their higher dimensional analogues, through the lens of diamond hierarchies. This allows us to determine the relationships involved in generating balanced hypercubic meshes and to propose a compact pointerless representation of such meshes. We also provide a local diamond-based triangulation algorithm to generate high-quality conforming simplicial meshes

Beyond high-resolution geometry in 3D Cultural Heritage: enhancing visualization realism in interactive contexts

La tesi, nell’ambito della computer graphics 3D interattiva, descrive la definizione e sviluppo di algoritmi per un migliore realismo nella visualizzazione di modelli tridimensionali di grandi dimensioni, con particolare attenzione alle applicazioni di queste tecnologie di visualizzazione 3D ai beni culturali