Mobile graphics: SIGGRAPH Asia 2017 course

Peer ReviewedPostprint (published version

On sparse voxel DAGs and memory efficient compression of surface attributes for real-time scenarios

The general shape of a 3D object can expeditiously be represented as, e.g., triangles or voxels, while smaller-scale features usually are parameterized over the surface of the object. Such features include, but are not limited to, color details, small-scale surface-normal variations, or even view-dependent properties required for the light-surface interactions. This thesis is a collection of four papers that focus on new ways to compress and efficiently utilize surface data in 3D for real-time usage.In Paper IA and IB, we extend upon the concept of sparse voxel DAGs, a real-time compression format of a voxel-grid, to allow an attribute mapping with a negligible impact on the size. The main contribution, however, is a novel real-time compression format of the mapped colors over such sparse voxel surfaces.Paper II aims to utilize the results of the previous papers to achieve uv-free texturing of surfaces, such as triangle meshes, with optimized run-time minification as well as magnification filtering.Paper III extends upon previous compact representations of view dependent radiance using spherical gaussians (SG). By using a convolutional neural network, we are able to compress the light-field by finding SGs with free directions, amplitudes and sharpnesses, whereas previous methods were limited to only free amplitudes in similar scenarios

QuadStack: An Efficient Representation and Direct Rendering of Layered Datasets

We introduce QuadStack, a novel algorithm for volumetric data compression and direct rendering. Our algorithm exploits the data redundancy often found in layered datasets which are common in science and engineering fields such as geology, biology, mechanical engineering, medicine, etc. QuadStack first compresses the volumetric data into vertical stacks which are then compressed into a quadtree that identifies and represents the layered structures at the internal nodes. The associated data (color, material, density, etc.) and shape of these layer structures are decoupled and encoded independently, leading to high compression rates (4× to 54× of the original voxel model memory footprint in our experiments). We also introduce an algorithm for value retrieving from the QuadStack representation and we show that the access has logarithmic complexity. Because of the fast access, QuadStack is suitable for efficient data representation and direct rendering. We show that our GPU implementation performs comparably in speed with the state-of-the-art algorithms (18-79 MRays/s in our implementation), while maintaining a significantly smaller memory footprint

Towards Predictive Rendering in Virtual Reality

The strive for generating predictive images, i.e., images representing radiometrically correct renditions of reality, has been a longstanding problem in computer graphics. The exactness of such images is extremely important for Virtual Reality applications like Virtual Prototyping, where users need to make decisions impacting large investments based on the simulated images. Unfortunately, generation of predictive imagery is still an unsolved problem due to manifold reasons, especially if real-time restrictions apply. First, existing scenes used for rendering are not modeled accurately enough to create predictive images. Second, even with huge computational efforts existing rendering algorithms are not able to produce radiometrically correct images. Third, current display devices need to convert rendered images into some low-dimensional color space, which prohibits display of radiometrically correct images. Overcoming these limitations is the focus of current state-of-the-art research. This thesis also contributes to this task. First, it briefly introduces the necessary background and identifies the steps required for real-time predictive image generation. Then, existing techniques targeting these steps are presented and their limitations are pointed out. To solve some of the remaining problems, novel techniques are proposed. They cover various steps in the predictive image generation process, ranging from accurate scene modeling over efficient data representation to high-quality, real-time rendering. A special focus of this thesis lays on real-time generation of predictive images using bidirectional texture functions (BTFs), i.e., very accurate representations for spatially varying surface materials. The techniques proposed by this thesis enable efficient handling of BTFs by compressing the huge amount of data contained in this material representation, applying them to geometric surfaces using texture and BTF synthesis techniques, and rendering BTF covered objects in real-time. Further approaches proposed in this thesis target inclusion of real-time global illumination effects or more efficient rendering using novel level-of-detail representations for geometric objects. Finally, this thesis assesses the rendering quality achievable with BTF materials, indicating a significant increase in realism but also confirming the remainder of problems to be solved to achieve truly predictive image generation

Scalable Realtime Rendering and Interaction with Digital Surface Models of Landscapes and Cities

Interactive, realistic rendering of landscapes and cities differs substantially from classical terrain rendering. Due to the sheer size and detail of the data which need to be processed, realtime rendering (i.e. more than 25 images per second) is only feasible with level of detail (LOD) models. Even the design and implementation of efficient, automatic LOD generation is ambitious for such out-of-core datasets considering the large number of scales that are covered in a single view and the necessity to maintain screen-space accuracy for realistic representation. Moreover, users want to interact with the model based on semantic information which needs to be linked to the LOD model. In this thesis I present LOD schemes for the efficient rendering of 2.5d digital surface models (DSMs) and 3d point-clouds, a method for the automatic derivation of city models from raw DSMs, and an approach allowing semantic interaction with complex LOD models. The hierarchical LOD model for digital surface models is based on a quadtree of precomputed, simplified triangle mesh approximations. The rendering of the proposed model is proved to allow real-time rendering of very large and complex models with pixel-accurate details. Moreover, the necessary preprocessing is scalable and fast. For 3d point clouds, I introduce an LOD scheme based on an octree of hybrid plane-polygon representations. For each LOD, the algorithm detects planar regions in an adequately subsampled point cloud and models them as textured rectangles. The rendering of the resulting hybrid model is an order of magnitude faster than comparable point-based LOD schemes. To automatically derive a city model from a DSM, I propose a constrained mesh simplification. Apart from the geometric distance between simplified and original model, it evaluates constraints based on detected planar structures and their mutual topological relations. The resulting models are much less complex than the original DSM but still represent the characteristic building structures faithfully. Finally, I present a method to combine semantic information with complex geometric models. My approach links the semantic entities to the geometric entities on-the-fly via coarser proxy geometries which carry the semantic information. Thus, semantic information can be layered on top of complex LOD models without an explicit attribution step. All findings are supported by experimental results which demonstrate the practical applicability and efficiency of the methods

Multiresolution Techniques for Real–Time Visualization of Urban Environments and Terrains

In recent times we are witnessing a steep increase in the availability of data coming from real–life environments. Nowadays, virtually everyone connected to the Internet may have instant access to a tremendous amount of data coming from satellite elevation maps, airborne time-of-flight scanners and digital cameras, street–level photographs and even cadastral maps. As for other, more traditional types of media such as pictures and videos, users of digital exploration softwares expect commodity hardware to exhibit good performance for interactive purposes, regardless of the dataset size. In this thesis we propose novel solutions to the problem of rendering large terrain and urban models on commodity platforms, both for local and remote exploration. Our solutions build on the concept of multiresolution representation, where alternative representations of the same data with different accuracy are used to selectively distribute the computational power, and consequently the visual accuracy, where it is more needed on the base of the user’s point of view. In particular, we will introduce an efficient multiresolution data compression technique for planar and spherical surfaces applied to terrain datasets which is able to handle huge amount of information at a planetary scale. We will also describe a novel data structure for compact storage and rendering of urban entities such as buildings to allow real–time exploration of cityscapes from a remote online repository. Moreover, we will show how recent technologies can be exploited to transparently integrate virtual exploration and general computer graphics techniques with web applications

Theory and algorithms for efficient physically-based illumination

Realistic image synthesis is one of the central fields of study within computer graphics. This thesis treats efficient methods for simulating light transport in situations where the incident illumination is produced by non-pointlike area light sources and distant illumination described by environment maps. We describe novel theory and algorithms for physically-based lighting computations, and expose the design choices and tradeoffs on which the techniques are based. Two publications included in this thesis deal with precomputed light transport. These techniques produce interactive renderings of static scenes under dynamic illumination and full global illumination effects. This is achieved through sacrificing the ability to freely deform and move the objects in the scene. We present a comprehensive mathematical framework for precomputed light transport. The framework, which is given as an abstract operator equation that extends the well-known rendering equation, encompasses a significant amount of prior work as its special cases. We also present a particular method for rendering objects in low-frequency lighting environments, where increased efficiency is gained through the use of compactly supported function bases. Physically-based shadows from area and environmental light sources are an important factor in perceived image realism. We present two algorithms for shadow computation. The first technique computes shadows cast by low-frequency environmental illumination on animated objects at interactive rates without requiring difficult precomputation or a priori knowledge of the animations. Here the capability to animate is gained by forfeiting indirect illumination. Another novel shadow algorithm for off-line rendering significantly enhances a previous physically-based soft shadow technique by introducing an improved spatial hierarchy that alleviates redundant computations at the cost of using more memory. This thesis advances the state of the art in realistic image synthesis by introducing several algorithms that are more efficient than their predecessors. Furthermore, the theoretical contributions should enable the transfer of ideas from one particular application to others through abstract generalization of the underlying mathematical concepts.Tämä tutkimus käsittelee realististen kuvien syntetisointia tietokoneella tilanteissa, jossa virtuaalisen ympäristön valonlähteet ovat fysikaalisesti mielekkäitä. Fysikaalisella mielekkyydellä tarkoitetaan sitä, että valonlähteet eivät ole idealisoituja eli pistemäisiä, vaan joko tavanomaisia pinta-alallisia valoja tai kaukaisia ympäristövalokenttiä (environment maps). Väitöskirjassa esitetään uusia algoritmeja, jotka soveltuvat matemaattisesti perusteltujen valaistusapproksimaatioiden laskentaan erilaisissa käyttötilanteissa. Esilaskettu valonkuljetus on yleisnimi reaaliaikaisille menetelmille, jotka tuottavat kuvia staattisista ympäristöistä siten, että valaistus voi muuttua ajon aikana vapaasti ennalta määrätyissä rajoissa. Tässä työssä esitetään esilasketulle valonkuljetukselle kattava matemaattinen kehys, joka selittää erikoistapauksinaan suuren määrän aiempaa tutkimusta. Kehys annetaan abstraktin lineaarisen operaattoriyhtälön muodossa, ja se yleistää tunnettua kuvanmuodostusyhtälöä (rendering equation). Työssä esitetään myös esilasketun valonkuljetuksen algoritmi, joka parantaa aiempien vastaavien menetelmien tehokkuutta esittämällä valaistuksen funktiokannassa, jonka ominaisuuksien vuoksi ajonaikainen laskenta vähenee huomattavasti. Fysikaalisesti mielekkäät valonlähteet tuottavat pehmeäreunaisia varjoja. Työssä esitetään uusi algoritmi pehmeiden varjojen laskemiseksi liikkuville ja muotoaan muuttaville kappaleille, joita valaisee matalataajuinen ympäristövalokenttä. Useimmista aiemmista menetelmistä poiketen algoritmi ei vaadi esitietoa siitä, kuinka kappale voi muuttaa muotoaan ajon aikana. Muodonmuutoksen aiheuttaman suuren laskentakuorman vuoksi epäsuoraa valaistusta ei huomioida. Työssä esitetään myös toinen uusi algoritmi pehmeiden varjojen laskemiseksi, jossa aiemman varjotilavuuksiin (shadow volumes) perustuvan algoritmin tehokkuutta parannetaan merkittävästi uuden hierarkkisen avaruudellisen hakurakenteen avulla. Uusi rakenne vähentää epäoleellista laskentaa muistinkulutuksen kustannuksella. Työssä esitetään aiempaa tehokkaampia algoritmeja fysikaalisesti perustellun valaistuksen laskentaan. Niiden lisäksi työn esilaskettua valonkuljetusta koskevat teoreettiset tulokset yleistävät suuren joukon aiempaa tutkimusta ja mahdollistavat näin ideoiden siirron erityisalalta toiselle.reviewe