DIP: Differentiable Interreflection-aware Physics-based Inverse Rendering

We present a physics-based inverse rendering method that learns the illumination, geometry, and materials of a scene from posed multi-view RGB images. To model the illumination of a scene, existing inverse rendering works either completely ignore the indirect illumination or model it by coarse approximations, leading to sub-optimal illumination, geometry, and material prediction of the scene. In this work, we propose a physics-based illumination model that explicitly traces the incoming indirect lights at each surface point based on interreflection, followed by estimating each identified indirect light through an efficient neural network. Furthermore, we utilize the Leibniz's integral rule to resolve non-differentiability in the proposed illumination model caused by one type of environment light -- the tangent lights. As a result, the proposed interreflection-aware illumination model can be learned end-to-end together with geometry and materials estimation. As a side product, our physics-based inverse rendering model also facilitates flexible and realistic material editing as well as relighting. Extensive experiments on both synthetic and real-world datasets demonstrate that the proposed method performs favorably against existing inverse rendering methods on novel view synthesis and inverse rendering

Practical global illumination for interactive particle visualization

ManuscriptParticle-based simulation methods are used to model a wide range of complex phenomena and to solve time-dependent problems of various scales. Effective visualizations of the resulting state will communicate subtle changes in the three-dimensional structure, spatial organization, and qualitative trends within a simulation as it evolves. We present two algorithms targeting upcoming, highly parallel multicore desktop systems to enable interactive navigation and exploration of large particle datasets with global illumination effects. Monte Carlo path tracing and texture mapping are used to capture computationally expensive illumination effects such as soft shadows and diffuse interreflection. The first approach is based on precomputation of luminance textures and removes expensive illumination calculations from the interactive rendering pipeline. The second approach is based on dynamic luminance texture generation and decouples interactive rendering from the computation of global illumination effects. These algorithms provide visual cues that enhance the ability to perform analysis and feature detection tasks while interrogating the data at interactive rates. We explore the performance of these algorithms and demonstrate their effectiveness using several large datasets

Vector occluders: an empirical approximation for rendering global illumination effects in real-time

Precomputation has been previously used as a means to get global illumination effects in real-time on consumer hardware of the day. Our work uses Sloan???s 2002 PRT method as a starting point, and builds on it with two new ideas. We first explore an alternative representation for PRT data. ???Cpherical harmonics??? (CH) are introduced as an alternative to spherical harmonics, by substituting the Chebyshev polynomial in the place of the Legendre polynomial as the orthogonal polynomial in the spherical harmonics definition. We show that CH can be used instead of SH for PRT with near-equivalent performance. ???Vector occluders??? (VO) are introduced as a novel, precomputed, real-time, empirical technique for adding global illumination effects including shadows, caustics and interreflections to a locally illuminated scene on static geometry. VO encodes PRT data as simple vectors instead of using SH. VO can handle point lights, whereas a standard SH implementation cannot

A general illumination model for molecular visualization

Several visual representations have been developed over the years to visualize molecular structures, and to enable a better understanding of their underlying chemical processes. Today, the most frequently used atom-based representations are the Space-filling, the Solvent Excluded Surface, the Balls-and-Sticks, and the Licorice models. While each of these representations has its individual benefits, when applied to large-scale models spatial arrangements can be difficult to interpret when employing current visualization techniques. In the past it has been shown that global illumination techniques improve the perception of molecular visualizations; unfortunately existing approaches are tailored towards a single visual representation. We propose a general illumination model for molecular visualization that is valid for different representations. With our illumination model, it becomes possible, for the first time, to achieve consistent illumination among all atom-based molecular representations. The proposed model can be further evaluated in real-time, as it employs an analytical solution to simulate diffuse light interactions between objects. To be able to derive such a solution for the rather complicated and diverse visual representations, we propose the use of regression analysis together with adapted parameter sampling strategies as well as shape parametrization guided sampling, which are applied to the geometric building blocks of the targeted visual representations. We will discuss the proposed sampling strategies, the derived illumination model, and demonstrate its capabilities when visualizing several dynamic molecules.Peer ReviewedPostprint (author's final draft

Daylight simulation with photon maps

Physically based image synthesis remains one of the most demanding tasks in the computer graphics field, whose applications have evolved along with the techniques in recent years, particularly with the decline in cost of powerful computing hardware. Physically based rendering is essentially a niche since it goes beyond the photorealistic look required by mainstream applications with the goal of computing actual lighting levels in physical quantities within a complex 3D scene. Unlike mainstream applications which merely demand visually convincing images and short rendering times, physically based rendering emphasises accuracy at the cost of increased computational overhead. Among the more specialised applications for physically based rendering is lighting simulation, particularly in conjunction with daylight. The aim of this thesis is to investigate the applicability of a novel image synthesis technique based on Monte Carlo particle transport to daylight simulation. Many materials used in daylight simulation are specifically designed to redirect light, and as such give rise to complex effects such as caustics. The photon map technique was chosen for its efficent handling of these effects. To assess its ability to produce physically correct results which can be applied to lighting simulation, a validation was carried out based on analytical case studies and on simple experimental setups. As prerequisite to validation, the photon map\u27s inherent bias/noise tradeoff is investigated. This tradeoff depends on the density estimate bandwidth used in the reconstruction of the illumination. The error analysis leads to the development of a bias compensating operator which adapts the bandwidth according to the estimated bias in the reconstructed illumination. The work presented here was developed at the Fraunhofer Institute for Solar Energy Systems (ISE) as part of the FARESYS project sponsored by the German national research foundation (DFG), and embedded into the RADIANCE rendering system.Die Erzeugung physikalisch basierter Bilder gilt heute noch als eine der rechenintensivsten Aufgaben in der Computergraphik, dessen Anwendungen sowie auch Verfahren in den letzten Jahren kontinuierlich weiterentwickelt wurden, vorangetrieben primär durch den Preisverfall leistungsstarker Hardware. Physikalisch basiertes Rendering hat sich als Nische etabliert, die über die photorealistischen Anforderungen typischer Mainstream-Applikationen hinausgeht, mit dem Ziel, Lichttechnische Größen innerhalb einer komplexen 3D Szene zu berechnen. Im Gegensatz zu Mainstream-Applikationen, die visuell überzeugend wirken sollen und kurze Rechenzeiten erforden, liegt der Schwerpunkt bei physikalisch basiertem Rendering in der Genauigkeit, auf Kosten des Rechenaufwands. Zu den eher spezialisierten Anwendungen im Gebiet des physikalisch basiertem Renderings gehört die Lichtsimulation, besonders in Bezug auf Tageslicht. Das Ziel dieser Dissertation liegt darin, die Anwendbarkeit eines neuartigen Renderingverfahrens basierend auf Monte Carlo Partikeltransport hinsichtlich Tageslichtsimulation zu untersuchen. Viele Materialien, die in der Tageslichtsimulation verwendet werden, sind speziell darauf konzipiert, Tageslicht umzulenken, und somit komplexe Phänomene wie Kaustiken hervorrufen. Das Photon-Map-Verfahren wurde aufgrund seiner effizienten Simulation solcher Effekte herangezogen. Zur Beurteilung seiner Fähigkeit, physikalisch korrekte Ergebnisse zu liefern, die in der Tageslichtsimulation anwendbar sind, wurde eine Validierung anhand analytischer Studien sowie eines einfachen experimentellen Aufbaus durchgeführt. Als Voraussetzung zur Validierung wurde der Photon Map bezüglich seiner inhärenten Wechselwirkung zwischen Rauschen und systematischem Fehler (Bias) untersucht. Diese Wechselwirkung hängt von der Bandbreite des Density Estimates ab, mit dem die Beleuchtung aus den Photonen rekonstruiert wird. Die Fehleranalyse führt zur Entwicklung eines Bias compensating Operators, der die Bandbreite dynamisch anhand des geschätzten Bias in der rekonstruierten Beleuchtung anpasst. Die hier vorgestellte Arbeit wurde am Fraunhofer Institut für Solare Energiesysteme (ISE) als teil des FARESYS Projekts entwickelt, daß von der Deutschen Forschungsgemeinschaft (DFG) finanziert wurde. Die Implementierung erfolgte im Rahmen des RADIANCE Renderingsystems

Distributed high-fidelity graphics using P2P

In the field of three-dimensional computer graphics, rendering refers to the process of generating images of a scene from a particular viewpoint. There are many different ways to do this, from the highly interactive real-time rendering methods to the more photorealistic and computationally intensive methods. This work is concerned with Physically Based Rendering (PBR), a class of rendering algorithms capable of achieving a very high level of realism. This is achievable thanks to physically accurate modelling of the way light interacts with objects in a scene, together with the use of accurately modelled materials and physical quantities.peer-reviewe

Photon Mapping

V rámci této práce byla provedena praktická implementace algoritmu photon mapping. Pro dosažení kvalitnějšího výstupu byly zkoumány některé základní a pokročilejší metody globálního osvětlení. Tyto náročné algoritmy jsou často prakticky nepoužitelné a je nutná jejich optimalizace. Základem praktické implementace je optimalizace raytraceru. Vzorky nepřímého difuzního osvětlení počítané metodou Monte Carlo je možné mezi sebou interpolovat s použitím vhodné techniky.This thesis deals with practical implementation of photon mapping algorithm. To achieve better results some basic and some more advanced methods of global illumination has been examined. These time demanding algorithms are often practically unusable and their further optimization is necessary. Optimized ray tracer is essential for practical implementation. Computing diffuse interreflection by Monte Carlo sampling is also very time demanding operation. Therefore it is appropriate to use it along with proper interpolation.

A General Two-Pass Method Integrating Specular and Diffuse Reflection

International audienceWe analyse some recent approaches to the global illumination problem by introducing the corresponding reflection operators, and we demonstrate the advantages of a two-pass method. A generalization of the system introduced by Wallace et al. at Siggraph '87 to integrate diffuse as well as specular effects is presented. It is based on the calculation of extended form-factors, which allows arbitrary geometries to be used in the scene description, as well as refraction effects. We also present a new sampling method for the calculation of form-factors, which is an Mternative to the hemi-cube technique introduced by Cohen and Greenberg for radiosity calculations. This method is particularly well suited to the extended form-factors calculation. The problem of interactive display of the picture being created is also addressed by using hardware-assisted projections and image composition to recreate a complete specular view of the scene

Doctor of Philosophy

dissertationReal-time global illumination is the next frontier in real-time rendering. In an attempt to generate realistic images, games have followed the film industry into physically based shading and will soon begin integrating global illumination techniques. Traditional methods require too much memory and too much time to compute for real-time use. With Modular and Delta Radiance Transfer we precompute a scene-independent, low-frequency basis that allows us to calculate complex indirect lighting calculations in a much lower dimensional subspace with a reduced memory footprint and real-time execution. The results are then applied as a light map on many different scenes. To improve the low frequency results, we also introduce a novel screen space ambient occlusion technique that allows us to generate a smoother result with fewer samples. These three techniques, low and high frequency used together, provide a viable indirect lighting solution that can be run in milliseconds on today's hardware, providing a useful new technique for indirect lighting in real-time graphics

Implementation and Analysis of an Image-Based Global Illumination Framework for Animated Environments

We describe a new framework for efficiently computing and storing global illumination effects for complex, animated environments. The new framework allows the rapid generation of sequences representing any arbitrary path in a view space within an environment in which both the viewer and objects move. The global illumination is stored as time sequences of range-images at base locations that span the view space. We present algorithms for determining locations for these base images, and the time steps required to adequately capture the effects of object motion. We also present algorithms for computing the global illumination in the base images that exploit spatial and temporal coherence by considering direct and indirect illumination separately. We discuss an initial implementation using the new framework. Results and analysis of our implementation demonstrate the effectiveness of the individual phases of the approach; we conclude with an application of the complete framework to a complex environment that includes object motion