A constructive theory of sampling for image synthesis using reproducing kernel bases

Sampling a scene by tracing rays and reconstructing an image from such pointwise samples is fundamental to computer graphics. To improve the efficacy of these computations, we propose an alternative theory of sampling. In contrast to traditional formulations for image synthesis, which appeal to nonconstructive Dirac deltas, our theory employs constructive reproducing kernels for the correspondence between continuous functions and pointwise samples. Conceptually, this allows us to obtain a common mathematical formulation of almost all existing numerical techniques for image synthesis. Practically, it enables novel sampling based numerical techniques designed for light transport that provide considerably improved performance per sample. We exemplify the practical benefits of our formulation with three applications: pointwise transport of color spectra, projection of the light energy density into spherical harmonics, and approximation of the shading equation from a photon map. Experimental results verify the utility of our sampling formulation, with lower numerical error rates and enhanced visual quality compared to existing techniques

Inverse Global Illumination using a Neural Radiometric Prior

Inverse rendering methods that account for global illumination are becoming more popular, but current methods require evaluating and automatically differentiating millions of path integrals by tracing multiple light bounces, which remains expensive and prone to noise. Instead, this paper proposes a radiometric prior as a simple alternative to building complete path integrals in a traditional differentiable path tracer, while still correctly accounting for global illumination. Inspired by the Neural Radiosity technique, we use a neural network as a radiance function, and we introduce a prior consisting of the norm of the residual of the rendering equation in the inverse rendering loss. We train our radiance network and optimize scene parameters simultaneously using a loss consisting of both a photometric term between renderings and the multi-view input images, and our radiometric prior (the residual term). This residual term enforces a physical constraint on the optimization that ensures that the radiance field accounts for global illumination. We compare our method to a vanilla differentiable path tracer, and more advanced techniques such as Path Replay Backpropagation. Despite the simplicity of our approach, we can recover scene parameters with comparable and in some cases better quality, at considerably lower computation times.Comment: Homepage: https://inverse-neural-radiosity.github.i

Artistic Path Space Editing of Physically Based Light Transport

Die Erzeugung realistischer Bilder ist ein wichtiges Ziel der Computergrafik, mit Anwendungen u.a. in der Spielfilmindustrie, Architektur und Medizin. Die physikalisch basierte Bildsynthese, welche in letzter Zeit anwendungsübergreifend weiten Anklang findet, bedient sich der numerischen Simulation des Lichttransports entlang durch die geometrische Optik vorgegebener Ausbreitungspfade; ein Modell, welches für übliche Szenen ausreicht, Photorealismus zu erzielen. Insgesamt gesehen ist heute das computergestützte Verfassen von Bildern und Animationen mit wohlgestalteter und theoretisch fundierter Schattierung stark vereinfacht. Allerdings ist bei der praktischen Umsetzung auch die Rücksichtnahme auf Details wie die Struktur des Ausgabegeräts wichtig und z.B. das Teilproblem der effizienten physikalisch basierten Bildsynthese in partizipierenden Medien ist noch weit davon entfernt, als gelöst zu gelten. Weiterhin ist die Bildsynthese als Teil eines weiteren Kontextes zu sehen: der effektiven Kommunikation von Ideen und Informationen. Seien es nun Form und Funktion eines Gebäudes, die medizinische Visualisierung einer Computertomografie oder aber die Stimmung einer Filmsequenz -- Botschaften in Form digitaler Bilder sind heutzutage omnipräsent. Leider hat die Verbreitung der -- auf Simulation ausgelegten -- Methodik der physikalisch basierten Bildsynthese generell zu einem Verlust intuitiver, feingestalteter und lokaler künstlerischer Kontrolle des finalen Bildinhalts geführt, welche in vorherigen, weniger strikten Paradigmen vorhanden war. Die Beiträge dieser Dissertation decken unterschiedliche Aspekte der Bildsynthese ab. Dies sind zunächst einmal die grundlegende Subpixel-Bildsynthese sowie effiziente Bildsyntheseverfahren für partizipierende Medien. Im Mittelpunkt der Arbeit stehen jedoch Ansätze zum effektiven visuellen Verständnis der Lichtausbreitung, die eine lokale künstlerische Einflussnahme ermöglichen und gleichzeitig auf globaler Ebene konsistente und glaubwürdige Ergebnisse erzielen. Hierbei ist die Kernidee, Visualisierung und Bearbeitung des Lichts direkt im alle möglichen Lichtpfade einschließenden "Pfadraum" durchzuführen. Dies steht im Gegensatz zu Verfahren nach Stand der Forschung, die entweder im Bildraum arbeiten oder auf bestimmte, isolierte Beleuchtungseffekte wie perfekte Spiegelungen, Schatten oder Kaustiken zugeschnitten sind. Die Erprobung der vorgestellten Verfahren hat gezeigt, dass mit ihnen real existierende Probleme der Bilderzeugung für Filmproduktionen gelöst werden können

Efficient Object-Based Hierarchical Radiosity Methods

The efficient generation of photorealistic images is one of the main subjects in the field of computer graphics. In contrast to simple image generation which is directly supported by standard 3D graphics hardware, photorealistic image synthesis strongly adheres to the physics describing the flow of light in a given environment. By simulating the energy flow in a 3D scene global effects like shadows and inter-reflections can be rendered accurately. The hierarchical radiosity method is one way of computing the global illumination in a scene. Due to its limitation to purely diffuse surfaces solutions computed by this method are view independent and can be examined in real-time walkthroughs. Additionally, the physically based algorithm makes it well suited for lighting design and architectural visualization. The focus of this thesis is the application of object-oriented methods to the radiosity problem. By consequently keeping and using object information throughout all stages of the algorithms several contributions to the field of radiosity rendering could be made. By introducing a new meshing scheme, it is shown how curved objects can be treated efficiently by hierarchical radiosity algorithms. Using the same paradigm the radiosity computation can be distributed in a network of computers. A parallel implementation is presented that minimizes communication costs while obtaining an efficient speedup. Radiosity solutions for very large scenes became possible by the use of clustering algorithms. Groups of objects are combined to clusters to simulate the energy exchange on a higher abstraction level. It is shown how the clustering technique can be improved without loss in image quality by applying the same data-structure for both, the visibility computations and the efficient radiosity simulation.Eines der Schwerpunktthemen in der Computergraphik ist die effiziente Erzeugung von fotorealistischen Bildern. Im Gegensatz zur einfachen Bilderzeugung, die bereits durch gaengige 3D-Grafikhardware unterstuetzt wird, gehorcht die fotorealistische Bildsynthese physikalischen Gesetzen, die die Lichtausbreitung innerhalb einer bestimmten Umgebung beschreiben. Durch die Simulation der Energieausbreitung in einer dreidimensionalen Szene koennen globale Effekte wie Schatten und mehrfache Reflektionen wirklichkeitstreu dargestellt werden. Die hierarchische Radiositymethode (Hierarchical Radiosity) ist eine Moeglichkeit, um die globale Beleuchtung innerhalb einer Szene zu berechnen. Da diese Methode auf die Verwendung von rein diffus reflektierenden Oberflaechen beschraenkt ist, sind damit errechnete Loesungen blickwinkelunabhaengig und lassen sich in Echtzeit am Bildschirm durchwandern. Zudem ist dieser Algorithmus aufgrund der verwendeten physikalischen Grundlagen sehr gut zur Beleuchtungssimulation und Architekturvisualisierung geeignet. Den Schwerpunkt dieser Doktorarbeit stellt die Anwendung objektbasierter Methoden auf das Radiosityproblem dar. Durch konsequente Ausnutzung von Objektinformationen waehrend aller Berechnungsschritte konnten verschiedene Verbesserungen im Rahmen der hierarchischen Radiositymethode erzielt werden. Gekruemmte Objekte koennen aufgrund eines neuen Flaechenunterteilungsverfahrens nun effizient durch den hierarchischen Radiosityalgorithmus dargestellt werden. Dieses Verfahren ermoeglicht ebenso eine effiziente Parallelisierung des hierarchischen Radiosityalgorithmus. Es wird ein parallele Implementierung vorgestellt, die unter Minimierung der Kommunikationskosten eine effiziente Geschwindigkeitssteigerung erzielt. Radiosityberechnungen fuer sehr grosse Szenen sind nur durch Verwendung sogenannter Clustering-Algorithmen moeglich. Dabei werden Gruppen von Objekten zu Clustern kombiniert um den Energieaustausch zwischen Oberflaechen stellvertretend auf einem hoeheren Abstraktionsniveau durchzufuehren. Durch Verwendung derselben Datenstruktur fuer Sichtbarkeitsberechnungen und fuer die Steuerung der Radiositysimulation wird gezeigt, wie das Clusteringverfahren ohne Qualitaetsverluste verbessert werden kann

Efficient representations of large radiosity matrices

The radiosity equation can be expressed as a linear system, where light interactions between patches of the scene are considered. Its resolution has been one of the main subjects in computer graphics, which has lead to the development of methods focused on different goals. For instance, in inverse lighting problems, it is convenient to solve the radiosity equation thousands of times for static geometries. Also, this calculation needs to consider many (or infinite) light bounces to achieve accurate global illumination results. Several methods have been developed to solve the linear system by finding approximations or other representations of the radiosity matrix, because the full storage of this matrix is memory demanding. Some examples are hierarchical radiosity, progressive refinement approaches, or wavelet radiosity. Even though these methods are memory efficient, they may become slow for many light bounces, due to their iterative nature. Recently, efficient methods have been developed for the direct resolution of the radiosity equation. In this case, the challenge is to reduce the memory requirements of the radiosity matrix, and its inverse. The main objective of this thesis is exploiting the properties of specific problems to reduce the memory requirements of the radiosity problem. Hereby, two types of problems are analyzed. The first problem is to solve radiosity for scenes with a high spatial coherence, such as it happens to some architectural models. The second involves scenes with a high occlusion factor between patches. For the high spatial coherence case, a novel and efficient error-bounded factorization method is presented. It is based on the use of multiple singular value decompositions along with a space filling curve, which allows to exploit spatial coherence. This technique accelerates the factorization of in-core matrices, and allows to work with out-of-core matrices passing only one time over them. In the experimental analysis, the presented method is applied to scenes up to 163K patches. After a precomputation stage, it is used to solve the radiosity equation for fixed geometries and infinite bounces, at interactive times. For the high occlusion problem, city models are used. In this case, the sparsity of the radiosity matrix is exploited. An approach for radiative exchange computation is proposed, where the inverse of the radiosity matrix is approximated. In this calculation, near-zero elements are removed, leading to a highly sparse result. This technique is applied to simulate daylight in urban environments composed by up to 140k patches.La ecuación de radiosidad tiene por objetivo el cálculo de la interacción de la luz con los elementos de la escena. Esta se puede expresar como un sistema lineal, cuya resolución ha derivado en el desarrollo de diversos métodos gráficos para satisfacer propósitos específicos. Por ejemplo, en problemas inversos de iluminación para geometrías estáticas, se debe resolver la ecuación de radiosidad miles de veces. Además, este cálculo debe considerar muchos (infinitos) rebotes de luz, si se quieren obtener resultados precisos de iluminación global. Entre los métodos desarrollados, se destacan aquellos que generan aproximaciones u otras representaciones de la matriz de radiosidad, debido a que su almacenamiento requiere grandes cantidades de memoria. Algunos ejemplos de estas técnicas son la radiosidad jerárquica, el refinamiento progresivo y la radiosidad basada en wavelets. Si bien estos métodos son eficientes en cuanto a memoria, pueden ser lentos cuando se requiere el cálculo de muchos rebotes de luz, debido a su naturaleza iterativa. Recientemente se han desarrollado métodos eficientes para la resolución directa de la ecuación de radiosidad, basados en el pre-cómputo de la inversa de la matriz de radiosidad. En estos casos, el desafío consiste en reducir los requerimientos de memoria y tiempo de ejecución para el cálculo de la matriz y de su inversa. El principal objetivo de la tesis consiste en explotar propiedades específicas de ciertos problemas de iluminación para reducir los requerimientos de memoria de la ecuación de radiosidad. En este contexto, se analizan dos casos diferentes. El primero consiste en hallar la radiosidad para escenas con alta coherencia espacial, tal como ocurre en algunos modelos arquitectónicos. El segundo involucra escenas con un elevado factor de oclusión entre parches. Para el caso de alta coherencia espacial, se presenta un nuevo método de factorización de matrices que es computacionalmente eficiente y que genera aproximaciones cuyo error es configurable. Está basado en el uso de múltiples descomposiciones en valores singulares (SVD) junto a una curva de recubrimiento espacial, lo que permite explotar la coherencia espacial. Esta técnica acelera la factorización de matrices que entran en memoria, y permite trabajar con matrices que no entran en memoria, recorriéndolas una única vez. En el análisis experimental, el método presentado es aplicado a escenas de hasta 163 mil parches. Luego de una etapa de precómputo, se logra resolver la ecuación de radiosidad en tiempos interactivos, para geométricas estáticas e infinitos rebotes. Para el problema de alta oclusión, se utilizan modelos de ciudades. En este caso, se aprovecha la baja densidad de la matriz de radiosidad, y se propone una técnica para el cálculo aproximado de su inversa. En este cálculo, los elementos cercanos a cero son eliminados. La técnica es aplicada a la simulación de la luz natural en ambientes urbanos compuestos por hasta 140 mil parches

異なる空間を繋ぐ光輸送シミュレーション

学位の種別: 課程博士審査委員会委員 : （主査）東京大学教授 稲葉 雅幸, 東京大学教授 千葉 滋, 東京大学教授 五十嵐 健夫, 東京大学教授 松尾 宇泰, 東京大学講師 中山 英樹, 東京大学講師 蜂須賀 恵也University of Tokyo(東京大学

Fast and interactive ray-based rendering

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonDespite their age, ray-based rendering methods are still a very active field of research with many challenges when it comes to interactive visualization. In this thesis, we present our work on Guided High-Quality Rendering, Foveated Ray Tracing for Head Mounted Displays and Hash-based Hierarchical Caching and Layered Filtering. Our system for Guided High-Quality Rendering allows for guiding the sampling rate of ray-based rendering methods by a user-specified Region of Interest (RoI). We propose two interaction methods for setting such an RoI when using a large display system and a desktop display, respectively. This makes it possible to compute images with a heterogeneous sample distribution across the image plane. Using such a non-uniform sample distribution, the rendering performance inside the RoI can be significantly improved in order to judge specific image features. However, a modified scheduling method is required to achieve sufficient performance. To solve this issue, we developed a scheduling method based on sparse matrix compression, which has shown significant improvements in our benchmarks. By filtering the sparsely sampled image appropriately, large brightness variations in areas outside the RoI are avoided and the overall image brightness is similar to the ground truth early in the rendering process. When using ray-based methods in a VR environment on head-mounted display de vices, it is crucial to provide sufficient frame rates in order to reduce motion sickness. This is a challenging task when moving through highly complex environments and the full image has to be rendered for each frame. With our foveated rendering sys tem, we provide a perception-based method for adjusting the sample density to the user’s gaze, measured with an eye tracker integrated into the HMD. In order to avoid disturbances through visual artifacts from low sampling rates, we introduce a reprojection-based rendering pipeline that allows for fast rendering and temporal accumulation of the sparsely placed samples. In our user study, we analyse the im pact our system has on visual quality. We then take a closer look at the recorded eye tracking data in order to determine tracking accuracy and connections between different fixation modes and perceived quality, leading to surprising insights. For previewing global illumination of a scene interactively by allowing for free scene exploration, we present a hash-based caching system. Building upon the concept of linkless octrees, which allow for constant-time queries of spatial data, our frame work is suited for rendering such previews of static scenes. Non-diffuse surfaces are supported by our hybrid reconstruction approach that allows for the visualization of view-dependent effects. In addition to our caching and reconstruction technique, we introduce a novel layered filtering framework, acting as a hybrid method between path space and image space filtering, that allows for the high-quality denoising of non-diffuse materials. Also, being designed as a framework instead of a concrete filtering method, it is possible to adapt most available denoising methods to our layered approach instead of relying only on the filtering of primary hitpoints