Relighting Photographs of Tree Canopies

International audienceWe present an image-based approach to relighting photographs of tree canopies. Our goal is to minimize capture overhead; thus the only input required is a set of photographs of the tree taken at a single time of day, while allowing relighting at any other time. We first analyze lighting in a tree canopy both theoretically and using simulations. From this analysis, we observe that tree canopy lighting is similar to volumetric illumination. We assume a single-scattering volumetric lighting model for tree canopies, and diffuse leaf reflectance; we validate our assumptions with synthetic renderings. We create a volumetric representation of the tree from 10-12 images taken at a single time of day and and use a single-scattering participating media lighting model. An analytical sun and sky illumination model provides consistent representation of lighting for the captured input and unknown target times. We relight the input image by applying a ratio of the target and input time lighting representations. We compute this representation efficiently by simultaneously coding transmittance from the sky and to the eye in spherical harmonics. We validate our method by relighting images of synthetic trees and comparing to path-traced solutions. We also present results for photographs where sparse, validating with time-lapse ground truth sequences

Estimation of environmental lighting from human face for illumination of augmented reality scenes

In this thesis, we propose a method to solve a common problem in augmented reality domain; estimating light sources in an outdoor scene and lighting virtual objects accordingly. As a basis of our method we developed a framework based on estimation of environmental lighting from well defined objects, specifically human faces. The method is tuned for outdoor use, and the algorithm is further enhanced to illuminate virtual objects exposed to direct sunlight. In the first part of this thesis, we propose a novel lighting estimation technique where we assume a user is looking straight to mobile devices camera. This technique extracts information from input images to calculate possible light sources to pass to the rendering stage. In the second part of this thesis, we propose a lighting model which uses the output from our lighting estimation in order to make objects appear as they are lit correctly by the sun light. This model uses a mathematical technique called Spherical Harmonics Lighting for real-time realistic rendering

Efficient rendering of atmospheric phenomena

Journal ArticleRendering of atmospheric bodies involves modeling the complex interaction of light throughout the highly scattering medium of water and air particles. Scattering by these particles creates many well-known atmospheric optical phenomena including rainbows, halos, the corona, and the glory. Unfortunately, most radiative transport approximations in computer graphics are ill-suited to render complex angularly dependent effects in the presence of multiple scattering at reasonable frame rates. Therefore, this paper introduces a multiple-model lighting system that efficiently captures these essential atmospheric effects. We have solved the rendering of fine angularly dependent effects in the presence of multiple scattering by designing a lighting approximation based upon multiple scattering phase functions. This model captures gradual blurring of chromatic atmospheric optical phenomena by handling the gradual angular spreading of the sunlight as it experiences multiple scattering events with anisotropic scattering particles. It has been designed to take advantage of modern graphics hardware; thus, it is capable of rendering these effects at near interactive frame rates

SOL-NeRF:Sunlight Modeling for Outdoor Scene Decomposition and Relighting

Outdoor scenes often involve large-scale geometry and complex unknown lighting conditions, making it difficult to decompose them into geometry, reflectance and illumination. Recently researchers made attempts to decompose outdoor scenes using Neural Radiance Fields (NeRF) and learning-based lighting and shadow representations. However, diverse lighting conditions and shadows in outdoor scenes are challenging for learning-based models. Moreover, existing methods may produce rough geometry and normal reconstruction and introduce notable shading artifacts when the scene is rendered under a novel illumination. To solve the above problems, we propose SOL-NeRF to decompose outdoor scenes with the help of a hybrid lighting representation and a signed distance field geometry reconstruction. We use a single Spherical Gaussian (SG) lobe to approximate the sun lighting, and a first-order Spherical Harmonic (SH) mixture to resemble the sky lighting. This hybrid representation is specifically designed for outdoor settings, and compactly models the outdoor lighting, ensuring robustness and efficiency. The shadow of the direct sun lighting can be obtained by casting the ray against the mesh extracted from the signed distance field, and the remaining shadow can be approximated by Ambient Occlusion (AO). Additionally, sun lighting color prior and a relaxed Manhattan-world assumption can be further applied to boost decomposition and relighting performance. When changing the lighting condition, our method can produce consistent relighting results with correct shadow effects. Experiments conducted on our hybrid lighting scheme and the entire decomposition pipeline show that our method achieves better reconstruction, decomposition, and relighting performance compared to previous methods both quantitatively and qualitatively.</p

Daylight simulation with photon maps

Physically based image synthesis remains one of the most demanding tasks in the computer graphics field, whose applications have evolved along with the techniques in recent years, particularly with the decline in cost of powerful computing hardware. Physically based rendering is essentially a niche since it goes beyond the photorealistic look required by mainstream applications with the goal of computing actual lighting levels in physical quantities within a complex 3D scene. Unlike mainstream applications which merely demand visually convincing images and short rendering times, physically based rendering emphasises accuracy at the cost of increased computational overhead. Among the more specialised applications for physically based rendering is lighting simulation, particularly in conjunction with daylight. The aim of this thesis is to investigate the applicability of a novel image synthesis technique based on Monte Carlo particle transport to daylight simulation. Many materials used in daylight simulation are specifically designed to redirect light, and as such give rise to complex effects such as caustics. The photon map technique was chosen for its efficent handling of these effects. To assess its ability to produce physically correct results which can be applied to lighting simulation, a validation was carried out based on analytical case studies and on simple experimental setups. As prerequisite to validation, the photon map\u27s inherent bias/noise tradeoff is investigated. This tradeoff depends on the density estimate bandwidth used in the reconstruction of the illumination. The error analysis leads to the development of a bias compensating operator which adapts the bandwidth according to the estimated bias in the reconstructed illumination. The work presented here was developed at the Fraunhofer Institute for Solar Energy Systems (ISE) as part of the FARESYS project sponsored by the German national research foundation (DFG), and embedded into the RADIANCE rendering system.Die Erzeugung physikalisch basierter Bilder gilt heute noch als eine der rechenintensivsten Aufgaben in der Computergraphik, dessen Anwendungen sowie auch Verfahren in den letzten Jahren kontinuierlich weiterentwickelt wurden, vorangetrieben primär durch den Preisverfall leistungsstarker Hardware. Physikalisch basiertes Rendering hat sich als Nische etabliert, die über die photorealistischen Anforderungen typischer Mainstream-Applikationen hinausgeht, mit dem Ziel, Lichttechnische Größen innerhalb einer komplexen 3D Szene zu berechnen. Im Gegensatz zu Mainstream-Applikationen, die visuell überzeugend wirken sollen und kurze Rechenzeiten erforden, liegt der Schwerpunkt bei physikalisch basiertem Rendering in der Genauigkeit, auf Kosten des Rechenaufwands. Zu den eher spezialisierten Anwendungen im Gebiet des physikalisch basiertem Renderings gehört die Lichtsimulation, besonders in Bezug auf Tageslicht. Das Ziel dieser Dissertation liegt darin, die Anwendbarkeit eines neuartigen Renderingverfahrens basierend auf Monte Carlo Partikeltransport hinsichtlich Tageslichtsimulation zu untersuchen. Viele Materialien, die in der Tageslichtsimulation verwendet werden, sind speziell darauf konzipiert, Tageslicht umzulenken, und somit komplexe Phänomene wie Kaustiken hervorrufen. Das Photon-Map-Verfahren wurde aufgrund seiner effizienten Simulation solcher Effekte herangezogen. Zur Beurteilung seiner Fähigkeit, physikalisch korrekte Ergebnisse zu liefern, die in der Tageslichtsimulation anwendbar sind, wurde eine Validierung anhand analytischer Studien sowie eines einfachen experimentellen Aufbaus durchgeführt. Als Voraussetzung zur Validierung wurde der Photon Map bezüglich seiner inhärenten Wechselwirkung zwischen Rauschen und systematischem Fehler (Bias) untersucht. Diese Wechselwirkung hängt von der Bandbreite des Density Estimates ab, mit dem die Beleuchtung aus den Photonen rekonstruiert wird. Die Fehleranalyse führt zur Entwicklung eines Bias compensating Operators, der die Bandbreite dynamisch anhand des geschätzten Bias in der rekonstruierten Beleuchtung anpasst. Die hier vorgestellte Arbeit wurde am Fraunhofer Institut für Solare Energiesysteme (ISE) als teil des FARESYS Projekts entwickelt, daß von der Deutschen Forschungsgemeinschaft (DFG) finanziert wurde. Die Implementierung erfolgte im Rahmen des RADIANCE Renderingsystems