Frequency Based Radiance Cache for Rendering Animations

International audienceWe propose a method to render animation sequences with direct distant lighting that only shades a fraction of the total pixels. We leverage frequency-based analyses of light transport to determine shading and image sampling rates across an animation using a samples cache. To do so, we derive frequency bandwidths that account for the complexity of distant lights, visibility, BRDF, and temporal coherence during animation. We finaly apply a cross-bilateral filter when rendering our final images from sparse sets of shading points placed according to our frequency-based oracles (generally < 25% of the pixels, per frame)

Nonlinearly Weighted First-order Regression for Denoising Monte Carlo Renderings

We address the problem of denoising Monte Carlo renderings by studying existing approaches and proposing a new algorithm that yields state-of-the-art performance on a wide range of scenes. We analyze existing approaches from a theoretical and empirical point of view, relating the strengths and limitations of their corresponding components with an emphasis on production requirements. The observations of our analysis instruct the design of our new filter that offers high-quality results and stable performance. A key observation of our analysis is that using auxiliary buffers (normal, albedo, etc.) to compute the regression weights greatly improves the robustness of zero-order models, but can be detrimental to first-order models. Consequently, our filter performs a first-order regression leveraging a rich set of auxiliary buffers only when fitting the data, and, unlike recent works, considers the pixel color alone when computing the regression weights. We further improve the quality of our output by using a collaborative denoising scheme. Lastly, we introduce a general mean squared error estimator, which can handle the collaborative nature of our filter and its nonlinear weights, to automatically set the bandwidth of our regression kernel

Real-Time Noise Removal in Foveated Path Tracing

Path tracing is a method for rendering photorealistic two-dimensional images of three-dimensional scenes based on computing intersection between the scene geometry and light rays traveling through the scene. The rise in parallel computation resources in devices such as graphics processing units (GPUs) have made it more and more viable to do path tracing in real time. To achieve real-time performance, path tracing can be further optimized by using foveated rendering, where the properties of the human visual system are exploited to reduce the number of rays outside the central point of vision (fovea), where the human eye cannot discern fine detail. The reduction in the number of rays can, however, lead to several issues. Noise appears in the image as a result of an inadequate number of path tracing samples allocated to each pixel. Furthermore, the variation in the noise from one animation frame to the next appears as flicker. Finally, artifacts can appear when the spatially subsampled image is upsampled to a uniform resolution for display. In this thesis, solutions to the aforementioned issues are explored by implementing three noise removal methods into a foveated path tracing rendering system. The computational performance and the visual quality of the implemented methods is evaluated. Of the implemented methods, cross-bilateral filter provides the best quality, but its runtime doesn't scale well to large filter sizes. Large filter sizes are enabled by the À-Trous approximation of the cross-bilateral filter, at the cost of generating more artifacts in the result. Overall, while the implemented methods are able to provide visually pleasing results in some scenarios, improvements in the algorithms (e.g., local filter parameter selection) are needed to reach the quality seen in offline methods

Learning Sample-Based Monte Carlo Denoising from Noisy Training Data

Monte Carlo rendering allows for the production of high-quality photorealistic images of 3D scenes. However, producing noise-free images can take a considerable amount of compute resources. To lessen this burden and speed up the rendering process while maintaining similar quality, a lower-sample count image can be rendered and then denoised after rendering with image-space denoising methods. These methods are widely used in industry, and have recently enabled advancements in areas such as real-time ray tracing. While hand-tuned denoisers are available, the most successful denoising methods are based on machine learning with deep convolutional neural networks (CNNs). These denoisers are trained on large datasets of rendered images, consisting of pairs of low-sample count noisy images and the corresponding high-sample count reference images. Unfortunately, generating these datasets can be prohibitively expensive because of the cost of rendering thousands of high-sample count reference images. A potential solution to this problem comes from the Noise2Noise method, where denoisers can be learned solely from noisy training data. Lehtinen et al. applied their technique to Monte Carlo denoising, and were able to achieve similar performance to using clean reference images. However, their model was a proof of concept, and it is unclear whether the technique would work equally well with state-of-the-art Monte Carlo denoising methods. The authors also do not test their hypothesis that better results could be achieved by training on the additional noisy training data that could be generated with the same compute budget that was previously allocated to generating clean training data. Finally, it remains to be seen whether the authors' suggested parameters are equally effective when Noise2Noise is used with different denoising methods. In this thesis, I answer the above questions by applying Noise2Noise to a state-of-the-art Monte Carlo denoising algorithm called Sample-Based Monte-Carlo Denoising (SBMC). I adapt the SBMC scene generator to produce a dataset of noisy image pairs, use this dataset to train an SBMC-like CNN, and conduct experiments to determine the impact of various parameters on the performance of the denoiser. My results show that the Noise2Noise technique can be effectively applied to a state-of-the-art Monte Carlo denoising algorithm. I achieved comparable results to the original implementation at a significantly lower cost. I find that using additional training data can further improve these results, although more investigation is needed in this area. Finally, I detail the parameters that were necessary to achieve these results

Path tracing for direct volume rendering with web technologies

Rendering of volumetric data is of great significance in numerous fields of science and technology, although it is far from being a trivial task. Interactive and real-time rendering is even more difficult to achieve. Majority of applications nowadays employ primitive methods to reach high execution speeds, and furthermore, they are often accessible only on specific platforms. The web revolution in recent years enabled us to use web browsers to access powerful graphics hardware and in turn build modern graphical application in a platform-agnostic manner. Therefore, in this work we combine state-of-the-art web technology with the latest advancements in volume rendering in a proof-of-concept web application for interactive, real-time and physically correct rendering of volumetric data of arbitrary origin that runs on a wide variety of desktop and mobile devices. The methods used are as general as possible so as to not impose any restrictions on interaction, scene, camera and lighting. With extensibility of the implementation in mind we propose a new pipeline model with a direct support for stochastic methods, which allows for simple extension of existing and fast testing of new rendering methods. With this work we bridge the currently ubiquitous gap between theory and practice in a wide range of use cases

Optimal prefilters for display enhancement

Creating images from a set of discrete samples is arguably the most common operation in computer graphics and image processing, lying, for example, at the heart of rendering and image downscaling techniques. Traditional tools for this task are based on classic sampling theory and are modeled under mathematical conditions which are, in most cases, unrealistic; for example, sinc reconstruction – required by Shannon theorem in order to recover a signal exactly – is impossible to achieve in practice because LCD displays perform a box-like interpolation of the samples. Moreover, when an image is made for a human to look at, it will necessarily undergo some modifications due to the human optical system and all the neural processes involved in vision. Finally, image processing practitioners noticed that sinc prefiltering – also required by Shannon theorem – often leads to visually unpleasant images. From these facts, we can deduce that we cannot guarantee, via classic sampling theory, that the signal we see in a display is the best representation of the original image we had in first place. In this work, we propose a novel family of image prefilters based on modern sampling theory, and on a simple model of how the human visual system perceives an image on a display. The use of modern sampling theory guarantees us that the perceived image, based on this model, is indeed the best representation possible, and at virtually no computational overhead. We analyze the spectral properties of these prefilters, showing that they offer the possibility of trading-off aliasing and ringing, while guaranteeing that images look sharper then those generated with both classic and state-of-the-art filters. Finally, we compare it against other solutions in a selection of applications which include Monte Carlo rendering and image downscaling, also giving directions on how to apply it in different contexts.Exibir imagens a partir de um conjunto discreto de amostras é certamente uma das operações mais comuns em computação gráfica e processamento de imagens. Ferramentas tradicionais para essa tarefa são baseadas no teorema de Shannon e são modeladas em condições matemáticas que são, na maior parte dos casos, irrealistas; por exemplo, reconstrução com sinc – necessária pelo teorema de Shannon para recuperar um sinal exatamente – é impossível na prática, já que displays LCD realizam uma reconstrução mais próxima de uma interpolação com kernel box. Além disso, profissionais em processamento de imagem perceberam que prefiltragem com sinc – também requerida pelo teorema de Shannon – em geral leva a imagens visualmente desagradáveis devido ao fenômeno de ringing: oscilações próximas a regiões de descontinuidade nas imagens. Desses fatos, deduzimos que não é possível garantir, via ferramentas tradicionais de amostragem e reconstrução, que a imagem que observamos em um display digital é a melhor representação para a imagem original. Neste trabalho, propomos uma família de prefiltros baseada em teoria de amostragem generalizada e em um modelo de como o sistema ótico do olho humano modifica uma imagem. Proposta por Unser and Aldroubi (1994), a teoria de amostragem generalizada é mais geral que o teorema proposto por Shannon, e mostra como é possível pré-filtrar e reconstruir sinais usando kernels diferentes do sinc. Modelamos o sistema ótico do olho como uma câmera com abertura finita e uma lente delgada, o que apesar de ser simples é suficiente para os nossos propósitos. Além de garantir aproximação ótima quando reconstruindo as amostras por um display e filtrando a imagem com o modelo do sistema ótico humano, a teoria de amostragem generalizada garante que essas operações são extremamente eficientes, todas lineares no número de pixels de entrada. Também, analisamos as propriedades espectrais desses filtros e de técnicas semelhantes na literatura, mostrando que é possível obter um bom tradeoff entre aliasing e ringing (principais artefatos quando lidamos com amostragem e reconstrução de imagens), enquanto garantimos que as imagens finais são mais nítidas que aquelas geradas por técnicas existentes na literatura. Finalmente, mostramos algumas aplicações da nossa técnica em melhoria de imagens, adaptação à distâncias de visualização diferentes, redução de imagens e renderização de imagens sintéticas por método de Monte Carlo