Predicted Virtual Soft Shadow Maps with High Quality Filtering

International audienceIn this paper we present a novel image based algorithm to render visually plausible anti-aliased soft shadows in a robust and efficient manner. To achieve both high visual quality and high performance, it employs an accurate shadow map filtering method which guarantees smooth penumbrae and high quality anisotropic anti-aliasing of the sharp transitions. Unlike approaches based on pre-filtering approximations, our approach does not suffer from light bleeding or losing contact shadows. Discretization artefacts are avoided by creating virtual shadow maps on the fly according to a novel shadow map resolution prediction model. This model takes into account the screen space frequency of the penumbrae via a perceptual metric which has been directly established from an appropriate user study. Consequently, our algorithm always generates shadow maps with minimal resolutions enabling high performance while guarantying high quality. Thanks to this perceptual model, our algorithm can sometimes be faster at rendering soft shadows than hard shadows. It can render game-like scenes at very high frame rates, and extremely large and complex scenes such as CAD models at interactive rates. In addition, our algorithm is highly scalable, and the quality versus performance trade-off can be easily tweaked

LivePhantom: Retrieving Virtual World Light Data to Real Environments.

To achieve realistic Augmented Reality (AR), shadows play an important role in creating a 3D impression of a scene. Casting virtual shadows on real and virtual objects is one of the topics of research being conducted in this area. In this paper, we propose a new method for creating complex AR indoor scenes using real time depth detection to exert virtual shadows on virtual and real environments. A Kinect camera was used to produce a depth map for the physical scene mixing into a single real-time transparent tacit surface. Once this is created, the camera's position can be tracked from the reconstructed 3D scene. Real objects are represented by virtual object phantoms in the AR scene enabling users holding a webcam and a standard Kinect camera to capture and reconstruct environments simultaneously. The tracking capability of the algorithm is shown and the findings are assessed drawing upon qualitative and quantitative methods making comparisons with previous AR phantom generation applications. The results demonstrate the robustness of the technique for realistic indoor rendering in AR systems

Controllable Shadow Generation Using Pixel Height Maps

Shadows are essential for realistic image compositing. Physics-based shadow rendering methods require 3D geometries, which are not always available. Deep learning-based shadow synthesis methods learn a mapping from the light information to an object's shadow without explicitly modeling the shadow geometry. Still, they lack control and are prone to visual artifacts. We introduce pixel heigh, a novel geometry representation that encodes the correlations between objects, ground, and camera pose. The pixel height can be calculated from 3D geometries, manually annotated on 2D images, and can also be predicted from a single-view RGB image by a supervised approach. It can be used to calculate hard shadows in a 2D image based on the projective geometry, providing precise control of the shadows' direction and shape. Furthermore, we propose a data-driven soft shadow generator to apply softness to a hard shadow based on a softness input parameter. Qualitative and quantitative evaluations demonstrate that the proposed pixel height significantly improves the quality of the shadow generation while allowing for controllability.Comment: 15 pages, 11 figure

Master of Science in Computing

thesisThis document introduces the Soft Shadow Mip-Maps technique, which consists of three methods for overcoming the fundamental limitations of filtering-oriented soft shadows. Filtering-oriented soft shadowing techniques filter shadow maps with varying filter sizes determined by desired penumbra widths. Different varieties of this approach have been commonly applied in interactive and real-time applications. Nonetheless, they share some fundamental limitations. First, soft shadow filter size is not always guaranteed to be the correct size for producing the right penumbra width based on the light source size. Second, filtering with large kernels for soft shadows requires a large number of samples, thereby increasing the cost of filtering. Stochastic approximations for filtering introduce noise and prefiltering leads to inaccuracies. Finally, calculating shadows based on a single blocker estimation can produce significantly inaccurate penumbra widths when the shadow penumbras of different blockers overlap. We discuss three methods to overcome these limitations. First, we introduce a method for computing the soft shadow filter size for a receiver with a blocker distance. Then, we present a filtering scheme based on shadow mip-maps. Mipmap-based filtering uses shadow mip-maps to efficiently generate soft shadows using a constant size filter kernel for each layer, and linear interpolation between layers. Finally, we introduce an improved blocker estimation approach. With the improved blocker estimaiton, we explore the shadow contribution of every blocker by calculating the light occluded by potential blockers. Hence, the calculated penumbra areas correspond to the blockers correctly. Finally, we discuss how to select filter kernels for filtering. These approaches successively solve issues regarding shadow penumbra width calculation apparent in prior techniques. Our result shows that we can produce correct penumbra widths, as evident in our comparisons to ray-traced soft shadows. Nonetheless, the Soft Shadow Mip-Maps technique suffers from light bleeding issues. This is because our method only calculates shadows using the geometry that is available in the shadow depth map. Therefore, the occluded geometry is not taken into consideration, which leads to light bleeding. Another limitation of our method is that using lower resolution shadow mip-map layers limits the resolution of the shadow placement. As a result, when a blocker moves slowly, its shadow follows it with discrete steps, the size of which is determined by the corresponding mip-map layer resolution

Rendering of light shaft and shadow for indoor environments enhancing technique

The ray marching methods have become the most attractive method to provide realism in rendering the effects of light scattering in the participating media of numerous applications. This has attracted significant attention from the scientific community. Up-sampling of ray marching methods is suitable to evaluate light scattering effects such as volumetric shadows and light shafts for rendering realistic scenes, but suffers of cost a lot for rendering. Therefore, some encouraging outcomes have been achieved by using down-sampling of ray marching approach to accelerate rendered scenes. However, these methods are inherently prone to artifacts, aliasing and incorrect boundaries due to the reduced number of sample points along view rays. This study proposed a new enhancing technique to render light shafts and shadows taking into consideration the integration light shafts, volumetric shadows, and shadows for indoor environments. This research has three major phases that cover species of the effects addressed in this thesis. The first phase includes the soft volumetric shadows creation technique called Soft Bilateral Filtering Volumetric Shadows (SoftBiF-VS). The soft shadow was created using a new algorithm called Soft Bilateral Filtering Shadow (SBFS). This technique was started by developing an algorithm called Imperfect Multi-View Soft Shadows (IMVSSs) based on down-sampling multiple point lights (DMPLs) and multiple depth maps, which are processed by using bilateral filtering to obtain soft shadows. Then, down-sampling light scattering model was used with (SBFS) to create volumetric shadows, which was improved using cross-bilateral filter to get soft volumetric shadows. In the second phase, soft light shaft was generated using a new technique called Realistic Real-Time Soft Bilateral Filtering Light Shafts (realTiSoftLS). This technique computed the light shaft depending on down-sampling volumetric light model and depth test, and was interpolated by bilateral filtering to gain soft light shafts. Finally, an enhancing technique for integrating all of these effects that represent the third phase of this research was achieved. The performance of the new enhanced technique was evaluated quantitatively and qualitatively a measured using standard dataset. Results from the experiment showed that 63% of the participants gave strong positive responses to this technique of improving realism. From the quantitative evaluation, the results revealed that the technique has dramatically outpaced the stateof- the-art techniques with a speed of 74 fps in improving the performance for indoor environments

Image-based rendering of ancient Chinese artifacts for multi-view displays - a multi-camera approach

Image-based rendering (IBR) is an emerging and promising technology for photo-realistic rendering of scenes and objects from a collection of densely sampled images and videos. This paper proposes an image-based approach to the rendering and multi-view display of ancient Chinese artifacts for cultural heritage preservation. A multiple-camera circular array was constructed to record images of the artifacts. Novel techniques for segmenting and rendering new views of the artifacts from the sampled images are developed. The multiple views so synthesized enable the ancient artifacts to be displayed in modern multi-view displays and conventional stereo systems. Several collections from the University Museum and Art Gallery at the University of Hong Kong are captured and excellent rendering results are obtained. ©2010 IEEE.published_or_final_versionThe 2010 IEEE International Symposium on Circuits and Systems (ISCAS), Paris, France, 30 May-2 June 2010. In IEEE International Symposium on Circuits and Systems Proceedings, 2010, p. 3252-325

Photorealistic rendering: a survey on evaluation

This article is a systematic collection of existing methods and techniques for evaluating rendering category in the field of computer graphics. The motive for doing this study was the difficulty of selecting appropriate methods for evaluating and validating specific results reported by many researchers. This difficulty lies in the availability of numerous methods and lack of robust discussion of them. To approach such problems, the features of well-known methods are critically reviewed to provide researchers with backgrounds on evaluating different styles in photo-realistic rendering part of computer graphics. There are many ways to evaluating a research. For this article, classification and systemization method is use. After reviewing the features of different methods, their future is also discussed. Finally, dome pointers are proposed as to the likely future issues in evaluating the research on realistic rendering. It is expected that this analysis helps researchers to overcome the difficulties of evaluation not only in research, but also in application

Differentiable Shadow Mapping for Efficient Inverse Graphics

We show how shadows can be efficiently generated in differentiable rendering of triangle meshes. Our central observation is that pre-filtered shadow mapping, a technique for approximating shadows based on rendering from the perspective of a light, can be combined with existing differentiable rasterizers to yield differentiable visibility information. We demonstrate at several inverse graphics problems that differentiable shadow maps are orders of magnitude faster than differentiable light transport simulation with similar accuracy -- while differentiable rasterization without shadows often fails to converge.Comment: CVPR 2023, project page: https://mworchel.github.io/differentiable-shadow-mappin

Efficient multi-bounce lightmap creation using GPU forward mapping

Computer graphics can nowadays produce images in realtime that are hard to distinguish from photos of a real scene. One of the most important aspects to achieve this is the interaction of light with materials in the virtual scene. The lighting computation can be separated in two different parts. The first part is concerned with the direct illumination that is applied to all surfaces lit by a light source; algorithms related to this have been greatly improved over the last decades and together with the improvements of the graphics hardware can now produce realistic effects. The second aspect is about the indirect illumination which describes the multiple reflections of light from each surface. In reality, light that hits a surface is never fully absorbed, but instead reflected back into the scene. And even this reflected light is then reflected again and again until its energy is depleted. These multiple reflections make indirect illumination very computationally expensive. The first problem regarding indirect illumination is therefore, how it can be simplified to compute it faster. Another question concerning indirect illumination is, where to compute it. It can either be computed in the fixed image that is created when rendering the scene or it can be stored in a light map. The drawback of the first approach is, that the results need to be recomputed for every frame in which the camera changed. The second approach, on the other hand, is already used for a long time. Once a static scene has been set up, the lighting situation is computed regardless of the time it takes and the result is then stored into a light map. This is a texture atlas for the scene in which each surface point in the virtual scene has exactly one surface point in the 2D texture atlas. When displaying the scene with this approach, the indirect illumination does not need to be recomputed, but is simply sampled from the light map. The main contribution of this thesis is the development of a technique that computes the indirect illumination solution for a scene at interactive rates and stores the result into a light atlas for visualizing it. To achieve this, we overcome two main obstacles. First, we need to be able to quickly project data from any given camera configuration into the parts of the texture that are currently used for visualizing the 3D scene. Since our approach for computing and storing indirect illumination requires a huge amount of these projections, it needs to be as fast as possible. Therefore, we introduce a technique that does this projection entirely on the graphics card with a single draw call. Second, the reflections of light into the scene need to be computed quickly. Therefore, we separate the computation into two steps, one that quickly approximates the spreading of the light into the scene and a second one that computes the visually smooth final result using the aforementioned projection technique. The final technique computes the indirect illumination at interactive rates even for big scenes. It is furthermore very flexible to let the user choose between high quality results or fast computations. This allows the method to be used for quickly editing the lighting situation with high speed previews and then computing the final result in perfect quality at still interactive rates. The technique introduced for projecting data into the texture atlas is in itself highly flexible and also allows for fast painting onto objects and projecting data onto it, considering all perspective distortions and self-occlusions