State of the Art on Neural Rendering

Efficient rendering of photo-realistic virtual worlds is a long standing effort of computer graphics. Modern graphics techniques have succeeded in synthesizing photo-realistic images from hand-crafted scene representations. However, the automatic generation of shape, materials, lighting, and other aspects of scenes remains a challenging problem that, if solved, would make photo-realistic computer graphics more widely accessible. Concurrently, progress in computer vision and machine learning have given rise to a new approach to image synthesis and editing, namely deep generative models. Neural rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e.g., by the integration of differentiable rendering into network training. With a plethora of applications in computer graphics and vision, neural rendering is poised to become a new area in the graphics community, yet no survey of this emerging field exists. This state-of-the-art report summarizes the recent trends and applications of neural rendering. We focus on approaches that combine classic computer graphics techniques with deep generative models to obtain controllable and photo-realistic outputs. Starting with an overview of the underlying computer graphics and machine learning concepts, we discuss critical aspects of neural rendering approaches. This state-of-the-art report is focused on the many important use cases for the described algorithms such as novel view synthesis, semantic photo manipulation, facial and body reenactment, relighting, free-viewpoint video, and the creation of photo-realistic avatars for virtual and augmented reality telepresence. Finally, we conclude with a discussion of the social implications of such technology and investigate open research problems

Realistic Aging of Materials in Computer Graphics

One of the most challenging tasks in Computer Graphics (CG) is depicting the accurate appearance of aging and weathered materials. This thesis examines the physical aging process of materials and translates that information into data that can be applied to CG materials resulting in a new prototype system for simulating realistic aging and weathering of CG materials. This new system will enable artists to quickly and accurately generate materials with a realistic appearance of aging and weathering. The resulting user interface generated from this system allows artists to create a variety of realistic and customized layered materials which offers a wide array of complexities. This prototype was then implemented into a studio setting which helped speed up the production process for material generation

Usage Based Materials by Simulating Layered Imperfections

Maintaining the ability to make quick iterations is very important to any artist in Computer Graphics, which is not always easy for simulating realistic materials based on how they are used. This thesis will examine imperfections in materials and the way different imperfections interact with each other based on how they are used. A new system will be created to save artist time by simulating how imperfections are layered and positioned

Usage Based Materials by Simulating Layered Imperfections

Maintaining the ability to make quick iterations is very important to any artist in Computer Graphics, which is not always easy for simulating realistic materials based on how they are used. This thesis will examine imperfections in materials and the way different imperfections interact with each other based on how they are used. A new system will be created to save artist time by simulating how imperfections are layered and positioned

REALISM IN VISUAL MATERIALS : CLEARING THE GROUND

Visuals via simple line drawings which are less realistic are more effective than realistic visuals with a high level of iconic stimuli when presented with ample time (Dwyer, 1978). In addition, a study (Wise, 1983) suggests that field dependence/independence would not significantly affect visual information processing of learners, while it is also suggested that field dependence/independence would be a very important factor in visual instructional materials (Witkin, 1977). For example, how field dependence/independence would affect such learning tasks as paper folding and unfolded figures which require visual-spatial skills is not intensively studied yet. Literature reviewed in this short paper suggests the necessity of intensive study on a learner\u27s field dependence/independence and visual learning on various learning tasks. Research indicates that field dependent/independent dimension would make a difference in a learning task which requires a learner\u27s visual-spatial skills. As more computer graphics are introduced to educational fields with more ease in production and utilization, computer-generated images will be more frequently seen in school settings whether school people like it or not. In this paper, previous studies on visual learning were reviewed with some future possible research suggested

A study of user perceptions of the relationship between bump-mapped and non-bump-mapped materials, and lighting intensity in a real-time virtual environment

The video and computer games industry has taken full advantage of the human sense of vision by producing games that utilize complex high-resolution textures and materials, and lighting technique. This results to the creation of an almost life-like real-time 3D virtual environment that can immerse the end-users. One of the visual techniques used is real-time display of bump-mapped materials. However, this sense of visual phenomenon has yet to be fully utilized for 3D design visualization in the architecture and construction domain. Virtual environments developed in the architecture and construction domain are often basic and use low-resolution images, which under represent the real physical environment. Such virtual environment is seen as being non-realistic to the user resulting in a misconception of the actual potential of it as a tool for 3D design visualization. A study was conducted to evaluate whether subjects can see the difference between bump-mapped and nonbump-mapped materials in different lighting conditions. The study utilized a real-time 3D virtual environment that was created using a custom-developed software application tool called BuildITC4. BuildITC4 was developed based upon the C4Engine which is classified as a next-generation 3D Game Engine. A total of thirty-five subjects were exposed to the virtual environment and were asked to compare the various types of material in different lighting conditions. The number of lights activated, the lighting intensity, and the materials used in the virtual environment were all interactive and changeable in real-time. The goal is to study how subjects perceived bump-mapped and non-bump mapped materials, and how different lighting conditions affect realistic representation. Results from this study indicate that subjects could tell the difference between the bump-mapped and non-bump mapped materials, and how different material reacts to different lighting condition

Compression, Modeling, and Real-Time Rendering of Realistic Materials and Objects

The realism of a scene basically depends on the quality of the geometry, the illumination and the materials that are used. Whereas many sources for the creation of three-dimensional geometry exist and numerous algorithms for the approximation of global illumination were presented, the acquisition and rendering of realistic materials remains a challenging problem. Realistic materials are very important in computer graphics, because they describe the reflectance properties of surfaces, which are based on the interaction of light and matter. In the real world, an enormous diversity of materials can be found, comprising very different properties. One important objective in computer graphics is to understand these processes, to formalize them and to finally simulate them. For this purpose various analytical models do already exist, but their parameterization remains difficult as the number of parameters is usually very high. Also, they fail for very complex materials that occur in the real world. Measured materials, on the other hand, are prone to long acquisition time and to huge input data size. Although very efficient statistical compression algorithms were presented, most of them do not allow for editability, such as altering the diffuse color or mesostructure. In this thesis, a material representation is introduced that makes it possible to edit these features. This makes it possible to re-use the acquisition results in order to easily and quickly create deviations of the original material. These deviations may be subtle, but also substantial, allowing for a wide spectrum of material appearances. The approach presented in this thesis is not based on compression, but on a decomposition of the surface into several materials with different reflection properties. Based on a microfacette model, the light-matter interaction is represented by a function that can be stored in an ordinary two-dimensional texture. Additionally, depth information, local rotations, and the diffuse color are stored in these textures. As a result of the decomposition, some of the original information is inevitably lost, therefore an algorithm for the efficient simulation of subsurface scattering is presented as well. Another contribution of this work is a novel perception-based simplification metric that includes the material of an object. This metric comprises features of the human visual system, for example trichromatic color perception or reduced resolution. The proposed metric allows for a more aggressive simplification in regions where geometric metrics do not simplif