Modeling Camera Effects to Improve Visual Learning from Synthetic Data

Recent work has focused on generating synthetic imagery to increase the size and variability of training data for learning visual tasks in urban scenes. This includes increasing the occurrence of occlusions or varying environmental and weather effects. However, few have addressed modeling variation in the sensor domain. Sensor effects can degrade real images, limiting generalizability of network performance on visual tasks trained on synthetic data and tested in real environments. This paper proposes an efficient, automatic, physically-based augmentation pipeline to vary sensor effects --chromatic aberration, blur, exposure, noise, and color cast-- for synthetic imagery. In particular, this paper illustrates that augmenting synthetic training datasets with the proposed pipeline reduces the domain gap between synthetic and real domains for the task of object detection in urban driving scenes

Rendering non-pictorial (Scientific) high dynamic range images

In recent years, the graphics community is seeing an increasing demand for the capture and usage of high-dynamic-range (HDR) images. Since the production of HDR imagery is not solely the domain of the visualization of real life or computer generated scenes, novel techniques are also required for imagery captured from non-visual sources such as remote sensing, medical imaging, astronomical imaging, etc. This research proposes to integrate the techniques used for the display of high-dynamic-range pictorial imagery for the practical visualization of non-pictorial (scientific) imagery for data mining and interpretation. Nine algorithms were utilized to overcome the problem associated with rendering the high-dynamic-range image data to low-dynamic-range display devices, and the results were evaluated using a psychophysical experiment. Two paired-comparison experiments and a target detection experiment were performed. Paired-comparison results indicate that the Zone System performs the best on average and the Local Color Correction method performs the worst. The results show that the performance of different encoding schemes depend on the type of data being visualized. The correlation between the preference and scientific usefulness judgments (R2 = 0.31) demonstrates that observers tend to use different criteria when judging the scientific usefulness versus image preference. The experiment was conducted using observers with expertise (Radiologists) for the Medical image to further elucidate the success of HDR rendering on these data. The results indicated that both Radiologists and Non-radiologists tend to use similar criteria regardless of their experience and expertise when judging the usefulness of rendered images. A target detection experiment was conducted to measure the detectability of an embedded noise target in the Medical image to demonstrate the effect of the tone mapping operators on target detection. The result of the target detection experiment illustrated that the detectability of targets the image is greatly influenced by the rendering algorithm due to the inherent differences in tone mapping among the algorithms

Compression, Modeling, and Real-Time Rendering of Realistic Materials and Objects

The realism of a scene basically depends on the quality of the geometry, the illumination and the materials that are used. Whereas many sources for the creation of three-dimensional geometry exist and numerous algorithms for the approximation of global illumination were presented, the acquisition and rendering of realistic materials remains a challenging problem. Realistic materials are very important in computer graphics, because they describe the reflectance properties of surfaces, which are based on the interaction of light and matter. In the real world, an enormous diversity of materials can be found, comprising very different properties. One important objective in computer graphics is to understand these processes, to formalize them and to finally simulate them. For this purpose various analytical models do already exist, but their parameterization remains difficult as the number of parameters is usually very high. Also, they fail for very complex materials that occur in the real world. Measured materials, on the other hand, are prone to long acquisition time and to huge input data size. Although very efficient statistical compression algorithms were presented, most of them do not allow for editability, such as altering the diffuse color or mesostructure. In this thesis, a material representation is introduced that makes it possible to edit these features. This makes it possible to re-use the acquisition results in order to easily and quickly create deviations of the original material. These deviations may be subtle, but also substantial, allowing for a wide spectrum of material appearances. The approach presented in this thesis is not based on compression, but on a decomposition of the surface into several materials with different reflection properties. Based on a microfacette model, the light-matter interaction is represented by a function that can be stored in an ordinary two-dimensional texture. Additionally, depth information, local rotations, and the diffuse color are stored in these textures. As a result of the decomposition, some of the original information is inevitably lost, therefore an algorithm for the efficient simulation of subsurface scattering is presented as well. Another contribution of this work is a novel perception-based simplification metric that includes the material of an object. This metric comprises features of the human visual system, for example trichromatic color perception or reduced resolution. The proposed metric allows for a more aggressive simplification in regions where geometric metrics do not simplif