Reflectance Hashing for Material Recognition

We introduce a novel method for using reflectance to identify materials. Reflectance offers a unique signature of the material but is challenging to measure and use for recognizing materials due to its high-dimensionality. In this work, one-shot reflectance is captured using a unique optical camera measuring {\it reflectance disks} where the pixel coordinates correspond to surface viewing angles. The reflectance has class-specific stucture and angular gradients computed in this reflectance space reveal the material class. These reflectance disks encode discriminative information for efficient and accurate material recognition. We introduce a framework called reflectance hashing that models the reflectance disks with dictionary learning and binary hashing. We demonstrate the effectiveness of reflectance hashing for material recognition with a number of real-world materials

Measurement, modeling and perception of painted surfaces : A Multi-scale analysis of the touch-up problem

Real-world surfaces typically have geometric features at a range of spatial scales. At the microscale, opaque surfaces are often characterized by bidirectional reflectance distribution functions (BRDF), which describes how a surface scatters incident light. At the mesoscale, surfaces often exhibit visible texture - stochastic or patterned arrangements of geometric features that provide visual information about surface properties such as roughness, smoothness, softness, etc. These textures also affect how light is scattered by the surface, but the effects are at a different spatial scale than those captured by the BRDF. Through this research, we investigate how microscale and mesoscale surface properties interact to contribute to overall surface appearance. This behavior is also the cause of the well-known touch-up problem in the paint industry, where two regions coated with exactly the same paint, look different in color, gloss and/or texture because of differences in application methods. At first, samples were created by applying latex paint to standard wallboard surfaces. Two application methods- spraying and rolling were used. The BRDF and texture properties of the samples were measured, which revealed differences at both the microscale and mesoscale. This data was then used as input for a physically-based image synthesis algorithm, to generate realistic images of the surfaces under different viewing conditions. In order to understand the factors that govern touch-up visibility, psychophysical tests were conducted using calibrated, digital photographs of the samples as stimuli. Images were presented in pairs and a two alternative forced choice design was used for the experiments. These judgments were then used as data for a Thurstonian scaling analysis to produce psychophysical scales of visibility, which helped determine the effect of paint formulation, application methods, and viewing and illumination conditions on the touch-up problem. The results can be used as base data towards development of a psychophysical model that relates physical differences in paint formulation and application methods to visual differences in surface appearance

The Impact of Surface Normals on Appearance

The appearance of an object is the result of complex light interaction with the object. Beyond the basic interplay between incident light and the object\u27s material, a multitude of physical events occur between this illumination and the microgeometry at the point of incidence, and also beneath the surface. A given object, made as smooth and opaque as possible, will have a completely different appearance if either one of these attributes - amount of surface mesostructure (small-scale surface orientation) or translucency - is altered. Indeed, while they are not always readily perceptible, the small-scale features of an object are as important to its appearance as its material properties. Moreover, surface mesostructure and translucency are inextricably linked in an overall effect on appearance. In this dissertation, we present several studies examining the importance of surface mesostructure (small-scale surface orientation) and translucency on an object\u27s appearance. First, we present an empirical study that establishes how poorly a mesostructure estimation technique can perform when translucent objects are used as input. We investigate the two major factors in determining an object\u27s translucency: mean free path and scattering albedo. We exhaustively vary the settings of these parameters within realistic bounds, examining the subsequent blurring effect on the output of a common shape estimation technique, photometric stereo. Based on our findings, we identify a dramatic effect that the input of a translucent material has on the quality of the resultant estimated mesostructure. In the next project, we discuss an optimization technique for both refining estimated surface orientation of translucent objects and determining the reflectance characteristics of the underlying material. For a globally planar object, we use simulation and real measurements to show that the blurring effect on normals that was observed in the previous study can be recovered. The key to this is the observation that the normalization factor for recovered normals is proportional to the error on the accuracy of the blur kernel created from estimated translucency parameters. Finally, we frame the study of the impact of surface normals in a practical, image-based context. We discuss our low-overhead, editing tool for natural images that enables the user to edit surface mesostructure while the system automatically updates the appearance in the natural image. Because a single photograph captures an instant of the incredibly complex interaction of light and an object, there is a wealth of information to extract from a photograph. Given a photograph of an object in natural lighting, we allow mesostructure edits and infer any missing reflectance information in a realistically plausible way

Acquisition, Modeling, and Augmentation of Reflectance for Synthetic Optical Flow Reference Data

This thesis is concerned with the acquisition, modeling, and augmentation of material reflectance to simulate high-fidelity synthetic data for computer vision tasks. The topic is covered in three chapters: I commence with exploring the upper limits of reflectance acquisition. I analyze state-of-the-art BTF reflectance field renderings and show that they can be applied to optical flow performance analysis with closely matching performance to real-world images. Next, I present two methods for fitting efficient BRDF reflectance models to measured BTF data. Both methods combined retain all relevant reflectance information as well as the surface normal details on a pixel level. I further show that the resulting synthesized images are suited for optical flow performance analysis, with a virtually identical performance for all material types. Finally, I present a novel method for augmenting real-world datasets with physically plausible precipitation effects, including ground surface wetting, water droplets on the windshield, and water spray and mists. This is achieved by projecting the realworld image data onto a reconstructed virtual scene, manipulating the scene and the surface reflectance, and performing unbiased light transport simulation of the precipitation effects