Stereo disparity improves color constancy

AbstractBinocular disparity is an aspect of natural viewing. This research investigates whether disparity affects surface color perception. Achromatic settings were obtained and compared for two stereograms of a scene with specular reflections, one stereogram with binocular disparity and one without it (cyclopean view). Binocular disparity was found to improve color constancy. Next, the geometry of specular highlights, which is distorted without binocular disparity, was specifically examined. Measurements compared color constancy with specular reflections that were either normal (with stereo disparity) or distorted (cyclopean view of the specularities). No significant change in constancy was found due to the geometrical distortion of specular highlights that occurs without stereo disparity, suggesting that constancy depends on other features of the percept affected by disparity. The results are discussed in terms of illuminant estimation in surface color perception

Segmentation of Specular Highlights from Object Surfaces

A major hindrance to image segmentation tasks are the presence of specular highlights on object surfaces. Specular highlights appear on object surfaces where the specular component of reflection from illuminating light sources is so dominant that most detail of the object surface is obscured by a bright region of reflected light. Specular highlights are very common artifacts of most lighting environments and are not part of the intrinsic visible detail of an object surface. As a result, in addition to obscuring visible detail, specular highlight regions of an image can easily deceive image understanding algorithms into interpreting these regions as separate objects or regions on an object with high albedo. Recently, a couple of approaches to identifying specular highlight regions in images of object surfaces have produced some good results using color analysis. Unfortunately these methods work only for dielectric materials (e.g. plastic, rubber etc.) and require that the color of the object be different from the color of the light source. In this paper a technique is presented exploiting the polarization properties of reflected light to identify specular highlight regions. This technique works for both dielectric and metal surfaces regardless of the color of the illuminating light source, or the color detail on the object surface. In addition to separating out diffuse and specular components of reflection, the technique presented here also as a bonus can identify whet her certain image regions correspond to a dielectric or metal object surface. Extensive experimentation will be presented for a variety of dielectric and metal surfaces, both polished and rough. Experimentation with coated surfaces using the technique presented here have not yet been studied

On Recognizing Transparent Objects in Domestic Environments Using Fusion of Multiple Sensor Modalities

Current object recognition methods fail on object sets that include both diffuse, reflective and transparent materials, although they are very common in domestic scenarios. We show that a combination of cues from multiple sensor modalities, including specular reflectance and unavailable depth information, allows us to capture a larger subset of household objects by extending a state of the art object recognition method. This leads to a significant increase in robustness of recognition over a larger set of commonly used objects.Comment: 12 page

Design principles of hardware-based phong shading and bump-mapping

The VISA+ hardware architecture is the first of a new generation of graphics accelerators designed primarily to render bump-, texture-, environment- and environment-bump-mapped polygons. This paper presents examples of the main graphical capabilities and discusses methods and simplifications used to create high quality images. One of the key concepts in the VISA+ design, the use of reflectance cubes, is predestined for environment mapping. In combination with bump- and texture-mapping it shows the strength of our new architecture. Furthermore it justifies some of the decisions made during simulation and development of the complex VISA+ architecture

Head Tracking via Robust Registration in Texture Map Images

A novel method for 3D head tracking in the presence of large head rotations and facial expression changes is described. Tracking is formulated in terms of color image registration in the texture map of a 3D surface model. Model appearance is recursively updated via image mosaicking in the texture map as the head orientation varies. The resulting dynamic texture map provides a stabilized view of the face that can be used as input to many existing 2D techniques for face recognition, facial expressions analysis, lip reading, and eye tracking. Parameters are estimated via a robust minimization procedure; this provides robustness to occlusions, wrinkles, shadows, and specular highlights. The system was tested on a variety of sequences taken with low quality, uncalibrated video cameras. Experimental results are reported

Photometric stereo for strong specular highlights

Photometric stereo (PS) is a fundamental technique in computer vision known to produce 3-D shape with high accuracy. The setting of PS is defined by using several input images of a static scene taken from one and the same camera position but under varying illumination. The vast majority of studies in this 3-D reconstruction method assume orthographic projection for the camera model. In addition, they mainly consider the Lambertian reflectance model as the way that light scatters at surfaces. So, providing reliable PS results from real world objects still remains a challenging task. We address 3-D reconstruction by PS using a more realistic set of assumptions combining for the first time the complete Blinn-Phong reflectance model and perspective projection. To this end, we will compare two different methods of incorporating the perspective projection into our model. Experiments are performed on both synthetic and real world images. Note that our real-world experiments do not benefit from laboratory conditions. The results show the high potential of our method even for complex real world applications such as medical endoscopy images which may include high amounts of specular highlights

The Use of Separated Reflection Components in Estimating Geometrical Parameters of Curved Surface Elements

Iterative least-squares estimation, requires accurate reflectance models to retrieve geometrical parameters of curved surface elements from an image projection. We investigate the use of separating the diffuse (body) reflection from the specular (surface) reflection being responsible for image highlights. Experiments show that the (smooth) diffuse component yields the best convergence properties, while the (sharp) specular component can contribute to the improvement of the noise insensitivit

Joint Material and Illumination Estimation from Photo Sets in the Wild

Faithful manipulation of shape, material, and illumination in 2D Internet images would greatly benefit from a reliable factorization of appearance into material (i.e., diffuse and specular) and illumination (i.e., environment maps). On the one hand, current methods that produce very high fidelity results, typically require controlled settings, expensive devices, or significant manual effort. To the other hand, methods that are automatic and work on 'in the wild' Internet images, often extract only low-frequency lighting or diffuse materials. In this work, we propose to make use of a set of photographs in order to jointly estimate the non-diffuse materials and sharp lighting in an uncontrolled setting. Our key observation is that seeing multiple instances of the same material under different illumination (i.e., environment), and different materials under the same illumination provide valuable constraints that can be exploited to yield a high-quality solution (i.e., specular materials and environment illumination) for all the observed materials and environments. Similar constraints also arise when observing multiple materials in a single environment, or a single material across multiple environments. The core of this approach is an optimization procedure that uses two neural networks that are trained on synthetic images to predict good gradients in parametric space given observation of reflected light. We evaluate our method on a range of synthetic and real examples to generate high-quality estimates, qualitatively compare our results against state-of-the-art alternatives via a user study, and demonstrate photo-consistent image manipulation that is otherwise very challenging to achieve