Cavlectometry: Towards Holistic Reconstruction of Large Mirror Objects

We introduce a method based on the deflectometry principle for the reconstruction of specular objects exhibiting significant size and geometric complexity. A key feature of our approach is the deployment of an Automatic Virtual Environment (CAVE) as pattern generator. To unfold the full power of this extraordinary experimental setup, an optical encoding scheme is developed which accounts for the distinctive topology of the CAVE. Furthermore, we devise an algorithm for detecting the object of interest in raw deflectometric images. The segmented foreground is used for single-view reconstruction, the background for estimation of the camera pose, necessary for calibrating the sensor system. Experiments suggest a significant gain of coverage in single measurements compared to previous methods. To facilitate research on specular surface reconstruction, we will make our data set publicly available

Specular surface recovery from reflections of a planar pattern undergoing an unknown pure translation

LNCS v. 6493 entitled: Computer Vision – ACCV 2010: 10th Asian Conference on Computer Vision, Queenstown, New Zealand, November 8-12, 2010, Revised Selected Papers, Part 2This paper addresses the problem of specular surface recovery, and proposes a novel solution based on observing the reflections of a translating planar pattern. Previous works have demonstrated that a specular surface can be recovered from the reflections of two calibrated planar patterns. In this paper, however, only one reference planar pattern is assumed to have been calibrated against a fixed camera observing the specular surface. Instead of introducing and calibrating a second pattern, the reference pattern is allowed to undergo an unknown pure translation, and a closed form solution is derived for recovering such a motion. Unlike previous methods which estimate the shape by directly triangulating the visual rays and reflection rays, a novel method based on computing the projections of the visual rays on the translating pattern is introduced. This produces a depth range for each pixel which also provides a measure of the accuracy of the estimation. The proposed approach enables a simple auto-calibration of the translating pattern, and data redundancy resulting from the translating pattern can improve both the robustness and accuracy of the shape estimation. Experimental results on both synthetic and real data are presented to demonstrate the effectiveness of the proposed approach. © 2011 Springer-Verlag Berlin Heidelberg.postprintThe 10th Asian Conference on Computer Vision, Queenstown, New Zealand, 8-12 November 2010. In Lecture Notes in Computer Science, 2010, v. 6493, p. 137-14

Local Analysis for 3D Reconstruction of Specular Surfaces

We explore the geometry linking the shape of a curved mirror surface to the distortions it produces on a scene it reflects. Our analysis is local and differential. We assume a simple calibrated scene composed of lines passing through a point. We demonstrate that local information about the geometry of the surface may be recovered up to the second order from either the orientation and curvature of the images of two intersecting lines, or from the orientation of the images of three or more intersecting lines. An explicit solution for calculating shape and position of spherical mirror surfaces is given

Rapid inference of object rigidity and reflectance using optic flow

Rigidity and reflectance are key object properties, important in their own rights, and they are key properties that stratify motion reconstruction algorithms. However, the inference of rigidity and reflectance are both difficult without additional information about the object's shape, the environment, or lighting. For humans, relative motions of object and observer provides rich information about object shape, rigidity, and reflectivity. We show that it is possible to detect rigid object motion for both specular and diffuse reflective surfaces using only optic flow, and that flow can distinguish specular and diffuse motion for rigid objects. Unlike nonrigid objects, optic flow fields for rigid moving surfaces are constrained by a global transformation, which can be detected using an optic flow matching procedure across time. In addition, using a Procrustes analysis of structure from motion reconstructed 3D points, we show how to classify specular from diffuse surfaces. © 2009 Springer Berlin Heidelberg

Real-World Normal Map Capture for Nearly Flat Reflective Surfaces

Although specular objects have gained interest in recent years, virtually no approaches exist for markerless reconstruction of reflective scenes in the wild. In this work, we present a practical approach to capturing normal maps in real-world scenes using video only. We focus on nearly planar surfaces such as windows, facades from glass or metal, or frames, screens and other indoor objects and show how normal maps of these can be obtained without the use of an artificial calibration object. Rather, we track the reflections of real-world straight lines, while moving with a hand-held or vehicle-mounted camera in front of the object. In contrast to error-prone local edge tracking, we obtain the reflections by a robust, global segmentation technique of an ortho-rectified 3D video cube that also naturally allows efficient user interaction. Then, at each point of the reflective surface, the resulting 2D-curve to 3D-line correspondence provides a novel quadratic constraint on the local surface normal. This allows to globally solve for the shape by integrability and smoothness constraints and easily supports the usage of multiple lines. We demonstrate the technique on several objects and facades

3D pose refinement from reflections

We demonstrate how to exploit reflections for accurate registration of shiny objects: The lighting environment can be retrieved from the reflections under a distant illumination assumption. Since it remains unchanged when the camera or the object of interest moves, this provides powerful additional constraints that can be incorporated into standard pose estimation algorithms

Key characteristics of specular stereo.

Because specular reflection is view-dependent, shiny surfaces behave radically differently from matte, textured surfaces when viewed with two eyes. As a result, specular reflections pose substantial problems for binocular stereopsis. Here we use a combination of computer graphics and geometrical analysis to characterize the key respects in which specular stereo differs from standard stereo, to identify how and why the human visual system fails to reconstruct depths correctly from specular reflections. We describe rendering of stereoscopic images of specular surfaces in which the disparity information can be varied parametrically and independently of monocular appearance. Using the generated surfaces and images, we explain how stereo correspondence can be established with known and unknown surface geometry. We show that even with known geometry, stereo matching for specular surfaces is nontrivial because points in one eye may have zero, one, or multiple matches in the other eye. Matching features typically yield skew (nonintersecting) rays, leading to substantial ortho-epipolar components to the disparities, which makes deriving depth values from matches nontrivial. We suggest that the human visual system may base its depth estimates solely on the epipolar components of disparities while treating the ortho-epipolar components as a measure of the underlying reliability of the disparity signals. Reconstructing virtual surfaces according to these principles reveals that they are piece-wise smooth with very large discontinuities close to inflection points on the physical surface. Together, these distinctive characteristics lead to cues that the visual system could use to diagnose specular reflections from binocular information.The work was funded by the Wellcome Trust (grants 08459/Z/07/Z & 095183/Z/10/Z) and the EU Marie Curie Initial Training Network “PRISM” (FP7-PEOPLE-2012-ITN, Agreement: 316746).This is the author accepted manuscript. The final version is available from ARVO via http://dx.doi.org/10.1167/14.14.1