Unifying diffuse and specular reflections for the photometric stereo problem

This is the author accepted manuscript. The final version is available from IEEE via http://dx.doi.org/10.1109/WACV.2016.7477643After thirty years of researching, the photometric stereo technique for 3D shape recovery still does not provide reliable results if it is not constrained into very well-controlled scenarios. In fact, dealing with realistic materials and lightings yields a non-linear bidirectional reflectance distribution function which is primarily difficult to parametrize and then arduous to solve. With the aim to let the photometric stereo approach face more realistic assumptions, in this work we firstly introduce a unified irradiance equation describing both diffuse and specular reflection components in a general lighting setting. After that, we define a new equation we call unifying due to its basic features modeling the photometric stereo problem for heterogeneous materials. It is provided by making the ratio of irradiance equations holding both diffuse and specular reflections as well as non-linear light propagation features simultaneously. Performing a wide range of experiments, we show that this new approach overcomes state-of-the-art since it leads to a system of unifying equations which can be solved in a very robust manner using an efficient variational approach.Experimental setups were provided by Toulouse Tech Transfer, and this collaboration was funded by CNRS GdR 2286 (MIA)

Linear Differential Constraints for Photo-polarimetric Height Estimation

In this paper we present a differential approach to photo-polarimetric shape estimation. We propose several alternative differential constraints based on polarisation and photometric shading information and show how to express them in a unified partial differential system. Our method uses the image ratios technique to combine shading and polarisation information in order to directly reconstruct surface height, without first computing surface normal vectors. Moreover, we are able to remove the non-linearities so that the problem reduces to solving a linear differential problem. We also introduce a new method for estimating a polarisation image from multichannel data and, finally, we show it is possible to estimate the illumination directions in a two source setup, extending the method into an uncalibrated scenario. From a numerical point of view, we use a least-squares formulation of the discrete version of the problem. To the best of our knowledge, this is the first work to consider a unified differential approach to solve photo-polarimetric shape estimation directly for height. Numerical results on synthetic and real-world data confirm the effectiveness of our proposed method.Comment: To appear at International Conference on Computer Vision (ICCV), Venice, Italy, October 22-29, 201

Analysis of surface parametrizations for modern photometric stereo modeling

Tridimensional shape recovery based on Photometric Stereo (PS) recently received a strong improvement due to new mathematical models based on partial differential irradiance equation ratios. This modern approach to PS faces more realistic physical effects among which light attenuation and radial light propagation from a point light source. Since the approximation of the surface is performed with single step method, accurate reconstruction is prevented by sensitiveness to noise. In this paper we analyse a well-known parametrization of the tridimensional surface extending it on any auxiliary convex projection functions. Experiments on synthetic data show preliminary results where more accurate reconstruction can be achieved using more suitable parametrization specially in case of noisy input images

Single-image RGB Photometric Stereo With Spatially-varying Albedo

We present a single-shot system to recover surface geometry of objects with spatially-varying albedos, from images captured under a calibrated RGB photometric stereo setup---with three light directions multiplexed across different color channels in the observed RGB image. Since the problem is ill-posed point-wise, we assume that the albedo map can be modeled as piece-wise constant with a restricted number of distinct albedo values. We show that under ideal conditions, the shape of a non-degenerate local constant albedo surface patch can theoretically be recovered exactly. Moreover, we present a practical and efficient algorithm that uses this model to robustly recover shape from real images. Our method first reasons about shape locally in a dense set of patches in the observed image, producing shape distributions for every patch. These local distributions are then combined to produce a single consistent surface normal map. We demonstrate the efficacy of the approach through experiments on both synthetic renderings as well as real captured images.Comment: 3DV 2016. Project page at http://www.ttic.edu/chakrabarti/rgbps