130 research outputs found
Unifying diffuse and specular reflections for the photometric stereo problem
This is the author accepted manuscript. The final version is available from IEEE via http://dx.doi.org/10.1109/WACV.2016.7477643After thirty years of researching, the photometric stereo technique for 3D shape recovery still does not provide reliable results if it is not constrained into very well-controlled scenarios. In fact, dealing with realistic materials and lightings yields a non-linear bidirectional reflectance distribution function which is primarily difficult to parametrize and then arduous to solve. With the aim to let the photometric stereo approach face more realistic assumptions, in this work we firstly introduce a unified irradiance equation describing both diffuse and specular reflection components in a general lighting setting. After that, we define a new equation we call unifying due to its basic features modeling the photometric stereo problem for heterogeneous materials. It is provided by making the ratio of irradiance equations holding both diffuse and specular reflections as well as non-linear light propagation features simultaneously. Performing a wide range of experiments, we show that this new approach overcomes state-of-the-art since it leads to a system of unifying equations which can be solved in a very robust manner using an efficient variational approach.Experimental setups were provided by Toulouse Tech Transfer, and this collaboration was funded by CNRS GdR 2286 (MIA)
Linear Differential Constraints for Photo-polarimetric Height Estimation
In this paper we present a differential approach to photo-polarimetric shape
estimation. We propose several alternative differential constraints based on
polarisation and photometric shading information and show how to express them
in a unified partial differential system. Our method uses the image ratios
technique to combine shading and polarisation information in order to directly
reconstruct surface height, without first computing surface normal vectors.
Moreover, we are able to remove the non-linearities so that the problem reduces
to solving a linear differential problem. We also introduce a new method for
estimating a polarisation image from multichannel data and, finally, we show it
is possible to estimate the illumination directions in a two source setup,
extending the method into an uncalibrated scenario. From a numerical point of
view, we use a least-squares formulation of the discrete version of the
problem. To the best of our knowledge, this is the first work to consider a
unified differential approach to solve photo-polarimetric shape estimation
directly for height. Numerical results on synthetic and real-world data confirm
the effectiveness of our proposed method.Comment: To appear at International Conference on Computer Vision (ICCV),
Venice, Italy, October 22-29, 201
A single-lobe photometric stereo approach for heterogeneous material
Shape from shading with multiple light sources is an active research area, and a diverse range of approaches have been proposed in recent decades. However, devising a robust reconstruction technique still remains a challenging goal, as the image acquisition process is highly nonlinear. Recent Photometric Stereo variants rely on simplifying assumptions in order to make the problem solvable: light propagation is still commonly assumed to be uniform, and the Bidirectional Reflectance Distribution Function is assumed to be diffuse, with limited interest for specular materials. In this work, we introduce a well-posed formulation based on partial differential equations (PDEs) for a unified reflectance function that can model both diffuse and specular reflections. We base our derivation on ratio of images, which makes the model independent from photometric invariants and yields a well-posed differential problem based on a system of quasi-linear PDEs with discontinuous coefficients. In addition, we directly solve a differential problem for the unknown depth, thus avoiding the intermediate step of approximating the normal field. A variational approach is presented ensuring robustness to noise and outliers (such as black shadows), and this is confirmed with a wide range of experiments on both synthetic and real data, where we compare favorably to the state of the art.Roberto Mecca is a Marie Curie fellow of the “Istituto Nazionale di Alta Matematica” (Italy) for a project shared with University of Cambridge, Department of Engineering and the Department of Mathematics, University of Bologna
Photometric stereo for strong specular highlights
Photometric stereo (PS) is a fundamental technique in computer vision known
to produce 3-D shape with high accuracy. The setting of PS is defined by using
several input images of a static scene taken from one and the same camera
position but under varying illumination. The vast majority of studies in this
3-D reconstruction method assume orthographic projection for the camera model.
In addition, they mainly consider the Lambertian reflectance model as the way
that light scatters at surfaces. So, providing reliable PS results from real
world objects still remains a challenging task. We address 3-D reconstruction
by PS using a more realistic set of assumptions combining for the first time
the complete Blinn-Phong reflectance model and perspective projection. To this
end, we will compare two different methods of incorporating the perspective
projection into our model. Experiments are performed on both synthetic and real
world images. Note that our real-world experiments do not benefit from
laboratory conditions. The results show the high potential of our method even
for complex real world applications such as medical endoscopy images which may
include high amounts of specular highlights
Linear Differential Constraints for Photo-polarimetric Height Estimation
In this paper we present a differential approach to photopolarimetric shape estimation. We propose several alternative differential constraints based on polarisation and photometric shading information and show how to express them in a unified partial differential system. Our method uses the image ratios technique to combine shading and polarisation information in order to directly reconstruct surface height, without first computing surface normal vectors. Moreover, we are able to remove the non-linearities so that the problem reduces to solving a linear differential problem. We also introduce a new method for estimating a polarisation image from multichannel data and, finally, we show it is possible to estimate the illumination directions in a two source setup, extending the method into an uncalibrated scenario. From a numerical point of view, we use a least-squares formulation of the discrete version of the problem. To the best of our knowledge, this is the first work to consider a unified differential approach to solve photo-polarimetric shape estimation directly for height. Numerical results on synthetic and real-world data confirm the effectiveness of our proposed method
Polarized 3D: High-Quality Depth Sensing with Polarization Cues
Coarse depth maps can be enhanced by using the shape information from polarization cues. We propose a framework to combine surface normals from polarization (hereafter polarization normals) with an aligned depth map. Polarization normals have not been used for depth enhancement before. This is because polarization normals suffer from physics-based artifacts, such as azimuthal ambiguity, refractive distortion and fronto-parallel signal degradation. We propose a framework to overcome these key challenges, allowing the benefits of polarization to be used to enhance depth maps. Our results demonstrate improvement with respect to state-of-the-art 3D reconstruction techniques.Charles Stark Draper Laboratory (Doctoral Fellowship)Singapore. Ministry of Education (Academic Research Foundation MOE2013-T2-1-159)Singapore. National Research Foundation (Singapore University of Technology and Design
A CNN Based Approach for the Point-Light Photometric Stereo Problem
Reconstructing the 3D shape of an object using several images under different
light sources is a very challenging task, especially when realistic assumptions
such as light propagation and attenuation, perspective viewing geometry and
specular light reflection are considered. Many of works tackling Photometric
Stereo (PS) problems often relax most of the aforementioned assumptions.
Especially they ignore specular reflection and global illumination effects. In
this work, we propose a CNN-based approach capable of handling these realistic
assumptions by leveraging recent improvements of deep neural networks for
far-field Photometric Stereo and adapt them to the point light setup. We
achieve this by employing an iterative procedure of point-light PS for shape
estimation which has two main steps. Firstly we train a per-pixel CNN to
predict surface normals from reflectance samples. Secondly, we compute the
depth by integrating the normal field in order to iteratively estimate light
directions and attenuation which is used to compensate the input images to
compute reflectance samples for the next iteration.
Our approach sigificantly outperforms the state-of-the-art on the DiLiGenT
real world dataset. Furthermore, in order to measure the performance of our
approach for near-field point-light source PS data, we introduce LUCES the
first real-world 'dataset for near-fieLd point light soUrCe photomEtric Stereo'
of 14 objects of different materials were the effects of point light sources
and perspective viewing are a lot more significant. Our approach also
outperforms the competition on this dataset as well. Data and test code are
available at the project page.Comment: arXiv admin note: text overlap with arXiv:2009.0579
- …