Perspective shape from shading and viscosity solutions

International audienceThis article proposes a solution of the Lambertian shape from shading (SFS) problem in the case of a pinhole camera model (performing a perspective projection). Our approach is based upon the notion of viscosity solutions of Hamilton-Jacobi equations. This approach allows us to naturally deal with nonsmooth solutions and provides a mathematical framework for proving correctness of our algorithms. Our work extends previous work in the area in three aspects. First, it models the camera as a pinhole whereas most authors assume an orthographic projection, thereby extending the applicability of shape from shading methods to more realistic images. Second, by adapting the brightness equation to the perspective problem, we obtain a new partial differential equation (PDE). Results about the existence and uniqueness of its solution are also obtained. Third, it allows us to come up with a new approximation scheme and a new algorithm for computing numerical approximations of the ?continuous? solution as well as a proof of their convergence toward that solution

A unifying and rigorous Shape From Shading method adapted to realistic data and applications

International audienceWe propose a new method for the Lambertian Shape From Shading (SFS) problem based on the notion of Crandall-Lions viscosity solution. This method has the advantage of requiring the knowledge of the solution (the surface to be reconstructed) only on some part of the boundary and/or of the singular set (the set of the points at maximal intensity). Moreover it unifies in an unique mathematical formulation the works of Rouy and Tourin, Falcone et al., Prados and Faugeras, based on the notion of viscosity solutions and the work of Dupuis and Oliensis dealing with classical solutions and value functions. Also, it allows to generalize their results to the "perspective SFS" problem

Analysis and approximation of some Shape-from-Shading models for non-Lambertian surfaces

The reconstruction of a 3D object or a scene is a classical inverse problem in Computer Vision. In the case of a single image this is called the Shape-from-Shading (SfS) problem and it is known to be ill-posed even in a simplified version like the vertical light source case. A huge number of works deals with the orthographic SfS problem based on the Lambertian reflectance model, the most common and simplest model which leads to an eikonal type equation when the light source is on the vertical axis. In this paper we want to study non-Lambertian models since they are more realistic and suitable whenever one has to deal with different kind of surfaces, rough or specular. We will present a unified mathematical formulation of some popular orthographic non-Lambertian models, considering vertical and oblique light directions as well as different viewer positions. These models lead to more complex stationary nonlinear partial differential equations of Hamilton-Jacobi type which can be regarded as the generalization of the classical eikonal equation corresponding to the Lambertian case. However, all the equations corresponding to the models considered here (Oren-Nayar and Phong) have a similar structure so we can look for weak solutions to this class in the viscosity solution framework. Via this unified approach, we are able to develop a semi-Lagrangian approximation scheme for the Oren-Nayar and the Phong model and to prove a general convergence result. Numerical simulations on synthetic and real images will illustrate the effectiveness of this approach and the main features of the scheme, also comparing the results with previous results in the literature.Comment: Accepted version to Journal of Mathematical Imaging and Vision, 57 page

A viscosity method for Shape-from-Shading without boundary data

This report proposes a solution of the Lambertian Shape From Shading (SFS) problem by designing a new mathematical framework based on the notion of viscosity solutions. The power of our approach is twofolds: 1) it defines a notion of weak solutions (in the viscosity sense) which does not necessarily require boundary data. Note that, in the previous SFS work of Rouy et al., Falcone et al. , Prados et al., the characterization of a viscosity solution and its computation require the knowledge of its values on the boundary of the image. This was quite unrealistic because in practice such values are not known. 2) it unifies the work of Rouy et al., Falcone et al., Prados et al., based on the notion of viscosity solutions and the work of Dupuis and Oliensis dealing with classical ($C^1$) solutions and value functions. Also, we generalize their work to the ``perspective SFS'' problem recently introduced by Prados and Faugeras. The notion of viscosity solutions described in this paper is obtained by slightly modifying the notion of singular viscosity solutions developped by Camilli and Siconolfi. We demonstrate the existence and the uniqueness of the solution for a class of Hamilton-Jacobi equations H(x,\nablau)=0, in a bounded open domain. Some stability results are proved. Moreover, we show that this framework allows to characterize the classical discontinuous viscosity solutions by their ``minimums''. In this report, we also propose some algorithms which provide numerical approximations of these new solutions. These provably convergent algorithms are quite robust and do not necessarily require boundary data. Finally, we have successfully applied our SFS method to real images and we have suggested a number of real-life applications

Object recognition using shape-from-shading

This paper investigates whether surface topography information extracted from intensity images using a recently reported shape-from-shading (SFS) algorithm can be used for the purposes of 3D object recognition. We consider how curvature and shape-index information delivered by this algorithm can be used to recognize objects based on their surface topography. We explore two contrasting object recognition strategies. The first of these is based on a low-level attribute summary and uses histograms of curvature and orientation measurements. The second approach is based on the structural arrangement of constant shape-index maximal patches and their associated region attributes. We show that region curvedness and a string ordering of the regions according to size provides recognition accuracy of about 96 percent. By polling various recognition schemes. including a graph matching method. we show that a recognition rate of 98-99 percent is achievable

A rigorous and realistic Shape From Shading method and some of its applications

This article proposes a rigorous and realistic solution of the Lambertian Shape From Shading (SFS) problem. The power of our approach is threefolds. First, our work is based on a rigorous mathematical method: we define a new notion of weak solutions (in the viscosity sense) which does not necessarily requires boundary data (contrary to the work of [rouy-tourin:92,prados-faugeras-etal:02,prados-faugeras:03,camilli-falcone:96,falcone-sagona-etal:01]) and which allows to define a solution as soon as the image is (Lipschitz) continuous (contrary to the work of [oliensis:91,dupuis-oliensis:94]). We prove the existence and uniqueness of this (new) solution and we approximate it by using a provably convergent algorithm. Second, it improves the applicability of the SFS to real images: we complete the realistic work of [prados-faugeras:03,tankus-sochen-etal:03], by modeling the problem with a pinhole camera and with a single point light source located at the optical center. This new modelization appears very relevant for applications. Moreover, our algorithm can deal with images containing discontinuities and black shadows. It is very robust to pixel noise and to errors on parameters. It is also generic: i.e. we propose a unique algorithm which can compute numerical solutions of the various perspective and orthographic SFS models. Finally, our algorithm seems to be the most efficient iterative algorithm of the SFS literature. Third, we propose three applications (in three different areas) based on our SFS method

Shape-from-shading using the heat equation

This paper offers two new directions to shape-from-shading, namely the use of the heat equation to smooth the field of surface normals and the recovery of surface height using a low-dimensional embedding. Turning our attention to the first of these contributions, we pose the problem of surface normal recovery as that of solving the steady state heat equation subject to the hard constraint that Lambert's law is satisfied. We perform our analysis on a plane perpendicular to the light source direction, where the z component of the surface normal is equal to the normalized image brightness. The x - y or azimuthal component of the surface normal is found by computing the gradient of a scalar field that evolves with time subject to the heat equation. We solve the heat equation for the scalar potential and, hence, recover the azimuthal component of the surface normal from the average image brightness, making use of a simple finite difference method. The second contribution is to pose the problem of recovering the surface height function as that of embedding the field of surface normals on a manifold so as to preserve the pattern of surface height differences and the lattice footprint of the surface normals. We experiment with the resulting method on a variety of real-world image data, where it produces qualitatively good reconstructed surfaces