A rigorous and realistic Shape From Shading method and some of its applications

This article proposes a rigorous and realistic solution of the Lambertian Shape From Shading (SFS) problem. The power of our approach is threefolds. First, our work is based on a rigorous mathematical method: we define a new notion of weak solutions (in the viscosity sense) which does not necessarily requires boundary data (contrary to the work of [rouy-tourin:92,prados-faugeras-etal:02,prados-faugeras:03,camilli-falcone:96,falcone-sagona-etal:01]) and which allows to define a solution as soon as the image is (Lipschitz) continuous (contrary to the work of [oliensis:91,dupuis-oliensis:94]). We prove the existence and uniqueness of this (new) solution and we approximate it by using a provably convergent algorithm. Second, it improves the applicability of the SFS to real images: we complete the realistic work of [prados-faugeras:03,tankus-sochen-etal:03], by modeling the problem with a pinhole camera and with a single point light source located at the optical center. This new modelization appears very relevant for applications. Moreover, our algorithm can deal with images containing discontinuities and black shadows. It is very robust to pixel noise and to errors on parameters. It is also generic: i.e. we propose a unique algorithm which can compute numerical solutions of the various perspective and orthographic SFS models. Finally, our algorithm seems to be the most efficient iterative algorithm of the SFS literature. Third, we propose three applications (in three different areas) based on our SFS method

Equivalence of oblique and frontal illumination in perspective shape from shading

In this paper, it is shown that any oblique illumination shape-from-shading problem under perspective projection for Lambertian reflection and a single distant light source can be converted to an equivalent frontal illumination problem by a simple nonlinear intensity transformation which is equivalent to a rectification in stereo vision. Remarkably, it involves no approximation of depth. The method is evaluated on perspective shape-from-shading involving wide range of oblique angles. © 2007 IEEE.published_or_final_versio

A unifying and rigorous Shape From Shading method adapted to realistic data and applications

International audienceWe propose a new method for the Lambertian Shape From Shading (SFS) problem based on the notion of Crandall-Lions viscosity solution. This method has the advantage of requiring the knowledge of the solution (the surface to be reconstructed) only on some part of the boundary and/or of the singular set (the set of the points at maximal intensity). Moreover it unifies in an unique mathematical formulation the works of Rouy and Tourin, Falcone et al., Prados and Faugeras, based on the notion of viscosity solutions and the work of Dupuis and Oliensis dealing with classical solutions and value functions. Also, it allows to generalize their results to the "perspective SFS" problem

Shape From Shading

Shape From Shading is the process of computing the threedimensional shape of a surface from one image of that surface. Contrary to most of the other three-dimensional reconstruction problems (for example, stereo and photometric stereo), in the Shape From Shading problem, data are minimal (we use a single image!). As a consequence, this inverse problem is intrinsically a difficult one. In this chapter we describe the main difficulties of the problem and the most recent theoretical results. We also give some examples of realistic modelings and of rigorous numerical methods

Fast Marching Method for Generic Shape from Shading

International audienceWe develop a fast numerical method to approximate the solutions of a wide class of equations associated to the Shape From Shading problem. Our method, which is based on the control theory and the interfaces propagation, is an extension of the ?Fast Marching Method? (FMM) [30,25]. In particular our method extends the FMM to some equations for which the solution is not systematically decreasing along the optimal trajectories. We apply with success our one-pass method to the Shape From Shading equations which are involved by the most relevant and recent modelings [22,21] and which cannot be handled by the most recent extensions of the FMM [26,8]

Accelerated volumetric reconstruction from uncalibrated camera views

While both work with images, computer graphics and computer vision are inverse problems. Computer graphics starts traditionally with input geometric models and produces image sequences. Computer vision starts with input image sequences and produces geometric models. In the last few years, there has been a convergence of research to bridge the gap between the two fields. This convergence has produced a new field called Image-based Rendering and Modeling (IBMR). IBMR represents the effort of using the geometric information recovered from real images to generate new images with the hope that the synthesized ones appear photorealistic, as well as reducing the time spent on model creation. In this dissertation, the capturing, geometric and photometric aspects of an IBMR system are studied. A versatile framework was developed that enables the reconstruction of scenes from images acquired with a handheld digital camera. The proposed system targets applications in areas such as Computer Gaming and Virtual Reality, from a lowcost perspective. In the spirit of IBMR, the human operator is allowed to provide the high-level information, while underlying algorithms are used to perform low-level computational work. Conforming to the latest architecture trends, we propose a streaming voxel carving method, allowing a fast GPU-based processing on commodity hardware

Statistical Approaches to Inferring Object Shape from Single Images

Depth inference is a fundamental problem of computer vision with a broad range of potential applications. Monocular depth inference techniques, particularly shape from shading dates back to as early as the 40's when it was first used to study the shape of the lunar surface. Since then there has been ample research to develop depth inference algorithms using monocular cues. Most of these are based on physical models of image formation and rely on a number of simplifying assumptions that do not hold for real world and natural imagery. Very few make use of the rich statistical information contained in real world images and their 3D information. There have been a few notable exceptions though. The study of statistics of natural scenes has been concentrated on outdoor scenes which are cluttered. Statistics of scenes of single objects has been less studied, but is an essential part of daily human interaction with the environment. Inferring shape of single objects is a very important computer vision problem which has captured the interest of many researchers over the past few decades and has applications in object recognition, robotic grasping, fault detection and Content Based Image Retrieval (CBIR). This thesis focuses on studying the statistical properties of single objects and their range images which can benefit shape inference techniques. I acquired two databases: Single Object Range and HDR (SORH) and the Eton Myers Database of single objects, including laser-acquired depth, binocular stereo, photometric stereo and High Dynamic Range (HDR) photography. I took a data driven approach and studied the statistics of color and range images of real scenes of single objects along with whole 3D objects and uncovered some interesting trends in the data. The fractal structure of natural images was previously well known, and thought to be a universal property. However, my research showed that the fractal structure of single objects and surfaces is governed by a wholly different set of rules. Classical computer vision problems of binocular and multi-view stereo, photometric stereo, shape from shading, structure from motion, and others, all rely on accurate and complete models of which 3D shapes and textures are plausible in nature, to avoid producing unlikely outputs. Bayesian approaches are common for these problems, and hopefully the findings on the statistics of the shape of single objects from this work and others will both inform new and more accurate Bayesian priors on shape, and also enable more efficient probabilistic inference procedures

Perspective shape from shading for Phong-type non-Lambertian surfaces

The shape-from-shading (SfS) problem in computer vision is to compute at hand of the shading variation in a given 2-D image the 3-D structure of depicted objects. We introduce an efficient numerical method for a new perspective SfS model for general non-Lambertian surfaces. First, the modelling process is given in detail. The model is based on the perspective model for Lambertian surfaces recently studied by Prados et al., which we extend by use of the Phong reflection model incorporating ambient, diffuse and specular components. The arising partial differential equation (PDE) is a non-linear time-independent Hamilton-Jacobi equation. In order to compute the sought viscosity supersolution of the PDE, we introduce an artificial time into the equation and solve for the steady state. Based on a multi-scale analysis of the PDE, we construct a fully explicit numerical method and elaborate on its stability. In order to achieve fast convergence of the resulting iterative scheme, a coarse-to-fine strategy combined with a sweeping technique is employed. Numerical experiments show the benefits of our approach: While computational times stay reasonable even for quite large images, a substantial qualitative gain can be achieved by use of the new model. Moreover, the computational technique is relatively easy to implement compared to other approaches in the field

A PDE approach to Shape from Shading via Photometric Stereo

We present a new analytic and numerical approach to the shape from shading using photometric stereo technique. That is, we solve the problem to find the 3D surface of an object starting from its several 2D pictures taken from the same point of view, but changing, for every image, the direction of the light source