Shape from Shading: a well-posed problem?

International audienceShape From Shading is known to be an ill-posed problem. Contrary to the previous work, we show here that if we model the problem in a more realistic way than it is usually done (we take into account the 1/r2 attenuation term of the lighting), Shape From Shading can be completely well-posed. Thus the shading allows to recover (almost) any surface from only one image (of this surface), without any additional data (in particular, without regularity assumptions and without the knowledge of the heights of the solution at the local "minima". More precisely, in this report we formulate the problem as that of solving a new PDE, we develop a complete mathematical study of this equation (existence and uniqueness of the solution) and we design a new provably convergent numerical method. Finally, we test our new SFS method on various synthetic images and on our database of real images of faces, with success

Three-dimensional modeling of the human jaw/teeth using optics and statistics.

Object modeling is a fundamental problem in engineering, involving talents from computer-aided design, computational geometry, computer vision and advanced manufacturing. The process of object modeling takes three stages: sensing, representation, and analysis. Various sensors may be used to capture information about objects; optical cameras and laser scanners are common with rigid objects, while X-ray, CT and MRI are common with biological organs. These sensors may provide a direct or an indirect inference about the object, requiring a geometric representation in the computer that is suitable for subsequent usage. Geometric representations that are compact, i.e., capture the main features of the objects with a minimal number of data points or vertices, fall into the domain of computational geometry. Once a compact object representation is in the computer, various analysis steps can be conducted, including recognition, coding, transmission, etc. The subject matter of this dissertation is object reconstruction from a sequence of optical images using shape from shading (SFS) and SFS with shape priors. The application domain is dentistry. Most of the SFS approaches focus on the computational part of the SFS problem, i.e. the numerical solution. As a result, the imaging model in most conventional SFS algorithms has been simplified under three simple, but restrictive assumptions: (1) the camera performs an orthographic projection of the scene, (2) the surface has a Lambertian reflectance and (3) the light source is a single point source at infinity. Unfortunately, such assumptions are no longer held in the case of reconstruction of real objects as intra-oral imaging environment for human teeth. In this work, we introduce a more realistic formulation of the SFS problem by considering the image formation components: the camera, the light source, and the surface reflectance. This dissertation proposes a non-Lambertian SFS algorithm under perspective projection which benefits from camera calibration parameters. The attenuation of illumination is taken account due to near-field imaging. The surface reflectance is modeled using the Oren-Nayar-Wolff model which accounts for the retro-reflection case. In this context, a new variational formulation is proposed that relates an evolving surface model with image information, taking into consideration that the image is taken by a perspective camera with known parameters. A new energy functional is formulated to incorporate brightness, smoothness and integrability constraints. In addition, to further improve the accuracy and practicality of the results, 3D shape priors are incorporated in the proposed SFS formulation. This strategy is motivated by the fact that humans rely on strong prior information about the 3D world around us in order to perceive 3D shape information. Such information is statistically extracted from training 3D models of the human teeth. The proposed SFS algorithms have been used in two different frameworks in this dissertation: a) holistic, which stitches a sequence of images in order to cover the entire jaw, and then apply the SFS, and b) piece-wise, which focuses on a specific tooth or a segment of the human jaw, and applies SFS using physical teeth illumination characteristics. To augment the visible portion, and in order to have the entire jaw reconstructed without the use of CT or MRI or even X-rays, prior information were added which gathered from a database of human jaws. This database has been constructed from an adult population with variations in teeth size, degradation and alignments. The database contains both shape and albedo information for the population. Using this database, a novel statistical shape from shading (SSFS) approach has been created. Extending the work on human teeth analysis, Finite Element Analysis (FEA) is adapted for analyzing and calculating stresses and strains of dental structures. Previous Finite Element (FE) studies used approximate 2D models. In this dissertation, an accurate three-dimensional CAD model is proposed. 3D stress and displacements of different teeth type are successfully carried out. A newly developed open-source finite element solver, Finite Elements for Biomechanics (FEBio), has been used. The limitations of the experimental and analytical approaches used for stress and displacement analysis are overcome by using FEA tool benefits such as dealing with complex geometry and complex loading conditions

A unified framework for document image restoration

Ph.DDOCTOR OF PHILOSOPH

Depth Recovery of Complex Surfaces from Texture-less Pairs of Stereo Images

In this paper, a novel framework is presented to recover the 3D shape information of a complex surface using its texture-less stereo images. First a linear and generalized Lambertian model is proposed to obtain the depth information by shape from shading (SfS) using an image from stereo pair. Then this depth data is corrected by integrating scale invariant features (SIFT) indexes. These SIFT indexes are defined by means of disparity between the matching invariant features in rectified stereo images. The integration process is based on correcting the 3D visible surfaces obtained from SfS using these SIFT indexes. The SIFT indexes based improvement of depth values which are obtained from generalized Lambertian reflectance model is performed by a feed-forward neural network. The experiments are performed to demonstrate the usability and accuracy of the proposed framework

Perspective shape from shading for Phong-type non-Lambertian surfaces

The shape-from-shading (SfS) problem in computer vision is to compute at hand of the shading variation in a given 2-D image the 3-D structure of depicted objects. We introduce an efficient numerical method for a new perspective SfS model for general non-Lambertian surfaces. First, the modelling process is given in detail. The model is based on the perspective model for Lambertian surfaces recently studied by Prados et al., which we extend by use of the Phong reflection model incorporating ambient, diffuse and specular components. The arising partial differential equation (PDE) is a non-linear time-independent Hamilton-Jacobi equation. In order to compute the sought viscosity supersolution of the PDE, we introduce an artificial time into the equation and solve for the steady state. Based on a multi-scale analysis of the PDE, we construct a fully explicit numerical method and elaborate on its stability. In order to achieve fast convergence of the resulting iterative scheme, a coarse-to-fine strategy combined with a sweeping technique is employed. Numerical experiments show the benefits of our approach: While computational times stay reasonable even for quite large images, a substantial qualitative gain can be achieved by use of the new model. Moreover, the computational technique is relatively easy to implement compared to other approaches in the field

A unifying and rigorous Shape From Shading method adapted to realistic data and applications

International audienceWe propose a new method for the Lambertian Shape From Shading (SFS) problem based on the notion of Crandall-Lions viscosity solution. This method has the advantage of requiring the knowledge of the solution (the surface to be reconstructed) only on some part of the boundary and/or of the singular set (the set of the points at maximal intensity). Moreover it unifies in an unique mathematical formulation the works of Rouy and Tourin, Falcone et al., Prados and Faugeras, based on the notion of viscosity solutions and the work of Dupuis and Oliensis dealing with classical solutions and value functions. Also, it allows to generalize their results to the "perspective SFS" problem

Shape From Shading

Shape From Shading is the process of computing the threedimensional shape of a surface from one image of that surface. Contrary to most of the other three-dimensional reconstruction problems (for example, stereo and photometric stereo), in the Shape From Shading problem, data are minimal (we use a single image!). As a consequence, this inverse problem is intrinsically a difficult one. In this chapter we describe the main difficulties of the problem and the most recent theoretical results. We also give some examples of realistic modelings and of rigorous numerical methods

3D Reconstruction using Active Illumination

In this thesis we present a pipeline for 3D model acquisition. Generating 3D models of real-world objects is an important task in computer vision with many applications, such as in 3D design, archaeology, entertainment, and virtual or augmented reality. The contribution of this thesis is threefold: we propose a calibration procedure for the cameras, we describe an approach for capturing and processing photometric normals using gradient illuminations in the hardware set-up, and finally we present a multi-view photometric stereo 3D reconstruction method. In order to obtain accurate results using multi-view and photometric stereo reconstruction, the cameras are calibrated geometrically and photometrically. For acquiring data, a light stage is used. This is a hardware set-up that allows to control the illumination during acquisition. The procedure used to generate appropriate illuminations and to process the acquired data to obtain accurate photometric normals is described. The core of the pipeline is a multi-view photometric stereo reconstruction method. In this method, we first generate a sparse reconstruction using the acquired images and computed normals. In the second step, the information from the normal maps is used to obtain a dense reconstruction of an object’s surface. Finally, the reconstructed surface is filtered to remove artifacts introduced by the dense reconstruction step