Shape from X: Psychophysics and Computation

This chapter contains sections titled: The Many Routes to Shape, The Need for Integration, Shape From Stereo and Shading (Local Measurements) 1 , Shape from Shading and Texture (Global Measurements), Shape from Disparate Shading (Intensity-Based Stereo), Shape from Highlights 2 , Integration of Depth Modules, A Bayesian Framework for Cue Integration 3 , Final Remarks, Acknowledgments, Appendices, Reference

Recovering facial shape using a statistical model of surface normal direction

In this paper, we show how a statistical model of facial shape can be embedded within a shape-from-shading algorithm. We describe how facial shape can be captured using a statistical model of variations in surface normal direction. To construct this model, we make use of the azimuthal equidistant projection to map the distribution of surface normals from the polar representation on a unit sphere to Cartesian points on a local tangent plane. The distribution of surface normal directions is captured using the covariance matrix for the projected point positions. The eigenvectors of the covariance matrix define the modes of shape-variation in the fields of transformed surface normals. We show how this model can be trained using surface normal data acquired from range images and how to fit the model to intensity images of faces using constraints on the surface normal direction provided by Lambert's law. We demonstrate that the combination of a global statistical constraint and local irradiance constraint yields an efficient and accurate approach to facial shape recovery and is capable of recovering fine local surface details. We assess the accuracy of the technique on a variety of images with ground truth and real-world images

Shape from Shading: a well-posed problem?

Shape From Shading is known to be an ill-posed problem. Contrary to the previous work, we show here that if we model the problem in a more realistic way than it is usually done (we take into account the $1/r^2$ attenuation term of the lighting), Shape From Shading can be completely well-posed. Thus the shading allows to recover (almost) any surface from only one image (of this surface), without any additional data (in particular, without regularity assumptions and without the knowledge of the heights of the solution at the local "minima". More precisely, in this report we formulate the problem as that of solving a new PDE, we develop a complete mathematical study of this equation (existence and uniqueness of the solution) and we design a new provably convergent numerical method. Finally, we test our new SFS method on various synthetic images and on our database of real images of faces, with success

The effects of illumination direction on the perception of 3D shape from shading

3rd Place at the Denman Undergraduate Research Forum 2016A fundamental problem for the perception of 3D shape from shading is to achieve some level of constancy over variations in the pattern of illumination. The present research was designed to investigate how changes in the direction of illumination influence the apparent shapes of surfaces. The stimuli included images of 3D surfaces with Lambertian reflectance functions that were illuminated by a rectangular area light source. The direction of illumination was systematically manipulated. Observers judged the 3D shapes of these surfaces by marking local depth minima and maxima along three designated scan lines using a hand-held mouse. The results revealed that the local depth maxima were shifted slightly toward the direction of illumination, while the local depth minima were shifted slightly away from the direction of illumination. However, these changes were much smaller than what would be expected based on differences in the pattern of luminance among the stimulus images. These findings demonstrate that there is a substantial amount of illumination constancy in the perception of 3D shape from shading, but that it is not perfect. Several hypotheses are considered about how this constancy could potentially be achieved.No embargoAcademic Major: Psycholog

Shape from Shading: a well-posed problem?

International audienceShape From Shading is known to be an ill-posed problem. Contrary to the previous work, we show here that if we model the problem in a more realistic way than it is usually done (we take into account the 1/r2 attenuation term of the lighting), Shape From Shading can be completely well-posed. Thus the shading allows to recover (almost) any surface from only one image (of this surface), without any additional data (in particular, without regularity assumptions and without the knowledge of the heights of the solution at the local "minima". More precisely, in this report we formulate the problem as that of solving a new PDE, we develop a complete mathematical study of this equation (existence and uniqueness of the solution) and we design a new provably convergent numerical method. Finally, we test our new SFS method on various synthetic images and on our database of real images of faces, with success

A perceptually validated model for surface depth hallucination

Capturing detailed surface geometry currently requires specialized equipment such as laser range scanners, which despite their high accuracy, leave gaps in the surfaces that must be reconciled with photographic capture for relighting applications. Using only a standard digital camera and a single view, we present a method for recovering models of predominantly diffuse textured surfaces that can be plausibly relit and viewed from any angle under any illumination. Our multiscale shape-from-shading technique uses diffuse-lit/flash-lit image pairs to produce an albedo map and textured height field. Using two lighting conditions enables us to subtract one from the other to estimate albedo. In the absence of a flash-lit image of a surface for which we already have a similar exemplar pair, we approximate both albedo and diffuse shading images using histogram matching. Our depth estimation is based on local visibility. Unlike other depth-from-shading approaches, all operations are performed on the diffuse shading image in image space, and we impose no constant albedo restrictions. An experimental validation shows our method works for a broad range of textured surfaces, and viewers are frequently unable to identify our results as synthetic in a randomized presentation. Furthermore, in side-by-side comparisons, subjects found a rendering of our depth map equally plausible to one generated from a laser range scan. We see this method as a significant advance in acquiring surface detail for texturing using a standard digital camera, with applications in architecture, archaeological reconstruction, games and special effects. © 2008 ACM

New constraints on data-closeness and needle map consistency for shape-from-shading

This paper makes two contributions to the problem of needle-map recovery using shape-from-shading. First, we provide a geometric update procedure which allows the image irradiance equation to be satisfied as a hard constraint. This not only improves the data closeness of the recovered needle-map, but also removes the necessity for extensive parameter tuning. Second, we exploit the improved ease of control of the new shape-from-shading process to investigate various types of needle-map consistency constraint. The first set of constraints are based on needle-map smoothness. The second avenue of investigation is to use curvature information to impose topographic constraints. Third, we explore ways in which the needle-map is recovered so as to be consistent with the image gradient field. In each case we explore a variety of robust error measures and consistency weighting schemes that can be used to impose the desired constraints on the recovered needle-map. We provide an experimental assessment of the new shape-from-shading framework on both real world images and synthetic images with known ground truth surface normals. The main conclusion drawn from our analysis is that the data-closeness constraint improves the efficiency of shape-from-shading and that both the topographic and gradient consistency constraints improve the fidelity of the recovered needle-map

Terrain analysis using radar shape-from-shading

This paper develops a maximum a posteriori (MAP) probability estimation framework for shape-from-shading (SFS) from synthetic aperture radar (SAR) images. The aim is to use this method to reconstruct surface topography from a single radar image of relatively complex terrain. Our MAP framework makes explicit how the recovery of local surface orientation depends on the whereabouts of terrain edge features and the available radar reflectance information. To apply the resulting process to real world radar data, we require probabilistic models for the appearance of terrain features and the relationship between the orientation of surface normals and the radar reflectance. We show that the SAR data can be modeled using a Rayleigh-Bessel distribution and use this distribution to develop a maximum likelihood algorithm for detecting and labeling terrain edge features. Moreover, we show how robust statistics can be used to estimate the characteristic parameters of this distribution. We also develop an empirical model for the SAR reflectance function. Using the reflectance model, we perform Lambertian correction so that a conventional SFS algorithm can be applied to the radar data. The initial surface normal direction is constrained to point in the direction of the nearest ridge or ravine feature. Each surface normal must fall within a conical envelope whose axis is in the direction of the radar illuminant. The extent of the envelope depends on the corrected radar reflectance and the variance of the radar signal statistics. We explore various ways of smoothing the field of surface normals using robust statistics. Finally, we show how to reconstruct the terrain surface from the smoothed field of surface normal vectors. The proposed algorithm is applied to various SAR data sets containing relatively complex terrain structure