3D facial shape estimation from a single image under arbitrary pose and illumination.

Humans have the uncanny ability to perceive the world in three dimensions (3D), otherwise known as depth perception. The amazing thing about this ability to determine distances is that it depends only on a simple two-dimensional (2D) image in the retina. It is an interesting problem to explain and mimic this phenomenon of getting a three-dimensional perception of a scene from a flat 2D image of the retina. The main objective of this dissertation is the computational aspect of this human ability to reconstruct the world in 3D using only 2D images from the retina. Specifically, the goal of this work is to recover 3D facial shape information from a single image of unknown pose and illumination. Prior shape and texture models from real data, which are metric in nature, are incorporated into the 3D shape recovery framework. The output recovered shape, likewise, is metric, unlike previous shape-from-shading (SFS) approaches that only provide relative shape. This work starts first with the simpler case of general illumination and fixed frontal pose. Three optimization approaches were developed to solve this 3D shape recovery problem, starting from a brute-force iterative approach to a computationally efficient regression method (Method II-PCR), where the classical shape-from-shading equation is cast as a regression framework. Results show that the output of the regression-like approach is faster in timing and similar in error metrics when compared to its iterative counterpart. The best of the three algorithms above, Method II-PCR, is compared to its two predecessors, namely: (a) Castelan et al. [1] and (b) Ahmed et al. [2]. Experimental results show that the proposed method (Method II-PCR) is superior in all aspects compared to the previous state-of-the-art. Robust statistics was also incorporated into the shape recovery framework to deal with noise and occlusion. Using multiple-view geometry concepts [3], the fixed frontal pose was relaxed to arbitrary pose. The best of the three algorithms above, Method II-PCR, once again is used as the primary 3D shape recovery method. Results show that the pose-invariant 3D shape recovery version (for input with pose) has similar error values compared to the frontal-pose version (for input with frontal pose), for input images of the same subject. Sensitivity experiments indicate that the proposed method is, indeed, invariant to pose, at least for the pan angle range of (-50° to 50°). The next major part of this work is the development of 3D facial shape recovery methods, given only the input 2D shape information, instead of both texture and 2D shape. The simpler case of output 3D sparse shapes was dealt with, initially. The proposed method, which also use a regression-based optimization approach, was compared with state-of-the art algorithms, showing decent performance. There were five conclusions that drawn from the sparse experiments, namely, the proposed approach: (a) is competitive due to its linear and non-iterative nature, (b) does not need explicit training, as opposed to [4], (c) has comparable results to [4], at a shorter computational time, (d) better in all aspects than Zhang and Samaras [5], and (e) has the limitation, together with [4] and [5], in terms of the need to manually annotate the input 2D feature points. The proposed method was then extended to output 3D dense shapes simply by replacing the sparse model with its dense equivalent, in the regression framework inside the 3D face recovery approach. The numerical values of the mean height and surface orientation error indicate that even if shading information is unavailable, a decent 3D dense reconstruction is still possible

Structure of Disk Dominated Galaxies I. Bulge/Disk Parameters, Simulations, and Secular Evolution

(Abridged) A robust analysis of galaxy structural parameters, based on the modeling of bulge and disk brightnesses in the BVRH bandpasses, is presented for 121 face-on and moderately inclined late-type spirals. Each surface brightness (SB) profile is decomposed into a sum of a generalized Sersic bulge and an exponential disk. The reliability and limitations of our bulge-to-disk (B/D) decompositions are tested with extensive simulations of galaxy brightness profiles (1D) and images (2D). Galaxy types are divided into 3 classes according to their SB profile shapes; Freeman Type-I and Type-II, and a third ``Transition'' class for galaxies whose profiles change from Type-II in the optical to Type-I in the infrared. We discuss possible interpretations of Freeman Type-II profiles. The Sersic bulge shape parameter for nearby Type-I late-type spirals shows a range between n=0.1-2 but, on average, the underlying surface density profile for the bulge and disk of these galaxies is adequately described by a double-exponential distribution. We confirm a coupling between the bulge and disk with a scale length ratio r_e/h=0.22+/-0.09, or h_bulge/h_disk=0.13+/-0.06 for late-type spirals, in agreement with recent N-body simulations of disk formation and models of secular evolution. This ratio increases from ~0.20 for late-type spirals to ~0.24 for earlier types. The similar scaling relations for early and late-type spirals suggest comparable formation and/or evolution scenarios for disk galaxies of all Hubble types.Comment: 78 pages with 23 embedded color figures + tables of galaxy structural parameters. Accepted for publication in the Astrophysical Journal. The interested reader is strongly encouraged to ignore some of the low res figures within; instead, download the high resolution version from http://www.astro.ubc.ca/people/courteau/public/macarthur02_disks.ps.g

3D-reconstruction of human jaw from a single image : integration between statistical shape from shading and shape from shading.

Object modeling is a fundamental problem in engineering, involving talents from computer-aided design, computational geometry, computer vision and advanced manufacturing. The process of object modeling takes three stages: sensing, representation, and analysis. Various sensors may be used to capture information about objects; optical cam- eras and laser scanners are common with rigid objects, while X-ray, CT and MRI are common with biological organs. These sensors may provide a direct or indirect inference about the object, requiring a geometric representation in the computer that is suitable for subsequent usage. Geometric representations that are compact, i.e., capture the main features of the objects with minimal number of data points or vertices, fall into the domain of computational geometry. Once a compact object representation is in the computer, various analysis steps can be conducted, including recognition, coding, transmission, etc. The subject matter of this thesis is object reconstruction from a sequence of optical images. An approach to estimate the depth of the visible portion of the human teeth from intraoral cameras has been developed, extending the classical shape from shading (SFS) solution to non-Lambertian surfaces with known object illumination characteristics. To augment the visible portion, and in order to have the entire jaw reconstructed without the use of CT or MRI or even X-rays, additional information will be added to database of human jaws. This database has been constructed from an adult population with variations in teeth size, degradation and alignments. The database contains both shape and albedo information for the population. Using this database, a novel statistical shape from shading (SSFS) approach has been created. To obtain accurate result from shape from shading and statistical shape from shading, final step will be integrated two approaches (SFS,SSFS) by using Iterative Closest Point algorithm (ICP). Keywords: computer vision, shading, 3D shape reconstruction, shape from shading, statistical, shape from shading, Iterative Closest Point

The Dark Energy Survey

We describe the Dark Energy Survey (DES), a proposed optical-near infrared survey of 5000 sq. deg of the South Galactic Cap to ~24th magnitude in SDSS griz, that would use a new 3 sq. deg CCD camera to be mounted on the Blanco 4-m telescope at Cerro Telolo Inter-American Observatory (CTIO). The survey data will allow us to measure the dark energy and dark matter densities and the dark energy equation of state through four independent methods: galaxy clusters, weak gravitational lensing tomography, galaxy angular clustering, and supernova distances. These methods are doubly complementary: they constrain different combinations of cosmological model parameters and are subject to different systematic errors. By deriving the four sets of measurements from the same data set with a common analysis framework, we will obtain important cross checks of the systematic errors and thereby make a substantial and robust advance in the precision of dark energy measurements.Comment: White Paper submitted to the Dark Energy Task Force, 42 page

Phenomenological modeling of image irradiance for non-Lambertian surfaces under natural illumination.

Various vision tasks are usually confronted by appearance variations due to changes of illumination. For instance, in a recognition system, it has been shown that the variability in human face appearance is owed to changes to lighting conditions rather than person\u27s identity. Theoretically, due to the arbitrariness of the lighting function, the space of all possible images of a fixed-pose object under all possible illumination conditions is infinite dimensional. Nonetheless, it has been proven that the set of images of a convex Lambertian surface under distant illumination lies near a low dimensional linear subspace. This result was also extended to include non-Lambertian objects with non-convex geometry. As such, vision applications, concerned with the recovery of illumination, reflectance or surface geometry from images, would benefit from a low-dimensional generative model which captures appearance variations w.r.t. illumination conditions and surface reflectance properties. This enables the formulation of such inverse problems as parameter estimation. Typically, subspace construction boils to performing a dimensionality reduction scheme, e.g. Principal Component Analysis (PCA), on a large set of (real/synthesized) images of object(s) of interest with fixed pose but different illumination conditions. However, this approach has two major problems. First, the acquired/rendered image ensemble should be statistically significant vis-a-vis capturing the full behavior of the sources of variations that is of interest, in particular illumination and reflectance. Second, the curse of dimensionality hinders numerical methods such as Singular Value Decomposition (SVD) which becomes intractable especially with large number of large-sized realizations in the image ensemble. One way to bypass the need of large image ensemble is to construct appearance subspaces using phenomenological models which capture appearance variations through mathematical abstraction of the reflection process. In particular, the harmonic expansion of the image irradiance equation can be used to derive an analytic subspace to represent images under fixed pose but different illumination conditions where the image irradiance equation has been formulated in a convolution framework. Due to their low-frequency nature, irradiance signals can be represented using low-order basis functions, where Spherical Harmonics (SH) has been extensively adopted. Typically, an ideal solution to the image irradiance (appearance) modeling problem should be able to incorporate complex illumination, cast shadows as well as realistic surface reflectance properties, while moving away from the simplifying assumptions of Lambertian reflectance and single-source distant illumination. By handling arbitrary complex illumination and non-Lambertian reflectance, the appearance model proposed in this dissertation moves the state of the art closer to the ideal solution. This work primarily addresses the geometrical compliance of the hemispherical basis for representing surface reflectance while presenting a compact, yet accurate representation for arbitrary materials. To maintain the plausibility of the resulting appearance, the proposed basis is constructed in a manner that satisfies the Helmholtz reciprocity property while avoiding high computational complexity. It is believed that having the illumination and surface reflectance represented in the spherical and hemispherical domains respectively, while complying with the physical properties of the surface reflectance would provide better approximation accuracy of image irradiance when compared to the representation in the spherical domain. Discounting subsurface scattering and surface emittance, this work proposes a surface reflectance basis, based on hemispherical harmonics (HSH), defined on the Cartesian product of the incoming and outgoing local hemispheres (i.e. w.r.t. surface points). This basis obeys physical properties of surface reflectance involving reciprocity and energy conservation. The basis functions are validated using analytical reflectance models as well as scattered reflectance measurements which might violate the Helmholtz reciprocity property (this can be filtered out through the process of projecting them on the subspace spanned by the proposed basis, where the reciprocity property is preserved in the least-squares sense). The image formation process of isotropic surfaces under arbitrary distant illumination is also formulated in the frequency space where the orthogonality relation between illumination and reflectance bases is encoded in what is termed as irradiance harmonics. Such harmonics decouple the effect of illumination and reflectance from the underlying pose and geometry. Further, a bilinear approach to analytically construct irradiance subspace is proposed in order to tackle the inherent problem of small-sample-size and curse of dimensionality. The process of finding the analytic subspace is posed as establishing a relation between its principal components and that of the irradiance harmonics basis functions. It is also shown how to incorporate prior information about natural illumination and real-world surface reflectance characteristics in order to capture the full behavior of complex illumination and non-Lambertian reflectance. The use of the presented theoretical framework to develop practical algorithms for shape recovery is further presented where the hitherto assumed Lambertian assumption is relaxed. With a single image of unknown general illumination, the underlying geometrical structure can be recovered while accounting explicitly for object reflectance characteristics (e.g. human skin types for facial images and teeth reflectance for human jaw reconstruction) as well as complex illumination conditions. Experiments on synthetic and real images illustrate the robustness of the proposed appearance model vis-a-vis illumination variation. Keywords: computer vision, computer graphics, shading, illumination modeling, reflectance representation, image irradiance, frequency space representations, {hemi)spherical harmonics, analytic bilinear PCA, model-based bilinear PCA, 3D shape reconstruction, statistical shape from shading

The Gemini/HST Cluster Project: Structural and Photometric Properties of Galaxies in Three z =0.28-0.89 Clusters

We present the data processing and analysis techniques we are using to determine structural and photometric properties of galaxies in our Gemini/HST Galaxy Cluster Project sample. The goal of this study is to understand cluster galaxy evolution in terms of scaling relations and structural properties of cluster galaxies at redshifts 0.15 < z < 1.0. To derive parameters such as total magnitude, half-light radius, effective surface brightness, and Sersic n, we fit r^{1/4} law and Sersic function 2-D surface brightness profiles to each of the galaxies in our sample. Using simulated galaxies, we test how the assumed profile affects the derived parameters and how the uncertainties affect our Fundamental Plane results. We find that while fitting galaxies which have Sersic index n < 4 with r^{1/4} law profiles systematically overestimates the galaxy radius and flux, the combination of profile parameters that enter the Fundamental Plane has uncertainties that are small. Average systematic offsets and associated random uncertainties in magnitude and log r_e for n > 2 galaxies fitted with r^{1/4} law profiles are -0.1+-0.3 and 0.1+-0.2 respectively. The combination of effective radius and surface brightness, log r_e - \beta log _e, that enters the Fundamental Plane produces offsets smaller than -0.02+-0.10. This systematic error is insignificant and independent of galaxy magnitude or size. A catalog of photometry and surface brightness profile parameters is presented for three of the clusters in our sample, RX J0142.0+2131, RX J0152.7-1357, and RX J1226.9+3332 at redshifts 0.28, 0.83, and 0.89 respectively.Comment: 31 pages, 15 figure

Reconstructing emission from pre-reionization sources with cosmic infrared background fluctuation measurements by the JWST

We present new methodology to use cosmic infrared background (CIB) fluctuations to probe sources at 10<z<30 from a JWST/NIRCam configuration that will isolate known galaxies to 28 AB mag at 0.5--5 micron. At present significant mutually consistent source-subtracted CIB fluctuations have been identified in the Spitzer and Akari data at 2--5 micron, but we demonstrate internal inconsistencies at shorter wavelengths in the recent CIBER data. We evaluate CIB contributions from remaining galaxies and show that the bulk of the high-z sources will be in the confusion noise of the NIRCam beam, requiring CIB studies. The accurate measurement of the angular spectrum of the fluctuations and probing the dependence of its clustering component on the remaining shot noise power would discriminate between the various currently proposed models for their origin and probe the flux distribution of its sources. We show that the contribution to CIB fluctuations from remaining galaxies is large at visible wavelengths for the current instruments precluding probing the putative Lyman-break of the CIB fluctuations. We demonstrate that with the proposed JWST configuration such measurements will enable probing the Lyman break. We develop a Lyman-break tomography method to use the NIRCam wavelength coverage to identify or constrain, via the adjacent two-band subtraction, the history of emissions over 10<z<30 as the Universe comes out of the 'Dark Ages'. We apply the proposed tomography to the current Spitzer/IRAC measurements at 3.6 and 4.5 micron, to find that it already leads to interestingly low upper limit on emissions at z>30.Comment: ApJ, in press. Minor revisions/additions to match the version in proof

Resolved stellar population of distant galaxies in the ELT era

The expected imaging capabilities of future Extremely Large Telescopes (ELTs) will offer the unique possibility to investigate the stellar population of distant galaxies from the photometry of the stars in very crowded fields. Using simulated images and photometric analysis we explore here two representative science cases aimed at recovering the characteristics of the stellar populations in the inner regions of distant galaxies. Specifically: case A) at the center of the disk of a giant spiral in the Centaurus Group, (mu B~21, distance of 4.6 Mpc); and, case B) at half of the effective radius of a giant elliptical in the Virgo Cluster (mu~19.5, distance of 18 Mpc). We generate synthetic frames by distributing model stellar populations and adopting a representative instrumental set up, i.e. a 42 m Telescope operating close to the diffraction limit. The effect of crowding is discussed in detail showing how stars are measured preferentially brighter than they are as the confusion limit is approached. We find that (i) accurate photometry (sigma~0.1, completeness >90%) can be obtained for case B) down to I~28.5, J~27.5 allowing us to recover the stellar metallicity distribution in the inner regions of ellipticals in Virgo to within ~0.1 dex; (ii) the same photometric accuracy holds for the science case A) down to J~28.0, K~27.0, enabling to reconstruct of the star formation history up to the Hubble time via simple star counts in diagnostic boxes. For this latter case we discuss the possibility of deriving more detailed information on the star formation history from the analysis of their Horizontal Branch stars. We show that the combined features of high sensitivity and angular resolution of ELTs may open a new era for our knowledge of the stellar content of galaxies of different morphological type up to the distance of the Virgo cluster.Comment: 21 pages, 17 figures, PASP accepted in pubblicatio

A fast 3D reconstruction system with a low-cost camera accessory

Photometric stereo is a three dimensional (3D) imaging technique that uses multiple 2D images, obtained from a fixed camera perspective, with different illumination directions. Compared to other 3D imaging methods such as geometry modeling and 3D-scanning, it comes with a number of advantages, such as having a simple and efficient reconstruction routine. In this work, we describe a low-cost accessory to a commercial digital single-lens reflex (DSLR) camera system allowing fast reconstruction of 3D objects using photometric stereo. The accessory consists of four white LED lights fixed to the lens of a commercial DSLR camera and a USB programmable controller board to sequentially control the illumination. 3D images are derived for different objects with varying geometric complexity and results are presented, showing a typical height error of &lt;3 mm for a 50 mm sized object