Resolving Ambiguities in Monocular 3D Reconstruction of Deformable Surfaces

In this thesis, we focus on the problem of recovering 3D shapes of deformable surfaces from a single camera. This problem is known to be ill-posed as for a given 2D input image there exist many 3D shapes that give visually identical projections. We present three methods which make headway towards resolving these ambiguities. We believe that our work represents a significant step towards making surface reconstruction methods of practical use. First, we propose a surface reconstruction method that overcomes the limitations of the state-of-the-art template-based and non-rigid structure from motion methods. We neither track points over many frames, nor require a sophisticated deformation model, or depend on a reference image. In our method, we establish correspondences between pairs of frames in which the shape is different and unknown. We then estimate homographies between corresponding local planar patches in both images. These yield approximate 3D reconstructions of points within each patch up to a scale factor. Since we consider overlapping patches, we can enforce them to be consistent over the whole surface. Finally, a local deformation model is used to fit a triangulated mesh to the 3D point cloud, which makes the reconstruction robust to both noise and outliers in the image data. Second, we propose a novel approach to recovering the 3D shape of a deformable surface from a monocular input by taking advantage of shading information in more generic contexts than conventional Shape-from-Shading (SfS) methods. This includes surfaces that may be fully or partially textured and lit by arbitrarily many light sources. To this end, given a lighting model, we learn the relationship between a shading pattern and the corresponding local surface shape. At run time, we first use this knowledge to recover the shape of surface patches and then enforce spatial consistency between the patches to produce a global 3D shape. Instead of treating texture as noise as in many SfS approaches, we exploit it as an additional source of information. We validate our approach quantitatively and qualitatively using both synthetic and real data. Third, we introduce a constrained latent variable model that inherently accounts for geometric constraints such as inextensibility defined on the mesh model. To this end, we learn a non-linear mapping from the latent space to the output space, which corresponds to vertex positions of a mesh model, such that the generated outputs comply with equality and inequality constraints expressed in terms of the problem variables. Since its output is encouraged to satisfy such constraints inherently, using our model removes the need for computationally expensive methods that enforce these constraints at run time. In addition, our approach is completely generic and could be used in many other different contexts as well, such as image classification to impose separation of the classes, and articulated tracking to constrain the space of possible poses

Simultaneous Point Matching and 3D Deformable Surface Reconstruction

It has been shown that the 3D shape of a deformable surface in an image can be recovered by establishing correspondences between that image and a reference one in which the shape is known. These matches can then be used to set-up a convex optimization problem in terms of the shape parameters, which is easily solved. However, in many cases, the correspondences are hard to establish reliably. In this paper, we show that we can solve simultaneously for both 3D shape and correspondences, thereby using 3D shape constraints to guide the image matching and increasing robustness, for example when the textures are repetitive. This involves solving a mixed integer quadratic problem. While optimizing this problem is NP-hard in general, we show that its solution can nevertheless be approximated effectively by a branch-and-bound algorithm

Least-Squares Minimization Under Constraints

Unconstrained Least-Squares minimization is a well-studied problem. For example, the Levenberg-Marquardt is extremely effective and numerous implementations are readily available. These algorithms are, however, not designed to perform least-squares minimization under hard constraints. This short report outlines two very simple approaches to doing this. The first relies on standard Lagrange multipliers. The second is inspired by inverse kinematics techniques

Template-free monocular reconstruction of deformable surfaces

It has recently been shown that deformable 3D surfaces could be recovered from single video streams. However, existing techniques either require a reference view in which the shape of the surface is known a priori, which often may not be available, or require tracking points over long sequences, which is hard to do. In this paper, we overcome these limitations. To this end, we establish correspondences between pairs of frames in which the shape is different and unknown. We then estimate homographies between corresponding local planar patches in both images. These yield approximate 3D reconstructions of points within each patch up to a scale factor. Since we consider overlapping patches, we can enforce them to be consistent over the whole surface. Finally, a local deformation model is used to fit a triangulated mesh to the 3D point cloud, which makes the reconstruction robust to both noise and outliers in the image data

Monocular 3D Reconstruction of Locally Textured Surfaces

Most recent approaches to monocular non-rigid 3D shape recovery rely on exploiting point correspondences and work best when the whole surface is well-textured. The alternative is to rely either on contours or shading information, which has only been demonstrated in very restrictive settings. Here, we propose a novel approach to monocular deformable shape recovery that can operate under complex lighting and handle partially textured surfaces. At the heart of our algorithm are a learned mapping from intensity patterns to the shape of local surface patches and a principled approach to piecing together the resulting local shape estimates. We validate our approach quantitatively and qualitatively using both synthetic and real data

A Constrained Latent Variable Model

Latent variable models provide valuable compact representations for learning and inference in many computer vision tasks. However, most existing models cannot directly encode prior knowledge about the specific problem at hand. In this paper, we introduce a constrained latent variable model whose generated output inherently accounts for such knowledge. To this end, we propose an approach that explicitly imposes equality and inequality constraints on the model's output during learning, thus avoiding the computational burden of having to account for these constraints at inference. Our learning mechanism can exploit non-linear kernels, while only involving sequential closed-form updates of the model parameters. We demonstrate the effectiveness of our constrained latent variable model on the problem of non-rigid 3D reconstruction from monocular images, and show that it yields qualitative and quantitative improvements over several baselines

Laplacian Meshes for Monocular 3D Shape Recovery

We show that by extending the Laplacian formalism, which was first introduced in the Graphics community to regularize 3D meshes, we can turn the monocular 3D shape reconstruction of a deformable surface given correspondences with a reference image into a well-posed problem. Furthermore, this does not require any training data and eliminates the need to pre-align the reference shape with the one to be reconstructed, as was done in earlier methods

Neogene Uplift and Magmatism of Anatolia: Insights from Drainage Analysis and Basaltic Geochemistry

It is generally agreed that mantle dynamics have played a significant role in generating and maintaining the elevated topography of Anatolia during Neogene times. However, there is much debate about the relative importance of subduction zone and asthenospheric processes. Key issues concern onset and cause of regional uplift, thickness of the lithospheric plate, and the presence or absence of temperature and/or compositional anomalies within the convecting mantle. Here, we tackle these interlinked issues by analyzing and modeling two disparate suites of observations. First, a drainage inventory of 1,844 longitudinal river profiles is assembled. This geomorphic database is inverted to calculate the variation of Neogene regional uplift through time and space by minimizing the misfit between observed and calculated river profiles subject to independent calibration. Our results suggest that regional uplift commenced in the east at 20 Ma and propagated westward. Secondly, we have assembled a database of geochemical analyses of basaltic rocks. Two different approaches have been used to quantitatively model this database with a view to determining the depth and degree of asthenospheric melting across Anatolia. Our results suggest that melting occurs at depths as shallow as 60 km in the presence of mantle potential temperatures as high as 1400°C. There is evidence that potential temperatures are higher in the east, consistent with the pattern of sub-plate shear wave velocity anomalies. Our combined results are consistent with isostatic and admittance analyses and suggest that elevated asthenospheric temperatures beneath thinned Anatolian lithosphere have played a first order role in generating and maintaining regional dynamic topography and basaltic magmatism

Relative contributions of crust and mantle to generation of Campanian high-K calc-alkaline I-type granitoids in a subduction setting, with special reference to the Harsit Pluton, Eastern Turkey

We present elemental and Sr-Nd-Pb isotopic data for the magmatic suite (similar to 79 Ma) of the Harsit pluton, from the Eastern Pontides (NE Turkey), with the aim of determining its magma source and geodynamic evolution. The pluton comprises granite, granodiorite, tonalite and minor diorite (SiO(2) = 59.43-76.95 wt%), with only minor gabbroic diorite mafic microgranular enclaves in composition (SiO(2) = 54.95-56.32 wt%), and exhibits low Mg# (<46). All samples show a high-K calc-alkaline differentiation trend and I-type features. The chondrite-normalized REE patterns are fractionated [(La/Yb)(n) = 2.40-12.44] and display weak Eu anomalies (Eu/Eu* = 0.30-0.76). The rocks are characterized by enrichment of LILE and depletion of HFSE. The Harsit host rocks have weak concave-upward REE patterns, suggesting that amphibole and garnet played a significant role in their generation during magma segregation. The host rocks and their enclaves are isotopically indistinguishable. Sr-Nd isotopic data for all of the samples display I(Sr) = 0.70676-0.70708, epsilon(Nd)(79 Ma) = -4.4 to -3.3, with T(DM) = 1.09-1.36 Ga. The lead isotopic ratios are ((206)Pb/(204)pb) = 18.79-18.87, ((207)Pb/(204)Pb) = 15.59-15.61 and ((208)Pb/(204)Pb) = 38.71-38.83. These geochemical data rule out pure crustal-derived magma genesis in a post-collision extensional stage and suggest mixed-origin magma generation in a subduction setting. The melting that generated these high-K granitoidic rocks may have resulted from the upper Cretaceous subduction of the Izmir-Ankara-Erzincan oceanic slab beneath the Eurasian block in the region. The back-arc extensional events would have caused melting of the enriched subcontinental lithospheric mantle and formed mafic magma. The underplating of the lower crust by mafic magmas would have played a significant role in the generation of high-K magma. Thus, a thermal anomaly induced by underplated basic magma into a hot crust would have caused partial melting in the lower part of the crust. In this scenario, the lithospheric mantle-derived basaltic melt first mixed with granitic magma of crustal origin at depth. Then, the melts, which subsequently underwent a fractional crystallization and crustal assimilation processes, could ascend to shallower crustal levels to generate a variety of rock types ranging from diorite to granite. Sr-Nd isotope modeling shows that the generation of these magmas involved similar to 65-75% of the lower crustal-derived melt and similar to 25-35% of subcontinental lithospheric mantle. Further, geochemical data and the Ar-Ar plateau age on hornblende, combined with regional studies, imply that the Harsit pluton formed in a subduction setting and that the back-arc extensional period started by least similar to 79 Ma in the Eastern Pontides.Geochemistry & GeophysicsMineralogySCI(E)33ARTICLE4467-48716