Trifocal Relative Pose from Lines at Points and its Efficient Solution

We present a new minimal problem for relative pose estimation mixing point features with lines incident at points observed in three views and its efficient homotopy continuation solver. We demonstrate the generality of the approach by analyzing and solving an additional problem with mixed point and line correspondences in three views. The minimal problems include correspondences of (i) three points and one line and (ii) three points and two lines through two of the points which is reported and analyzed here for the first time. These are difficult to solve, as they have 216 and - as shown here - 312 solutions, but cover important practical situations when line and point features appear together, e.g., in urban scenes or when observing curves. We demonstrate that even such difficult problems can be solved robustly using a suitable homotopy continuation technique and we provide an implementation optimized for minimal problems that can be integrated into engineering applications. Our simulated and real experiments demonstrate our solvers in the camera geometry computation task in structure from motion. We show that new solvers allow for reconstructing challenging scenes where the standard two-view initialization of structure from motion fails.Comment: This material is based upon work supported by the National Science Foundation under Grant No. DMS-1439786 while most authors were in residence at Brown University's Institute for Computational and Experimental Research in Mathematics -- ICERM, in Providence, R

The mathematics of surface reconstruction

This thesis discusses mathematics engineers use to produce computerized three dimensional im- ages of surfaces. It is self-contained in that all background information is included. As a result, mathematicians who know very little about the technology involved in three dimensional imaging should be able to understand the topics herein, and engineers with no differential geometry background will be able to understand the mathematics. The purpose of this thesis is to unify and understand the notation commonly used by engineers, understand their terminology, and appreciate the difficulties faced by engineers in their pursuitsIt is also intended to bridge the gap between mathematics and engineering. This paper proceeds as follows. Chapter one introduces the topic and provides a brief overview of this thesis. Chapter two provides background information on technology and differential geometry. Chapter three discusses various methods by which normal vectors are estimated. In Chapter four, we discuss methods by which curvature is estimatedIn Chapter six, we put it all together to recreate the surface. Finally, in chapter seven, we conclude with a discussion of future research. Each chapter concludes with a comparison of the methods discussed. The study of these reconstruction algorithms originated from various engineering papers on surface reconstruction. The background information was gathered from a thesis and various differential geometry texts. The challenge arises in the nature of the data with which we work. The surface must be recreated based on a set of discrete points. However, the study of surfaces is one of differential geometry which assumes differentiable functions representing the surface. Since we only have a discrete set of points, methods to overcome this shortcoming must be developed. Two categories of surface reconstruction have been developed to overcome this shortcoming. The first category estimates the data by data by smooth functionsThe second reconstructs the surface using the discrete data directly. We found that various aspects of surface reconstruction are very reliable, while others are only marginally so. We found that methods recreating the surface from discrete data directly produce very similar results suggesting that some underlying facts about surfaces represented by discrete information may be influencing the results

Instant recovery of shape from spectrum via latent space connections

We introduce the first learning-based method for recovering shapes from Laplacian spectra. Our model consists of a cycle-consistent module that maps between learned latent vectors of an auto-encoder and sequences of eigenvalues. This module provides an efficient and effective linkage between Laplacian spectrum and geometry. Our data-driven approach replaces the need for ad-hoc regularizers required by prior methods, while providing more accurate results at a fraction of the computational cost. Our learning model applies without modifications across different dimensions (2D and 3D shapes alike), representations (meshes, contours and point clouds), as well as across different shape classes, and admits arbitrary resolution of the input spectrum without affecting complexity. The increased flexibility allows us to address notoriously difficult tasks in 3D vision and geometry processing within a unified framework, including shape generation from spectrum, mesh superresolution, shape exploration, style transfer, spectrum estimation from point clouds, segmentation transfer and pointtopoint matching

Multimodal Three Dimensional Scene Reconstruction, The Gaussian Fields Framework

The focus of this research is on building 3D representations of real world scenes and objects using different imaging sensors. Primarily range acquisition devices (such as laser scanners and stereo systems) that allow the recovery of 3D geometry, and multi-spectral image sequences including visual and thermal IR images that provide additional scene characteristics. The crucial technical challenge that we addressed is the automatic point-sets registration task. In this context our main contribution is the development of an optimization-based method at the core of which lies a unified criterion that solves simultaneously for the dense point correspondence and transformation recovery problems. The new criterion has a straightforward expression in terms of the datasets and the alignment parameters and was used primarily for 3D rigid registration of point-sets. However it proved also useful for feature-based multimodal image alignment. We derived our method from simple Boolean matching principles by approximation and relaxation. One of the main advantages of the proposed approach, as compared to the widely used class of Iterative Closest Point (ICP) algorithms, is convexity in the neighborhood of the registration parameters and continuous differentiability, allowing for the use of standard gradient-based optimization techniques. Physically the criterion is interpreted in terms of a Gaussian Force Field exerted by one point-set on the other. Such formulation proved useful for controlling and increasing the region of convergence, and hence allowing for more autonomy in correspondence tasks. Furthermore, the criterion can be computed with linear complexity using recently developed Fast Gauss Transform numerical techniques. In addition, we also introduced a new local feature descriptor that was derived from visual saliency principles and which enhanced significantly the performance of the registration algorithm. The resulting technique was subjected to a thorough experimental analysis that highlighted its strength and showed its limitations. Our current applications are in the field of 3D modeling for inspection, surveillance, and biometrics. However, since this matching framework can be applied to any type of data, that can be represented as N-dimensional point-sets, the scope of the method is shown to reach many more pattern analysis applications

Construction of C\u3csup\u3e∞\u3c/sup\u3e Surfaces From Triangular Meshes Using Parametric Pseudo-Manifolds

We present a new constructive solution for the problem of fitting a smooth surface to a given triangle mesh. Our construction is based on the manifold-based approach pioneered by Grimm and Hughes. The key idea behind this approach is to define a surface by overlapping surface patches via a gluing process, as opposed to stitching them together along their common boundary curves. The manifold based approach has proved to be well-suited to fit with relative ease, Ck-continuous parametric surfaces to triangle and quadrilateral meshes, for any arbitrary finite k or even k = ∞. Smooth surfaces generated by the manifold-based approach share some of the most important properties of splines surfaces, such as local shape control and fixed-sized local support for basis functions. In addition, the differential structure of a manifold provides us with a natural setting for solving equations on the surface boundary of 3D shapes. Our new manifold-based solution possesses most of the best features of previous constructions. In particular, our construction is simple, compact, powerful, and flexible in ways of defining the geometry of the resulting surface. Unlike some of the most recent manifold-based solutions, ours has been devised to work with triangle meshes. These meshes are far more popular than any other kind of mesh encountered in computer graphics and geometry processing applications. We also provide a mathematically sound theoretical framework to undergird our solution. This theoretical framework slightly improves upon the one given by Grimm and Hughes, which was used by most manifold-based constructions introduced before

Robotic manipulation of cloth: mechanical modeling and perception

(Eng) In this work we study various mathematical problems arising from the robotic manipulation of cloth. First, we develop a locking-free continuous model for the physical simulation of inextensible textiles. We present a novel 'finite element' discretization of our inextensibility constraints which results in a unified treatment of triangle and quadrilateral meshings of the cloth. Next, we explain how to incorporate contacts, self-collisions and friction into the equations of motion, so that frictional forces and inextensibility and collision constraints may be integrated implicitly and without any decoupling. We develop an efficient 'active-set' solver tailored to our non-linear problem which takes into account past active constraints to accelerate the resolution of unresolved contacts and moreover can be initialized from any non-necessarily feasible point. Then, we embark ourselves in the empirical validation of the developed model. We record in a laboratory setting --with depth cameras and motion capture systems-- the motions of seven types of textiles (including e.g. cotton, denim and polyester) of various sizes and at different speeds and end up with more than 80 recordings. The scenarios considered are all dynamic and involve rapid shaking and twisting of the textiles, collisions with frictional objects and even strong hits with a long stick. We then, compare the recorded textiles with the simulations given by our inextensible model, and find that on average the mean error is of the order of 1 cm even for the largest sizes (DIN A2) and the most challenging scenarios. Furthermore, we also tackle other problems relevant to robotic cloth manipulation, such as cloth perception and classification of its states. We present a reconstruction algorithm based on Morse theory that proceeds directly from a point-cloud to obtain a cellular decomposition of a surface with or without boundary: the results are a piecewise parametrization of the cloth surface as a union of Morse cells. From the cellular decomposition the topology of the surface can be then deduced immediately. Finally, we study the configuration space of a piece of cloth: since the original state of a piece of cloth is flat, the set of possible states under the inextensible assumption is the set of developable surfaces isometric to a fixed one. We prove that a generic simple, closed, piecewise regular curve in space can be the boundary of only finitely many developable surfaces with nonvanishing mean curvature. Inspired on this result we introduce the dGLI cloth coordinates, a low-dimensional representation of the state of a piece of cloth based on a directional derivative of the Gauss Linking Integral. These coordinates --computed from the position of the cloth's boundary-- allow to distinguish key qualitative changes in folding sequences.(Esp) En este trabajo estudiamos varios problemas matemáticos relacionados con la manipulación robótica de textiles. En primer lugar, desarrollamos un modelo continuo libre de 'locking' para la simulación física de textiles inextensibles. Presentamos una novedosa discretización usando 'elementos finitos' de nuestras restricciones de inextensibilidad resultando en un tratamiento unificado de mallados triangulares y cuadrangulares de la tela. A continuación, explicamos cómo incorporar contactos, autocolisiones y fricción en las ecuaciones de movimiento, de modo que las fuerzas de fricción y las restricciones de inextensibilidad y colisiones puedan integrarse implícitamente y sin ningún desacoplamiento. Desarrollamos un 'solver' de tipo 'conjunto-activo' adaptado a nuestro problema no lineal que tiene en cuenta las restricciones activas pasadas para acelerar la resolución de los contactos no resueltos y, además, puede inicializarse desde cualquier punto no necesariamente factible. Posteriormente, nos embarcamos en la validación empírica del modelo desarrollado. Grabamos en un entorno de laboratorio -con cámaras de profundidad y sistemas de captura de movimiento- los movimientos de siete tipos de textiles (entre los que se incluyen, por ejemplo, algodón, tela vaquera y poliéster) de varios tamaños y a diferentes velocidades, terminando con más de 80 grabaciones. Los escenarios considerados son todos dinámicos e implican sacudidas y torsiones rápidas de los textiles, colisiones con obstáculos e incluso golpes con una varilla cilíndrica. Finalmente, comparamos las grabaciones con las simulaciones dadas por nuestro modelo inextensible, y encontramos que, de media, el error es del orden de 1 cm incluso para las telas más grandes (DIN A2) y los escenarios más complicados. Además, también abordamos otros problemas relevantes para la manipulación robótica de telas, como son la percepción y la clasificación de sus estados. Presentamos un algoritmo de reconstrucción basado en la teoría de Morse que procede directamente de una nube de puntos para obtener una descomposición celular de una superficie con o sin borde: los resultados son una parametrización a trozos de la superficie de la tela como una unión de celdas de Morse. A partir de la descomposición celular puede deducirse inmediatamente la topología de la superficie. Por último, estudiamos el espacio de configuración de un trozo de tela: dado que el estado original de la tela es plano, el conjunto de estados posibles bajo la hipótesis de inextensibilidad es el conjunto de superficies desarrollables isométricas a una fija. Demostramos que una curva genérica simple, cerrada y regular a trozos en el espacio puede ser el borde de un número finito de superficies desarrollables con curvatura media no nula. Inspirándonos en este resultado, introducimos las coordenadas dGLI, una representación de dimensión baja del estado de un pedazo de tela basada en una derivada direccional de la integral de enlazamiento de Gauss. Estas coordenadas -calculadas a partir de la posición del borde de la tela- permiten distinguir cambios cualitativos clave en distintas secuencias de plegado.Postprint (published version