Affine registration of point clouds based on point-to-plane approach

The problem of aligning of 3D point data is the known registration task. The most popular registration algorithm is the Iterative Closest Point (ICP). This paper proposes a new algorithm for affine registration of point clouds by incorporating the affine transformation into the point-toplane ICP algorithm. At each iterative step of the algorithm, a closed-form solution for the affine transformation is derived.The work was supported by the Ministry of Education and Science of Russian Federation (grant № 2.1743.2017)

Muestreo de imágenes de rango en el espacio de variación de la orientación

La información proveniente de un conjunto de imágenes de rango puede utilizarse para producir modelos computacionales en 3D de la escena. Sin embargo, cada imagen esta referida a las coordenadas de la cámara en el momento de la adquisición. Encontrar un conjunto de transformaciones tales que, aplicadas a las imágenes, lleven el conjunto a un sistema coordenado común, es usualmente denominado registro de imágenes de rango. Uno de los grandes problemas en dicho proceso está relacionado con datos seleccionados de forma arbitraria y que no son relevantes en el registro. En éste artículo se propone un nuevo método de muestreo, basado en el uso de la información local de variación de la orientación (curvatura). Se muestra que, el rendimiento del algoritmo es adecuado en comparación con las técnicas habituales en la reconstrucción de modelos a partir de imágenes de rango.Palabras claves: Imágenes de rango, métodos de muestreo, registro

Registration of range images using a histogram based metric

In this paper, a new approach for coarse registration of triangular meshes for three-dimensional object reconstruction is proposed by using the shape-index measure for topological description and the Earth Mover's Distance (EMD) metric for region similarity estimation. Two of its main advantages are independence from initial pre-alignment and spatial transformation, obtaining a more accurate registration regardless of the initial viewing position

Vision numérique et modèles 3D pour imagerie moléculaire sur petits animaux par tomographie optique

Typiquement, en tomographie optique diffuse (TOD), les mesures optiques sont prises en amenant des fibres optiques en contact avec le sujet ou en faisant baigner le sujet dans un fluide adaptateur d'indice. Ces deux approches simplifient grandement le problème inverse pour la reconstruction tomographique, car seule la propagation de la lumière dans les tissus biologiques doit être considérée. Dans le cas de l'imagerie sur petits animaux, il est très difficile d'amener des fibres optiques en contact avec le sujet de façon automatisée sans l'écraser et sans changer sa géométrie. L'utilisation de fluides adaptateurs d'indice simplifie la géométrie du problème à celle du contenant, généralement de forme cylindrique, où se trouve l'animal. Par contre, il n'est pas pratique d'avoir à entretenir un tel système et il est difficile de mettre l'animal dans un fluide sans le noyer. L'utilisation de fluides adaptateurs d'indice atténue le signal optique menant à des mesures plus bruitées. Les sytèmes sans contact permettent d'éviter tous les problèmes mentionnés précédemment, mais nécessitent la mesure de la forme extérieure du sujet. Dans le cadre des présents travaux de recherche, un système de vision numérique utilisant une paire de caméras et un laser pour mesurer la forme extérieure 3D de sujets est présenté. La conception du système de vision numérique 3D vise à faciliter son intégration au système de TOD qui est présentement développé au sein du groupe TomOptUS. Le principal avantage du système de vision numérique est de minimiser la complexité du système de TOD en utlisant le même laser pour les mesures tomographiques optiques et pour les mesures 3D, permettant simultanément l'acquisition de modèles 3D et de données optiques. Cette approche permet de mesurer la position exacte à laquelle la lumière du laser est injectée dans le sujet, alors que cette postion est habituellement déduite et non mesurée par les autres systèmes. Cette information est extrêmement importante pour la reconstruction tomographique. Des mesures 3D précises (<1mm) sont obtenues à l'aide d'algorithmes pour l'étalonnage de l'axe de rotation et de translation. Des mesures 3D d'une forme de référence et d'une souris sont présentées démontrant la précision et l'efficacité du système

An initial matching and mapping for dense 3D object tracking in augmented reality applications

Augmented Reality (AR) applications rely on efficient and robust methods of tracking. One type of tracking uses dense 3D point data representations of the object to track. As opposed to sparse, dense tracking approaches are highly accurate and precise by considering all of the available data from a camera. A major challenge to dense tracking is that it requires a rough initial matching and mapping to begin. A matching means that from a known object, we can determine the object exists in the scene, and a mapping means that we can identify the position and orientation of an object with respect to the camera. Current methods to provide the initial matching and mapping require the user to manually input parameters, or wait an extended amount of time for a brute force automatic approach. The research presented in this thesis develops an automatic initial matching and mapping for dense tracking for AR, facilitating natural AR systems that track 3D objects. To do this, an existing offline method for registration of ideal 3D object point sets is proposed as a starting point. The method is improved and optimized in four steps to address the requirements and challenges for dense tracking in AR with a noisy consumer sensor. A series of experiments verifies the suitability of the optimizations, using increasingly large and more complex scene point clouds, and the results are presented

Pose Invariant 3D Face Authentication based on Gaussian Fields Approach

This thesis presents a novel illuminant invariant approach to recognize the identity of an individual from his 3D facial scan in any pose, by matching it with a set of frontal models stored in the gallery. In view of today’s security concerns, 3D face reconstruction and recognition has gained a significant position in computer vision research. The non intrusive nature of facial data acquisition makes face recognition one of the most popular approaches for biometrics-based identity recognition. Depth information of a 3D face can be used to solve the problems of illumination and pose variation associated with face recognition. The proposed method makes use of 3D geometric (point sets) face representations for recognizing faces. The use of 3D point sets to represent human faces in lieu of 2D texture makes this method robust to changes in illumination and pose. The method first automatically registers facial point-sets of the probe with the gallery models through a criterion based on Gaussian force fields. The registration method defines a simple energy function, which is always differentiable and convex in a large neighborhood of the alignment parameters; allowing for the use of powerful standard optimization techniques. The new method overcomes the necessity of close initialization and converges in much less iterations as compared to the Iterative Closest Point algorithm. The use of an optimization method, the Fast Gauss Transform, allows a considerable reduction in the computational complexity of the registration algorithm. Recognition is then performed by using the robust similarity score generated by registering 3D point sets of faces. Our approach has been tested on a large database of 85 individuals with 521 scans at different poses, where the gallery and the probe images have been acquired at significantly different times. The results show the potential of our approach toward a fully pose and illumination invariant system. Our method can be successfully used as a potential biometric system in various applications such as mug shot matching, user verification and access control, and enhanced human computer interaction

Multimodal Three Dimensional Scene Reconstruction, The Gaussian Fields Framework

The focus of this research is on building 3D representations of real world scenes and objects using different imaging sensors. Primarily range acquisition devices (such as laser scanners and stereo systems) that allow the recovery of 3D geometry, and multi-spectral image sequences including visual and thermal IR images that provide additional scene characteristics. The crucial technical challenge that we addressed is the automatic point-sets registration task. In this context our main contribution is the development of an optimization-based method at the core of which lies a unified criterion that solves simultaneously for the dense point correspondence and transformation recovery problems. The new criterion has a straightforward expression in terms of the datasets and the alignment parameters and was used primarily for 3D rigid registration of point-sets. However it proved also useful for feature-based multimodal image alignment. We derived our method from simple Boolean matching principles by approximation and relaxation. One of the main advantages of the proposed approach, as compared to the widely used class of Iterative Closest Point (ICP) algorithms, is convexity in the neighborhood of the registration parameters and continuous differentiability, allowing for the use of standard gradient-based optimization techniques. Physically the criterion is interpreted in terms of a Gaussian Force Field exerted by one point-set on the other. Such formulation proved useful for controlling and increasing the region of convergence, and hence allowing for more autonomy in correspondence tasks. Furthermore, the criterion can be computed with linear complexity using recently developed Fast Gauss Transform numerical techniques. In addition, we also introduced a new local feature descriptor that was derived from visual saliency principles and which enhanced significantly the performance of the registration algorithm. The resulting technique was subjected to a thorough experimental analysis that highlighted its strength and showed its limitations. Our current applications are in the field of 3D modeling for inspection, surveillance, and biometrics. However, since this matching framework can be applied to any type of data, that can be represented as N-dimensional point-sets, the scope of the method is shown to reach many more pattern analysis applications

Single View Modeling and View Synthesis

This thesis develops new algorithms to produce 3D content from a single camera. Today, amateurs can use hand-held camcorders to capture and display the 3D world in 2D, using mature technologies. However, there is always a strong desire to record and re-explore the 3D world in 3D. To achieve this goal, current approaches usually make use of a camera array, which suffers from tedious setup and calibration processes, as well as lack of portability, limiting its application to lab experiments. In this thesis, I try to produce the 3D contents using a single camera, making it as simple as shooting pictures. It requires a new front end capturing device rather than a regular camcorder, as well as more sophisticated algorithms. First, in order to capture the highly detailed object surfaces, I designed and developed a depth camera based on a novel technique called light fall-off stereo (LFS). The LFS depth camera outputs color+depth image sequences and achieves 30 fps, which is necessary for capturing dynamic scenes. Based on the output color+depth images, I developed a new approach that builds 3D models of dynamic and deformable objects. While the camera can only capture part of a whole object at any instance, partial surfaces are assembled together to form a complete 3D model by a novel warping algorithm. Inspired by the success of single view 3D modeling, I extended my exploration into 2D-3D video conversion that does not utilize a depth camera. I developed a semi-automatic system that converts monocular videos into stereoscopic videos, via view synthesis. It combines motion analysis with user interaction, aiming to transfer as much depth inferring work from the user to the computer. I developed two new methods that analyze the optical flow in order to provide additional qualitative depth constraints. The automatically extracted depth information is presented in the user interface to assist with user labeling work. In this thesis, I developed new algorithms to produce 3D contents from a single camera. Depending on the input data, my algorithm can build high fidelity 3D models for dynamic and deformable objects if depth maps are provided. Otherwise, it can turn the video clips into stereoscopic video