Enhancement Of Stereo Imagery By Artificial Texture Projection Generated Using A Lidar

Passive stereo imaging is capable of producing dense 3D data, but image matching algorithms generally perform poorly on images with large regions of homogenous texture due to ambiguous match costs. Stereo systems can be augmented with an additional light source that can project some form of unique texture onto surfaces in the scene. Methods include structured light, laser projection through diffractive optical elements, data projectors and laser speckle. Pattern projection using lasers has the advantage of producing images with a high signal to noise ratio. We have investigated the use of a scanning visible-beam LIDAR to simultaneously provide enhanced texture within the scene and to provide additional opportunities for data fusion in unmatched regions. The use of a LIDAR rather than a laser alone allows us to generate highly accurate ground truth data sets by scanning the scene at high resolution. This is necessary for evaluating different pattern projection schemes. Results from LIDAR generated random dots are presented and compared to other texture projection techniques. Finally, we investigate the use of image texture analysis to intelligently project texture where it is required while exploiting the texture available in the ambient light image

Fast 3-D fingertip reconstruction using a single two-view structured light acquisition

Current contactless fingertip recognition systems based on three-dimensional finger models mostly use multiple views (N gt;; 2) or structured light illumination with multiple patterns projected over a period of time. In this paper, we present a novel methodology able to obtain a fast and accurate three-dimensional reconstruction of the fingertip by using a single two-view acquisition and a static projected pattern. The acquisition setup is less constrained than the ones proposed in the literature and requires only that the finger is placed according to the depth of focus of the cameras, and in the overlapping field of views. The obtained pairs of images are processed in order to extract the information related to the fingertip and the projected pattern. The projected pattern permits to extract a set of reference points in the two images, which are then matched by using a correlation approach. The information related to a previous calibration of the cameras is then used in order to estimate the finger model, and one input image is wrapped on the resulting three-dimensional model, obtaining a three-dimensional pattern with a limited distortion of the ridges. In order to obtain data that can be treated by traditional algorithms, the obtained three-dimensional models are then unwrapped into bidimensional images. The quality of the unwrapped images is evaluated by using a software designed for contact-based fingerprint images. The obtained results show that the methodology is feasible and a realistic three-dimensional reconstruction can be achieved with few constraints. These results also show that the fingertip models computed by using our approach can be processed by both specific three-dimensional matching algorithms and traditional matching approaches. We also compared the results with the ones obtained without using structured light techniques, showing that the use of a projector achieves a faster and more accurate fingertip reconstruction

Projected texture stereo

Abstract — Passive stereo vision is widely used as a range sensing technology in robots, but suffers from dropouts: areas of low texture where stereo matching fails. By supplementing a stereo system with a strong texture projector, dropouts can be eliminated or reduced. This paper develops a practical stereo projector system, first by finding good patterns to project in the ideal case, then by analyzing the effects of system blur and phase noise on these patterns, and finally by designing a compact projector that is capable of good performance out to 3m in indoor scenes. The system has been implemented and has excellent depth precision and resolution, especially in the range out to 1.5m. I

Robot guidance using machine vision techniques in industrial environments: A comparative review

In the factory of the future, most of the operations will be done by autonomous robots that need visual feedback to move around the working space avoiding obstacles, to work collaboratively with humans, to identify and locate the working parts, to complete the information provided by other sensors to improve their positioning accuracy, etc. Different vision techniques, such as photogrammetry, stereo vision, structured light, time of flight and laser triangulation, among others, are widely used for inspection and quality control processes in the industry and now for robot guidance. Choosing which type of vision system to use is highly dependent on the parts that need to be located or measured. Thus, in this paper a comparative review of different machine vision techniques for robot guidance is presented. This work analyzes accuracy, range and weight of the sensors, safety, processing time and environmental influences. Researchers and developers can take it as a background information for their future works

Analysis of 3D human gait reconstructed with a depth camera and mirrors

L'évaluation de la démarche humaine est l'une des composantes essentielles dans les soins de santé. Les systèmes à base de marqueurs avec plusieurs caméras sont largement utilisés pour faire cette analyse. Cependant, ces systèmes nécessitent généralement des équipements spécifiques à prix élevé et/ou des moyens de calcul intensif. Afin de réduire le coût de ces dispositifs, nous nous concentrons sur un système d'analyse de la marche qui utilise une seule caméra de profondeur. Le principe de notre travail est similaire aux systèmes multi-caméras, mais l'ensemble de caméras est remplacé par un seul capteur de profondeur et des miroirs. Chaque miroir dans notre configuration joue le rôle d'une caméra qui capture la scène sous un point de vue différent. Puisque nous n'utilisons qu'une seule caméra, il est ainsi possible d'éviter l'étape de synchronisation et également de réduire le coût de l'appareillage. Notre thèse peut être divisée en deux sections: reconstruction 3D et analyse de la marche. Le résultat de la première section est utilisé comme entrée de la seconde. Notre système pour la reconstruction 3D est constitué d'une caméra de profondeur et deux miroirs. Deux types de capteurs de profondeur, qui se distinguent sur la base du mécanisme d'estimation de profondeur, ont été utilisés dans nos travaux. Avec la technique de lumière structurée (SL) intégrée dans le capteur Kinect 1, nous effectuons la reconstruction 3D à partir des principes de l'optique géométrique. Pour augmenter le niveau des détails du modèle reconstruit en 3D, la Kinect 2 qui estime la profondeur par temps de vol (ToF), est ensuite utilisée pour l'acquisition d'images. Cependant, en raison de réflections multiples sur les miroirs, il se produit une distorsion de la profondeur dans notre système. Nous proposons donc une approche simple pour réduire cette distorsion avant d'appliquer les techniques d'optique géométrique pour reconstruire un nuage de points de l'objet 3D. Pour l'analyse de la démarche, nous proposons diverses alternatives centrées sur la normalité de la marche et la mesure de sa symétrie. Cela devrait être utile lors de traitements cliniques pour évaluer, par exemple, la récupération du patient après une intervention chirurgicale. Ces méthodes se composent d'approches avec ou sans modèle qui ont des inconvénients et avantages différents. Dans cette thèse, nous présentons 3 méthodes qui traitent directement les nuages de points reconstruits dans la section précédente. La première utilise la corrélation croisée des demi-corps gauche et droit pour évaluer la symétrie de la démarche, tandis que les deux autres methodes utilisent des autoencodeurs issus de l'apprentissage profond pour mesurer la normalité de la démarche.The problem of assessing human gaits has received a great attention in the literature since gait analysis is one of key components in healthcare. Marker-based and multi-camera systems are widely employed to deal with this problem. However, such systems usually require specific equipments with high price and/or high computational cost. In order to reduce the cost of devices, we focus on a system of gait analysis which employs only one depth sensor. The principle of our work is similar to multi-camera systems, but the collection of cameras is replaced by one depth sensor and mirrors. Each mirror in our setup plays the role of a camera which captures the scene at a different viewpoint. Since we use only one camera, the step of synchronization can thus be avoided and the cost of devices is also reduced. Our studies can be separated into two categories: 3D reconstruction and gait analysis. The result of the former category is used as the input of the latter one. Our system for 3D reconstruction is built with a depth camera and two mirrors. Two types of depth sensor, which are distinguished based on the scheme of depth estimation, have been employed in our works. With the structured light (SL) technique integrated into the Kinect 1, we perform the 3D reconstruction based on geometrical optics. In order to increase the level of details of the 3D reconstructed model, the Kinect 2 with time-of-flight (ToF) depth measurement is used for image acquisition instead of the previous generation. However, due to multiple reflections on the mirrors, depth distortion occurs in our setup. We thus propose a simple approach for reducing such distortion before applying geometrical optics to reconstruct a point cloud of the 3D object. For the task of gait analysis, we propose various alternative approaches focusing on the problem of gait normality/symmetry measurement. They are expected to be useful for clinical treatments such as monitoring patient's recovery after surgery. These methods consist of model-free and model-based approaches that have different cons and pros. In this dissertation, we present 3 methods that directly process point clouds reconstructed from the previous work. The first one uses cross-correlation of left and right half-bodies to assess gait symmetry while the other ones employ deep auto-encoders to measure gait normality

Apprentissage de modèles probabilistes pour la vision stéréoscopique en temps réel

RÉSUMÉ Il existe de nombreuses approches pour capturer des scènes en trois dimensions, la plus couramment utilisée est d'avoir recours à la stéréoscopie, qu'elle soit active ou passive. Le principe sous-jacent à de telles techniques est toujours le même et consiste à retrouver des correspondances aux travers différentes prises de vues. La recherche de ces correspondances aboutit à la création de cartes de disparités. Nous présentons ici une étude de différentes approches, aussi bien passive qu'active, pour construire de telles cartes. Nous nous intéressons également aux modèles probabilistes qui permettent leur débruitage et l'amélioration des résultats obtenus. Enfin, nous proposons également une approche basée sur l'utilisation de modèles de base et la combinaison de différentes techniques de calcul de disparités pour construire notre propre modèle. Il existe deux critères d'évaluation pour les approches de numérisation tridimensionnelle : le temps de traitement et la qualité des captures. Contrairement à d'autres systèmes cherchant à optimiser seulement l'un de ces deux critères, les résultats des travaux présentés ici sont obtenus en concentrant nos efforts sur la qualité mais aussi sur le temps nécessaire aux calculs. Pour cette dernière raison, nous avons choisi d'utiliser des techniques de parallélisation massive sur processeur graphique (\ac{GPGPU}). Nous étudierons donc des implémentations parallélisées et optimisées pour traiter nos images stéréoscopiques ou nos cartes de disparités, tels que l'algorithme du Census ou de Viterbi, entièrement sur un processeur graphique. Enfin nous verrons comment combiner divers sources d'informations telles qu'une Kinect et une caméra stéréo pour obtenir la meilleure qualité de carte de disparités possible. Nous verrons également un montage optimisé pour la numérisation de visages et l'évaluation et la comparaison de nouveaux modèles.----------ABSTRACT Among the various existing strategies to capture 3D information out of a scene, the most commonly used is stereoscopy, either active or passive. Underneath theses strategies there is always the same principle of finding corresponding points across captured views. Dense corresponding points are used to generate disparity map that can then be transformed in a depth map. This shows the importance of finding good approaches to build theses disparity maps. We will explore several approaches both active and passive to do so. We also present various probabilistic models that allow to denoise data and improve the quality of our disparity maps. We will also introduce a new way of combining these models and various strategies of getting disparity to build our own new model. We present a probabilistic framework to compute disparity maps focusing on both quality, efficiency and speed. To reduce the time needed for computing our disparity maps we chose to use general purpose computing on graphics processing units techniques to massively parallelize our algorithms. Hence we will present some optimized parallelized implementation of algorithms to treat both stereoscopic images and disparity maps such as Census algorithm or Viterbi, such that all the processing is done on a GPU. Finally we show how to combine various sources by adding a Kinect to our model based on stereo camera to improve either the quality of our outputted disparity map or the time required by our algorithms. We will also present a mount optimized for face scanning and new models evaluation and comparison

Optimized projection pattern supplementing stereo systems

Abstract — Stereo camera systems are widely used in many real applications including indoor and outdoor robotics. They are very easy to use and provide accurate depth estimates on well-textured scenes, but often fail when the scene does not have enough texture. It is possible to help the system work better in this situation by actively projecting certain light patterns to the scene to create artificial texture on the scene surface. The question we try to answer in ths paper is what would be the best pattern(s) to project. This paper introduces optimized projection patterns based on a novel concept of (symmetric) non-recurring De Bruijn sequences, and describes algorithms to generate such sequences. A projected pattern creates an artificial texture which does not contain any duplicate patterns over epipolar lines within certain range, thus it makes the correspondence match simple and unique. The proposed patterns are compatible with most existing stereo algorithms, meaning that they can be used without any changes in the stereo algorithm and one can immediately get much denser depth estimates without any additional computational cost. It is also argued that the proposed patterns are optimal binary patterns, and finally a few experimental result using stereo and space-time stereo algorithms are presented. I

Fusion of LIDAR with stereo camera data - an assessment

This thesis explores data fusion of LIDAR (laser range-finding) with stereo matching, with a particular emphasis on close-range industrial 3D imaging. Recently there has been interest in improving the robustness of stereo matching using data fusion with active range data. These range data have typically been acquired using time of flight cameras (ToFCs), however ToFCs offer poor spatial resolution and are noisy. Comparatively little work has been performed using LIDAR. It is argued that stereo and LIDAR are complementary and there are numerous advantages to integrating LIDAR into stereo systems. For instance, camera calibration is a necessary prerequisite for stereo 3D reconstruction, but the process is often tedious and requires precise calibration targets. It is shown that a visible-beam LIDAR enables automatic, accurate (sub-pixel) extrinsic and intrinsic camera calibration without any explicit targets. Two methods for using LIDAR to assist dense disparity maps from featureless scenes were investigated. The first involved using a LIDAR to provide high-confidence seed points for a region growing stereo matching algorithm. It is shown that these seed points allow dense matching in scenes which fail to match using stereo alone. Secondly, LIDAR was used to provide artificial texture in featureless image regions. Texture was generated by combining real or simulated images of every point the laser hits to form a pseudo-random pattern. Machine learning was used to determine the image regions that are most likely to be stereo- matched, reducing the number of LIDAR points required. Results are compared to competing techniques such as laser speckle, data projection and diffractive optical elements