ThirdLight: low-cost and high-speed 3D interaction using photosensor markers

We present a low-cost 3D tracking system for virtual reality, gesture modeling, and robot manipulation applications which require fast and precise localization of headsets, data gloves, props, or controllers. Our system removes the need for cameras or projectors for sensing, and instead uses cheap LEDs and printed masks for illumination, and low-cost photosensitive markers. The illumination device transmits a spatiotemporal pattern as a series of binary Gray-code patterns. Multiple illumination devices can be combined to localize each marker in 3D at high speed (333Hz). Our method has strengths in accuracy, speed, cost, ambient performance, large working space (1m-5m) and robustness to noise compared with conventional techniques. We compare with a state-of-the-art instrumented glove and vision-based systems to demonstrate the accuracy, scalability, and robustness of our approach. We propose a fast and accurate method for hand gesture modeling using an inverse kinematics approach with the six photosensitive markers. We additionally propose a passive markers system and demonstrate various interaction scenarios as practical applications

Real-time 3-D Reconstruction by Means of Structured Light Illumination

Structured light illumination (SLI) is the process of projecting a series of light striped patterns such that, when viewed at an angle, a digital camera can reconstruct a 3-D model of a target object\u27s surface. But by relying on a series of time multiplexed patterns, SLI is not typically associated with video applications. For this purpose of acquiring 3-D video, a common SLI technique is to drive the projector/camera pair at very high frame rates such that any object\u27s motion is small over the pattern set. But at these high frame rates, the speed at which the incoming video can be processed becomes an issue. So much so that many video-based SLI systems record camera frames to memory and then apply off-line processing. In order to overcome this processing bottleneck and produce 3-D point clouds in real-time, we present a lookup-table (LUT) based solution that in our experiments, using a 640 by 480 video stream, can generate intermediate phase data at 1063.8 frames per second and full 3-D coordinate point clouds at 228.3 frames per second. These achievements are 25 and 10 times faster than previously reported studies. At the same time, a novel dual-frequency pattern is developed which combines a high-frequency sinusoid component with a unit-frequency sinusoid component, where the high-frequency component is used to generate robust phase information and the unit-frequency component is used to reduce phase unwrapping ambiguities. Finally, we developed a gamma model for SLI, which can correct the non-linear distortion caused by the optical devices. For three-step phase measuring profilometry (PMP), analysis of the root mean squared error of the corrected phase showed a 60х reduction in phase error when the gamma calibration is performed versus 33х reduction without calibration

Fusion de données multi-capteurs pour la construction incrémentale du modèle tridimensionnel texturé d'un environnement intérieur par un robot mobile

Ce travail traite la Modélisation 3D d'un environnement intérieur par un robot mobile. La principale contribution concerne la construction d'un modèle géométrique hétérogène combinant des amers plans texturés, des lignes 3D et des points d'intérêt. Pour cela, nous devons fusionner des données géométriques et photométriques. Ainsi, nous avons d'abord amélioré la stéréovision, en proposant une approche de la mise en correspondance stéréoscopique par coupure de graphe. Notre contribution réside dans la construction d'un graphe réduit qui a permis d'accélérer la méthode globale et d'obtenir de meilleurs résultats que les méthodes locales. Aussi, pour percevoir l'environnement, le robot est équipé d'un télémètre laser 3D et d'une caméra. Nous proposons une chaîne algorithmique permettant de construire une carte hétérogène, par l'algorithme de Cartographie et Localisation Simultanées (EKF-SLAM). Le placage de la texture sur les facettes planes a permis de solidifier l'association de données.This thesis examines the problem of 3D Modelling of indoor environment by a mobile robot. Our main contribution consists in constructing a heterogeneous geometrical model containing textured planar landmarks, 3D lines and interest points. For that, we must fuse geometrical and photometrical data. Hence, we began by improving the stereo vision algorithm, and proposed a new approach of stereo matching by graph cuts. The most significant contribution is the construction of a reduced graph that allows to accelerate the global method and to provide better results than the local methods. Also, to perceive the environment, the robot is equipped by a 3D laser scanner and by a camera. We proposed an algorithmic chain allowing to incrementally constructing a heterogeneous map, using the algorithm of Simultaneous Localization and Mapping based (EKF-SLAM). Mapping the texture on the planar landmarks makes more robust the phase of data association