Reconstruction 3D et localisation simultanée de caméras mobiles : une approche temps-réel par ajustement de faisceaux local

The Structure from Motion problem is an intense research topic in computer vision and has been the subject of much investigation. This thesis presents a method for estimating the motion of a calibrated camera and the threedimensional geometry of the filmed environment. The main idea is to take advantage of both offline methods (based on an optimization of all 3D parameters by global bundle adjustment) and fast incremental methods. The new approach may be seen as an acceleration of conventional 3D reconstruction techniques that make use of bundle adjustment, and thus enables to treat very long video sequences. The introduced algorithm may be summarized as follows : interest points detection and matching between frames, subsampling of the video into "key frames", full 3D reconstruction of these key frames (3D points and camera poses), and localization of all frames. The keystone of the method is the local bundle adjustment : reconstruction parameters are refined at the end of the sequence only, for all current frame selected as key frame. This method is applied initially to a perspective camera model, then extended to a generic camera model to describe most existing kinds of cameras like catadioptric cameras or stereo rigs. Experiments have shown that results are very similar to those obtained by methods with global optimisation, with much lower computing times. We can envisage applications like realtime visual odometry for mobile robots or car assisted driving.Le problème de la reconstruction 3D à partir d'une séquence d'images acquise par une caméra en mouvement est un sujet important dans le domaine de la vision par ordinateur. Ce travail de thèse présente une méthode qui permet d'estimer conjointement des points 3D de la scène filmée et le mouvement de la caméra en combinant la précision des méthodes "horsligne" (basées sur une optimisation globale de tous les paramètres par ajustement de faisceaux) et la vitesse de calcul des méthodes incrémentales. La nouvelle approche est considérée comme une accélération des techniques classiques de reconstruction 3D qui utilisent l'ajustement de faisceaux, permettant ainsi de traiter de longues séquences vidéos. L'algorithme développé peut être résumé de la façon suivante : détection de points d'intérêt dans les images, mise en correspondance de ces points et souséchantillonnage temporel de la vidéo. En effet, seul un sousensemble d'images dites "images clef" est sélectionné pour la reconstruction des points 3D alors que la localisation de la caméra est calculée pour chaque image. Le point clef de l'approche est l'ajustement de faisceaux local : les paramètres de la reconstruction sont affinés sur la fin de la séquence uniquement, à chaque fois qu'une image est choisie comme nouvelle image clef. La méthode, initialement prévue pour les caméras perspectives, a ensuite été généralisée de manière à rendre possible l'utilisation d'autres types de caméras, comme les caméras catadioptriques ou encore les paires rigides de caméras. Les résultats obtenus montrent que la précision atteinte est du même ordre que celle des méthodes par optimisation globale, avec des temps de calcul très réduits, ce qui permet de viser des applications d'odométrie visuelle temps réel pour la robotique mobile ou l'aide à la conduite en automobile. Realtim

3D Reconstruction of Complex Structures with Bundle Adjustment: an Incremental Approach

This paper introduces an incremental method for "Structure From Motion" of complex scenes from a video sequence. More precisely, we estimate the 3D positions of the viewed points in images and the camera positions and orientations through the sequence. The method can be seen as a fast but accurate alternative to classical reconstruction methods that use bundle adjustment, and that can become slow and computation time expensive for very long scenes. Our results are compared to the reconstruction obtained by the classical hierarchical bundle adjustment method. They have also been successfully used as a reference sequence for the vision based localization of an autonomous mobile robot

Real-Time Localization and 3D Reconstruction

In this paper we describe a method that estimates the motion of a calibrated camera (settled on an experimental vehicle) and the tridimensional geometry of the environment. The only data used is a video input. In fact, interest points are tracked and matched between frames at video rate. Robust estimates of the camera motion are computed in real-time, key-frames are selected and permit the features 3D reconstruction. The algorithm is particularly appropriate to the reconstruction of long images sequences thanks to the introduction of a fast and local bundle adjustment method that ensures both good accuracy and consistency of the estimated camera poses along the sequence. It also largely reduces computational complexity compared to a global bundle adjustment. Experiments on real data were carried out to evaluate speed and robustness of the method for a sequence of about one kilometer long. Results are also compared to the ground truth measured with a differential GPS

Mixed Reality and Remote Sensing Application of Unmanned Aerial Vehicle in Fire and Smoke Detection

This paper proposes the development of a system incorporating inertial measurement unit (IMU), a consumer-grade digital camera and a fire detection algorithm simultaneously with a nano Unmanned Aerial Vehicle (UAV) for inspection purposes. The video streams are collected through the monocular camera and navigation relied on the state-of-the-art indoor/outdoor Simultaneous Localisation and Mapping (SLAM) system. It implements the robotic operating system (ROS) and computer vision algorithm to provide a robust, accurate and unique inter-frame motion estimation. The collected onboard data are communicated to the ground station and used the SLAM system to generate a map of the environment. A robust and efficient re-localization was performed to recover from tracking failure, motion blur, and frame lost in the data received. The fire detection algorithm was deployed based on the colour, movement attributes, temporal variation of fire intensity and its accumulation around a point. The cumulative time derivative matrix was utilized to analyze the frame-by-frame changes and to detect areas with high-frequency luminance flicker (random characteristic). Colour, surface coarseness, boundary roughness, and skewness features were perceived as the quadrotor flew autonomously within the clutter and congested area. Mixed Reality system was adopted to visualize and test the proposed system in a physical environment, and the virtual simulation was conducted through the Unity game engine. The results showed that the UAV could successfully detect fire and flame, autonomously fly towards and hover around it, communicate with the ground station and simultaneously generate a map of the environment. There was a slight error between the real and virtual UAV calibration due to the ground truth data and the correlation complexity of tracking real and virtual camera coordinate frames

Determining Ship Position In A Harbour Based On Omnidirectional Image Of The Coastline

The following article presents researches aimed at the evaluation of precision in determining a ship’s position through comparing a omnidirectional map image to a real vision image of the coast line. The first part establishes the thesis and preparatory forms in conducting the research. It also presents designed and built equipment including a research tool software. A system equipped with a spherical catadioptric camera that aids data collection on board ship designated to processing and analyzing data collected on board in connection with the spherical images of an electronic navigational chart with a software module. The second part explains procedures followed in conducting the research. The foreword note explains the procedure in data collection aboard a ship maneuvering in the port after which the algorithm for position placement and precise parametrical count was presented. The concluding part shows analyses of obtained research result. It bears a performance on the evaluation of precision at determining position. As a measure, an average error value and distance fluctuation of obtained position from referential position. As our conclusion, primary agents having rudimentary influence on the quality of correlating spherical map image to coastline visual image were characterized

Generic and Real Time Structure from Motion using Local Bundle Adjustment

This paper describes a method for estimating the motion of a calibrated camera and the three dimensional geometry of the filmed environment. The only data used is video input. Interest points are tracked and matched between frames at video rate. Robust estimates of the camera motion are computed in real-time, key frames are selected to enable 3D reconstruction of the features. We introduce a local bundle adjustment allowing 3D points and camera poses to be refined simultaneously through the sequence. This significantly reduces computational complexity when compared with global bundle adjustment. This method is applied initially to a perspective camera model, then extended to a generic camera model to describe most existing kinds of cameras. Experiments performed using real world data provide evaluations of the speed and robustness of the method. Results are compared to the ground truth measured with a differential GPS. The generalized method is also evaluated experimentally, using three types of calibrated cameras: stereo rig, perspective and catadioptric

Generic and Real-Time Structure from Motion

International audienceWe introduce a generic and incremental Structure from Motion method. By generic, we mean that the proposed method is independent of any specific camera model. During the incremental 3D reconstruction, parameters of 3D points and camera poses are refined simultaneously by a generic local bundle adjustment that minimizes an angular error between rays. This method has three main advantages: it is generic, fast and accurate. The proposed method is evaluated by experiments on real data with three kinds of calibrated cameras: stereo rig, perspective and catadioptric cameras