Estimation de la structure 3D d'un environnement urbain à partir d'un flux vidéo

In computer vision, the 3D structure estimation from 2D images remains a fundamental problem. One of the emergent applications is 3D urban modelling and mapping. Here, we are interested in street-level monocular 3D reconstruction from mobile vehicle. In this particular case, several challenges arise at different stages of the 3D reconstruction pipeline. Mainly, lacking textured areas in urban scenes produces low density reconstructed point cloud. Also, the continuous motion of the vehicle prevents having redundant views of the scene with short feature points lifetime. In this context, we adopt the piecewise planar 3D reconstruction where the planarity assumption overcomes the aforementioned challenges.In this thesis, we introduce several improvements to the 3D structure estimation pipeline. In particular, the planar piecewise scene representation and modelling. First, we propose a novel approach that aims at creating 3D geometry respecting superpixel segmentation, which is a gradient-based boundary probability estimation by fusing colour and flow information using weighted multi-layered model. A pixel-wise weighting is used in the fusion process which takes into account the uncertainty of the computed flow. This method produces non-constrained superpixels in terms of size and shape. For the applications that imply a constrained size superpixels, such as 3D reconstruction from an image sequence, we develop a flow based SLIC method to produce superpixels that are adapted to reconstructed points density for better planar structure fitting. This is achieved by the mean of new distance measure that takes into account an input density map, in addition to the flow and spatial information. To increase the density of the reconstructed point cloud used to performthe planar structure fitting, we propose a new approach that uses several matching methods and dense optical flow. A weighting scheme assigns a learned weight to each reconstructed point to control its impact to fitting the structure relative to the accuracy of the used matching method. Then, a weighted total least square model uses the reconstructed points and learned weights to fit a planar structure with the help of superpixel segmentation of the input image sequence. Moreover, themodel handles the occlusion boundaries between neighbouring scene patches to encourage connectivity and co-planarity to produce more realistic models. The final output is a complete dense visually appealing 3Dmodels. The validity of the proposed approaches has been substantiated by comprehensive experiments and comparisons with state-of-the-art methodsDans le domaine de la vision par ordinateur, l’estimation de la structure d’une scène 3D à partir d’images 2D constitue un problème fondamental. Parmi les applications concernées par cette problématique, nous nous sommes intéressés dans le cadre de cette thèse à la modélisation d’un environnement urbain. Nous nous sommes intéressés à la reconstruction de scènes 3D à partir d’images monoculaires générées par un véhicule en mouvement. Ici, plusieurs défis se posent à travers les différentes étapes de la chaine de traitement inhérente à la reconstruction 3D. L’un de ces défis vient du fait de l’absence de zones suffisamment texturées dans certaines scènes urbaines, d’où une reconstruction 3D (un nuage de points 3D) trop éparse. De plus, du fait du mouvement du véhicule, d’une image à l’autre il n’y a pas toujours un recouvrement suffisant entre différentes vues consécutives d’une même scène. Dans ce contexte, et ce afin de lever les verrous ci-dessus mentionnés, nous proposons d’estimer, de reconstruire, la structure d’une scène 3D par morceaux en se basant sur une hypothèse de planéité. Nous proposons plusieurs améliorations à la chaine de traitement associée à la reconstruction 3D. D’abord, afin de structurer, de représenter, la scène sous la forme d’entités planes nous proposons une nouvelle méthode de reconstruction 3D, basée sur le regroupement de pixels similaires (superpixel segmentation), qui à travers une représentation multi-échelle pondérée fusionne les informations de couleur et de mouvement. Cette méthode est basée sur l’estimation de la probabilité de discontinuités locales aux frontières des régions calculées à partir du gradient (gradientbased boundary probability estimation). Afin de prendre en compte l’incertitude liée à l’estimation du mouvement, une pondération par morceaux est appliquée à chaque pixel en fonction de cette incertitude. Cette méthode génère des regroupements de pixels (superpixels) non contraints en termes de taille et de forme. Pour certaines applications, telle que la reconstruction 3D à partir d’une séquence d’images, des contraintes de taille sont nécessaires. Nous avons donc proposé une méthode qui intègre à l’algorithme SLIC (Simple Linear Iterative Clustering) l’information de mouvement. L’objectif étant d’obtenir une reconstruction 3D plus dense qui estime mieux la structure de la scène. Pour atteindre cet objectif, nous avons aussi introduit une nouvelle distance qui, en complément de l’information de mouvement et de données images, prend en compte la densité du nuage de points. Afin d’augmenter la densité du nuage de points utilisé pour reconstruire la structure de la scène sous la forme de surfaces planes, nous proposons une nouvelle approche qui mixte plusieurs méthodes d’appariement et une méthode de flot optique dense. Cette méthode est basée sur un système de pondération qui attribue un poids pré-calculé par apprentissage à chaque point reconstruit. L’objectif est de contrôler l’impact de ce système de pondération, autrement dit la qualité de la reconstruction, en fonction de la précision de la méthode d’appariement utilisée. Pour atteindre cet objectif, nous avons appliqué un processus des moindres carrés pondérés aux données reconstruites pondérées par les calculés par apprentissage, qui en complément de la segmentation par morceaux de la séquence d’images, permet une meilleure reconstruction de la structure de la scène sous la forme de surfaces planes. Nous avons également proposé un processus de gestion des discontinuités locales aux frontières de régions voisines dues à des occlusions (occlusion boundaries) qui favorise la coplanarité et la connectivité des régions connexes. L’ensemble des modèles proposés permet de générer une reconstruction 3D dense représentative à la réalité de la scène. La pertinence des modèles proposés a été étudiée et comparée à l’état de l’art. Plusieurs expérimentations ont été réalisées afin de démontrer, d’étayer, la validité de notre approch

Fusion of Dense Spatial Features and Sparse Temporal Features for 3D Structure Estimation in Urban Scenes

International audienceThe authors present a novel approach to improve three-dimensional (3D) structure estimation from an image stream in urban scenes. The authors consider a particular setup, where the camera is installed on a moving vehicle. Applying traditional structure from motion (SfM) technique in this case generates poor estimation of the 3D structure because of several reasons such as texture-less images, small baseline variations and dominant forward camera motion. The authors idea is to introduce the monocular depth cues that exist in a single image, and add time constraints on the estimated 3D structure. The scene is modelled as a set of small planar patches obtained using over-segmentation, and the goal is to estimate the 3D positioning of these planes. The authors propose a fusion scheme that employs Markov random field model to integrate spatial and temporal depth features. Spatial depth is obtained by learning a set of global and local image features. Temporal depth is obtained via sparse optical flow based SfM approach. That allows decreasing the estimation ambiguity by forcing some constraints on camera motion. Finally, the authors apply a fusion scheme to create unique 3D structure estimatio

Joint Spatio-temporal Depth Features Fusion Framework for 3D Structure Estimation in Urban Environment

International audienceWe present a novel approach to improve 3D structure estimation from an image stream in urban scenes. We consider a particular setup where the camera is installed on a moving vehicle. Applying traditional structure from motion (SfM) technique in this case generates poor estimation of the 3d structure due to several reasons such as texture-less images, small baseline variations and dominant forward camera motion. Our idea is to introduce the monocular depth cues that exist in a single image, and add time constraints on the estimated 3D structure. We assume that our scene is made up of small planar patches which are obtained using over-segmentation method, and our goal is to estimate the 3D positioning for each of these planes. We propose a fusion framework that employs Markov Random Field (MRF) model to integrate both spatial and temporal depth information. An advantage of our model is that it performs well even in the absence of some depth information. Spatial depth information is obtained through a global and local feature extraction method inspired by Saxena et al. [1]. Temporal depth information is obtained via sparse optical flow based structure from motion approach. That allows decreasing the estimation ambiguity by forcing some constraints on camera motion. Finally, we apply a fusion scheme to create unique 3D structure estimatio

Monocular 3D structure estimation for urban scenes

International audienc

Replica Update Strategy in Mobile Ad Hoc Networks

International audienceIn mobile ad hoc networks, partitioning occurs frequently. Data replication techniques are used to improve data accessibility but require data consistency to be maintained in case of update. In this paper, we propose hybrid push-pull data update propagation. The idea is to divide replica holders into SH(Push) and LL(Pull) categories. Updates are pushed to SH nodes whenever they occur. LL nodes pull the updates from SH nodes in a frequency suitable for their needs. The novelty of this method that it minimizes communication cost when saving an adapted level –to mobile hosts needs- for data consistenc

Inferring linear and nonlinear Interaction networks using neighborhood support vector machines

International audienc

Color and Flow Based Superpixels for 3D Geometry Respecting Meshing

International audienc

Fast Visual Odometry for a Low-Cost Underwater Embedded Stereo System †

This paper provides details of hardware and software conception and realization of a stereo embedded system for underwater imaging. The system provides several functions that facilitate underwater surveys and run smoothly in real-time. A first post-image acquisition module provides direct visual feedback on the quality of the taken images which helps appropriate actions to be taken regarding movement speed and lighting conditions. Our main contribution is a light visual odometry method adapted to the underwater context. The proposed method uses the captured stereo image stream to provide real-time navigation and a site coverage map which is necessary to conduct a complete underwater survey. The visual odometry uses a stochastic pose representation and semi-global optimization approach to handle large sites and provides long-term autonomy, whereas a novel stereo matching approach adapted to underwater imaging and system attached lighting allows fast processing and suitability to low computational resource systems. The system is tested in a real context and shows its robustness and promising future potential

Underwater photogrammetry, coded target and plenoptic technology: A Set of tools for monitoring red coral in mediterranean sea in the framework of the "perfect" project

3D Virtual Reconstruction and Visualization of Complex Architectures, 1-3 March 2017, Nafplio, Greece.-- 8 pages, 14 figuresPErfECT "Photogrammetry, gEnetic, Ecology for red coral ConservaTion" is a project leaded by the Laboratoire des Sciences de lInformation et des Systmes (LSIS - UMR 7296 CNRS) from the Aix-Marseille University (France) in collaboration with the Spanish National Agency for Scientific Research (CSIC, Spain). The main objective of the project is to develop innovative Tools for the conservation of the Mediterranean red coral, Corallium rubrum. PErfECT was funded by the Total Fundation. The adaptation of digital photogrammetric techniques for use in submarine is rapidly increasing in recent years. In fact, these techniques are particularly well suited for use in underwater environments. PErfECT developed different photogrammetry tools to enhance the red coral population surveys based in: (i) automatic orientation on coded quadrats, (ii) use of NPR (Non Photo realistic Rendering) techniques, (iii) the calculation of distances between colonies within local populations and finally (iv) the use of plenoptic approaches in underwater conditionsThis work is partially done in the framework of the PERfECT project, funded by the Foundation TOTAL, project 2014/257Peer Reviewe

Étude comparative de réseaux de neurones pour la reconnaissance des émotions avec les images plénoptiques

International audienc