Visual words for 3D reconstruction and pose computation

International audienceVisual vocabularies are standard tools in the object/image classification literature, and are emerging as a new tool for building point correspondences for pose estimation. This paper proposes several visual word based methods for point matching, with structure from motion and pose estimation applications in view. The three dimensional geometry of a scene is first extracted with bundle adjustment techniques based on the keypoint correspondences. These correspondences are obtained by grouping the set of all SIFT descriptors from the training images into visual words. We obtain a more accurate 3D geometry than with classical image-to-image point matching. In the second step, these visual words serve as 3D point descriptors robust to viewpoint change, and are then used for building 2D-3D correspondences for a test image, yielding the pose of the camera by solving the PnP problem. We compare several visual word formation techniques w.r.t robustness to viewpoint change between the learning and test images and discuss the required computational time

Viewpoint simulation for camera pose estimation from an unstructured scene model

International audienceWe consider the problem of camera pose estimation from a scene model obtained beforehand by a structure-from-motion (SfM) algorithm. The model is made of 3D points, each one of them being represented by its coordinates and a set of photometric descriptors such as SIFT, extracted from some of the input images of the SfM stage. Pose estimation is based on the matching of interest points from a test view with model points, using the descriptors. Descriptors having a limited invariance with respect to viewpoint changes, such an approach is likely to fail when the test view is far away from the images used to construct the model. Viewpoint simulation techniques, as ASIFT, have proved effective for wide-baseline image matching. This paper explores how these techniques can enrich a scene model by adding descriptors from simulated views, using either orthographic or pinhole virtual cameras. Viewpoint simulation permits pose estimation in situations where the approach based on the sole SIFT descriptors simply fails

Simulation de point de vue pour la mise en correspondance et la localisation.

National audienceWe consider the problem of camera pose estimation from a scene model obtained beforehand by a structure from-motion (SfM) algorithm. The model is made of 3D points, each one of them being represented by its coordinates and a set of photometric descriptors such as SIFT, extracted from some of the input images of the SfM stage. Pose estimation is based on the matching of interest points from a test view with model points, using the descriptors. Descriptors having a limited invariance with respect to viewpoint changes, such an approach is likely to fail when the test view is far away from the images used to construct the model. Viewpoint simulation techniques, as ASIFT, have proved effective for wide-baseline image matching. This paper explores how these techniques can enrich a scene model by adding descriptors from simulated views, and evaluate the respective benefits of affine and homographic simulations. In particular we show that viewpoint simulation increases the proportion of correct correspondances, and permits pose estimation in situations where the approach based on the sole SIFT descriptors simply fails.On considère le problème de la localisation d'une caméra à partir d'un modèle non structuré obtenu par un algorithme de type structure from motion. Dans ce modèle, un point est représenté par ses coordonnées et un ensemble de descripteurs photométriques issus des images dans lesquelles il est observé. La localisation repose sur l'appariement de points d'intérêt de la vue courante avec des points du modèle, sur la base des descripteurs. Cependant le manque d'invariance des descripteurs aux changements de point de vue rend difficile la mise en correspondance dès que la vue courante est éloignée des images ayant servi à construire le modèle. Les techniques de simulation de point de vue, comme ASIFT, ont récemment montré leur intérêt pour la mise en correspondance entre images. Cet article explore l'apport de ces techniques pour enrichir le modèle initial par des descripteurs simulés et évalue le bénéfice respectif de simulations affines et homographiques. Nous montrons en particulier que la simulation augmente la proportion de bons appariements et la précision du calcul de pose et permet de calculer une pose là où l'approche basée uniquement sur les descripteurs SIFT échoue