3D Model Assisted Image Segmentation

The problem of segmenting a given image into coherent regions is important in Computer Vision and many industrial applications require segmenting a known object into its components. Examples include identifying individual parts of a component for proces

3D Model Assisted Image Segmentation

The problem of segmenting a given image into coherent regions is important in Computer Vision and many industrial applications require segmenting a known object into its components. Examples include identifying individual parts of a component for process control work in a manufacturing plant and identifying parts of a car from a photo for automatic damage detection. Unfortunately most of an object’s parts of interest in such applications share the same pixel characteristics, having similar colour and texture. This makes segmenting the object into its components a non-trivial task for conventional image segmentation algorithms. In this paper, we propose a “Model Assisted Segmentation ” method to tackle this problem. A 3D model of the object is registered over the given image by optimising a novel gradient based loss function. This registration obtains the full 3D pose from an image of the object. The image can have an arbitrary view of the object and is not limited to a particular set of views. The segmentation is subsequently performed using a level-set based method, using the projected contours of the registered 3D model as initialisation curves. The method is fully automatic and requires no user interaction. Also, the system does not require any prior training. We present our results on photographs of a real car

3D pose estimation by directly matching polyhedral models to gray value gradients

This contribution addresses the problem of pose estimation and tracking of vehicles in image sequences from traffic scenes recorded by a stationary camera. In a new algorithm, the vehicle pose is estimated by directly matching polyhedral vehicle models to image gradients without an edge segment extraction process. The new approach is significantly more robust than approaches that rely on feature extraction since the new approach exploits more information from the image data. We successfully tracked vehicles that were partially occluded by textured objects, e.g. foliage, where a previous approach based on edge segment extraction failed. Moreover, the new pose estimation approach is also used to determine the orientation and position of the road relative to the camera by matching an intersection model directly to image gradients. Results from various experiments with real world traffic scenes are presented

Efficient tracking of 3D objects from appearance

In this article, we propose an efficient tracking algorithm to follow 3D objects in image sequences. 3D objects are represented by a collection of reference images. The originality of this method is not to use high-level primitives (points of interest) to follow the movement of the object in the image but rather the difference between the vectors of gray-levels of the tracked reference pattern and the current pattern sampled in an area of interest. The tracking problem is reduced then to the estimate of the parameters representing the possible movements of the object in the image by the determination of interaction matrices learned during an off-line training stage, and that for each reference view. The first one relates the variations of intensity of the 2D current pattern to be tracked to its fronto parallel movement (parallel movement to the image plane). The aspect of the pattern representing the tracked object is not modified by this movement. However, the position, the orientation and the size of the pattern can change. The second matrix relates the variations of appearance of the currently tracked pattern to a change of attitude between the object and the camera (modification of the angular values in rolling and pitching). We show that the on-line use of these interaction matrices for the correction of the predicted position of the object in the image and the estimate of the variations of aspect of the tracked pattern allows a real time implementation of this algorithm (a matrix multiplied by a vector). Moreover, we also show how the problem of occlusions can be managed.Dans cet article, nous proposons un algorithme efficace de suivi d'un objet 3D dans une séquence d'images. Pour cela, l'objet 3D est représenté par une collection d'images de référence. L'originalité de cette méthode est de ne pas utiliser des primitives de haut niveau (points d'intérêt) pour suivre le déplacement de l'objet dans l'image mais plutôt la différence de vecteurs de niveaux de gris entre le motif de référence suivi et le motif courant échantillonné dans une zone d'intérêt de l'image. Le problème du suivi se ramène alors à l'estimation des paramètres qui caractérisent les mouvements possibles de l'objet dans l'image par la détermination de matrices dites d'interaction apprises lors d'une phase d'apprentissage hors ligne, et cela pour chacune des vues de référence. La première matrice lie les variations d'intensité lumineuse du motif de référence 2D de l'objet suivi à son déplacement fronto parallèle (déplacement paralièle au plan image). Sous l'hypothèse d'un tel mouvement, l'aspect apparent de l'objet suivi n'est pas modifié. Toutefois, sa position, son orientation planaire et sa taille peuvent changer. La deuxième matrice relie les variations d'apparence du motif suivi suite à un changement d'orientation par rapport au capteur (modification des angles de site et d'azimut). Nous montrons que l'utilisation en ligne de ces matrices pour la correction de la position prédite de l'objet dans l'image et de l'estimation des variations d'aspect du motif suivi correspond à un coût algorithmique très faible (multiplication d'une matrice par un vecteur) permettant une mise en oeuvre temps réel. De plus, nous évoquons le problème des occultations lors du suivi par une méthode de seuillage adaptatif

Object tracking using variational optic flow methods

We propose an algorithm for tracking of objects in video sequences by computing a spatiotemporal optical flow field, based on the method of Brox et al., and the application of a spatiotemporal watershed segmentation algorithm with region merging on the previously obtained vector field.Es wird ein Algorithmus zum Verfolgen von Objekten in Videosequenzen durch die Berechnung eines zeitlich-räumlichen optischen Flussfeldes präsentiert, basierend auf der Methode von Brox et al., und der darauffolgenden Anwendung eines zeitlich-räumlichen Wasserscheiden-Segmentierungsalgorithmus mit Region Merging auf dem durch den opti- schen Fluss erhaltenen Vektorfeld

Object tracking using variational optic flow methods

We propose an algorithm for tracking of objects in video sequences by computing a spatiotemporal optical flow field, based on the method of Brox et al., and the application of a spatiotemporal watershed segmentation algorithm with region merging on the previously obtained vector field.Es wird ein Algorithmus zum Verfolgen von Objekten in Videosequenzen durch die Berechnung eines zeitlich-räumlichen optischen Flussfeldes präsentiert, basierend auf der Methode von Brox et al., und der darauffolgenden Anwendung eines zeitlich-räumlichen Wasserscheiden-Segmentierungsalgorithmus mit Region Merging auf dem durch den opti- schen Fluss erhaltenen Vektorfeld