Automatic Image Registration in Infrared-Visible Videos using Polygon Vertices

In this paper, an automatic method is proposed to perform image registration in visible and infrared pair of video sequences for multiple targets. In multimodal image analysis like image fusion systems, color and IR sensors are placed close to each other and capture a same scene simultaneously, but the videos are not properly aligned by default because of different fields of view, image capturing information, working principle and other camera specifications. Because the scenes are usually not planar, alignment needs to be performed continuously by extracting relevant common information. In this paper, we approximate the shape of the targets by polygons and use affine transformation for aligning the two video sequences. After background subtraction, keypoints on the contour of the foreground blobs are detected using DCE (Discrete Curve Evolution)technique. These keypoints are then described by the local shape at each point of the obtained polygon. The keypoints are matched based on the convexity of polygon's vertices and Euclidean distance between them. Only good matches for each local shape polygon in a frame, are kept. To achieve a global affine transformation that maximises the overlapping of infrared and visible foreground pixels, the matched keypoints of each local shape polygon are stored temporally in a buffer for a few number of frames. The matrix is evaluated at each frame using the temporal buffer and the best matrix is selected, based on an overlapping ratio criterion. Our experimental results demonstrate that this method can provide highly accurate registered images and that we outperform a previous related method

A Multiscale Region-Based Motion Detection and Background Subtraction Algorithm

This paper presents a region-based method for background subtraction. It relies on color histograms, texture information, and successive division of candidate rectangular image regions to model the background and detect motion. Our proposed algorithm uses this principle and combines it with Gaussian Mixture background modeling to produce a new method which outperforms the classic Gaussian Mixture background subtraction method. Our method has the advantages of filtering noise during image differentiation and providing a selectable level of detail for the contour of the moving shapes. The algorithm is tested on various video sequences and is shown to outperform state-of-the-art background subtraction methods

An algorithm for multiple object tracking

Background for multiple object tracking -- Data association -- The model of object

Appariement de points caractéristiques trouvés à même les régions d'avant-plan de vidéos à spectres visible et infrarouge

Systèmes de caméras calibrés -- Extraction des points caractéristiques -- Appariement des points caractéristiques -- Filtrage des paires de points caractéristiques -- Filtrage des paires de points caractéristiques -- Le projet de référence -- Aperçu de la méthode -- Prétraitements -- La méthode du squelette -- Le processus DCE -- Filtrage par paires de blobs -- Filtrage par RANSAC -- Calcul des disparités -- Complexité des méthodes

Recalage automatique de séquences vidéo infrarouge et visible basé sur les trajectoires des objets en mouvement

RÉSUMÉ Le projet présenté dans ce document porte sur une nouvelle méthode pour effectuer le recalage entre deux séquences vidéo de spectres différents soit l’une du spectre visible et l’autre du spectre infrarouge. Un recalage consiste à retrouver la matrice de transformation qui permet de passer d’un référentiel à un autre. Dans ce projet en particulier, il s’agit de trouver la matrice de transformation affine qui permet de passer du référentiel de la caméra infrarouge à celui de la caméra couleur. Ce recalage est effectué pour chaque trame. Pour y arriver, la méthode proposée se base sur les trajectoires des objets en mouvement ainsi que sur des combinaisons d’images d’avantplan en recouvrement. Plusieurs méthodes de recalage multispectral ont été proposées dans la littérature. Cependant, il n’est pas toujours possible d’appliquer ces techniques dans le cas spécifique d’un recalage entre des images couleur et infrarouge. Il existe cependant une méthode qui semble avoir un bon potentiel. Il s’agit de la méthode de Caspi, Simakov, & Irani (2006). Cette méthode utilise uniquement les trajectoires des objets en mouvement pour retrouver la matrice de transformation entre les deux caméras. Cet article fut la principale source d’inspiration de ce mémoire.----------ABSTRACT This project is about a new registration method between a color video and an infrared video. Registration is to find the transformation matrix from which you can pass from one set of coordinates to another, for example, between two cameras. In this project, this transformation matrix is an affine transformation matrix. This registration is done frame by frame. It uses two kinds of information: the trajectories of moving objects and an overlapped picture made from combinations of foreground pictures. Multiple multispectral registration methods were found in the literature but not all of these methods can be used for the specific case of an infrared-color registration. Even so, a specific method (Caspi et al., 2006) seems promising to achieve this kind of registration. This method uses only trajectories of moving objects to find the transformation matrix. This article is the principal inspiration of this work

A practical vision system for the detection of moving objects

The main goal of this thesis is to review and offer robust and efficient algorithms for the detection (or the segmentation) of foreground objects in indoor and outdoor scenes using colour image sequences captured by a stationary camera. For this purpose, the block diagram of a simple vision system is offered in Chapter 2. First this block diagram gives the idea of a precise order of blocks and their tasks, which should be performed to detect moving foreground objects. Second, a check mark () on the top right corner of a block indicates that this thesis contains a review of the most recent algorithms and/or some relevant research about it. In many computer vision applications, segmenting and extraction of moving objects in video sequences is an essential task. Background subtraction has been widely used for this purpose as the first step. In this work, a review of the efficiency of a number of important background subtraction and modelling algorithms, along with their major features, are presented. In addition, two background approaches are offered. The first approach is a Pixel-based technique whereas the second one works at object level. For each approach, three algorithms are presented. They are called Selective Update Using Non-Foreground Pixels of the Input Image , Selective Update Using Temporal Averaging and Selective Update Using Temporal Median , respectively in this thesis. The first approach has some deficiencies, which makes it incapable to produce a correct dynamic background. Three methods of the second approach use an invariant colour filter and a suitable motion tracking technique, which selectively exclude foreground objects (or blobs) from the background frames. The difference between the three algorithms of the second approach is in updating process of the background pixels. It is shown that the Selective Update Using Temporal Median method produces the correct background image for each input frame. Representing foreground regions using their boundaries is also an important task. Thus, an appropriate RLE contour tracing algorithm has been implemented for this purpose. However, after the thresholding process, the boundaries of foreground regions often have jagged appearances. Thus, foreground regions may not correctly be recognised reliably due to their corrupted boundaries. A very efficient boundary smoothing method based on the RLE data is proposed in Chapter 7. It just smoothes the external and internal boundaries of foreground objects and does not distort the silhouettes of foreground objects. As a result, it is very fast and does not blur the image. Finally, the goal of this thesis has been presenting simple, practical and efficient algorithms with little constraints which can run in real time

Low and Variable Frame Rate Face Tracking Using an IP PTZ Camera

RÉSUMÉ En vision par ordinateur, le suivi d'objets avec des caméras PTZ a des applications dans divers domaines, tels que la surveillance vidéo, la surveillance du trafic, la surveillance de personnes et la reconnaissance de visage. Toutefois, un suivi plus précis, efficace, et fiable est requis pour une utilisation courante dans ces domaines. Dans cette thèse, le suivi est appliqué au haut du corps d'un humain, en incluant son visage. Le suivi du visage permet de déterminer son emplacement pour chaque trame d'une vidéo. Il peut être utilisé pour obtenir des images du visage d'un humain dans des poses différentes. Dans ce travail, nous proposons de suivre le visage d'un humain à l’aide d'une caméra IP PTZ (caméra réseau orientable). Une caméra IP PTZ répond à une commande via son serveur Web intégré et permet un accès distribué à partir d'Internet. Le suivi avec ce type de caméra inclut un bon nombre de défis, tels que des temps de réponse irrégulier aux commandes de contrôle, des taux de trame faibles et irréguliers, de grand mouvements de la cible entre deux trames, des occlusions, des modifications au champ de vue, des changements d'échelle, etc. Dans notre travail, nous souhaitons solutionner les problèmes des grands mouvements de la cible entre deux trames consécutives, du faible taux de trame, des modifications de l'arrière-plan, et du suivi avec divers changements d'échelle. En outre, l'algorithme de suivi doit prévoir les temps de réponse irréguliers de la caméra. Notre solution se compose d’une phase d’initialisation pour modéliser la cible (haut du corps), d’une adaptation du filtre de particules qui utilise le flux optique pour générer des échantillons à chaque trame (APF-OFS), et du contrôle de la caméra. Chaque composante exige des stratégies différentes. Lors de l'initialisation, on suppose que la caméra est statique. Ainsi, la détection du mouvement par soustraction d’arrière-plan est utilisée pour détecter l'emplacement initial de la personne. Ensuite, pour supprimer les faux positifs, un classificateur Bayesien est appliqué sur la région détectée afin de localiser les régions avec de la peau. Ensuite, une détection du visage basée sur la méthode de Viola et Jones est effectuée sur les régions de la peau. Si un visage est détecté, le suivi est lancé sur le haut du corps de la personne.----------ABSTRACT Object tracking with PTZ cameras has various applications in different computer vision topics such as video surveillance, traffic monitoring, people monitoring and face recognition. Accurate, efficient, and reliable tracking is required for this task. Here, object tracking is applied to human upper body tracking and face tracking. Face tracking determines the location of the human face for each input image of a video. It can be used to get images of the face of a human target under different poses. We propose to track the human face by means of an Internet Protocol (IP) Pan-Tilt-Zoom (PTZ) camera (i.e. a network-based camera that pans, tilts and zooms). An IP PTZ camera responds to command via its integrated web server. It allows a distributed access from Internet (access from everywhere, but with non-defined delay). Tracking with such camera includes many challenges such as irregular response times to camera control commands, low and irregular frame rate, large motions of the target between two frames, target occlusion, changing field of view (FOV), various scale changes, etc. In our work, we want to cope with the problem of large inter-frame motion of targets, low usable frame rate, background changes, and tracking with various scale changes. In addition, the tracking algorithm should handle the camera response time and zooming. Our solution consists of a system initialization phase which is the processing before camera motion and a tracker based on an Adaptive Particle Filter using Optical Flow based Sampling (APF-OFS) tracker, and camera control that are the processing after the motion of the camera. Each part requires different strategies. For initialization, when the camera is stationary, motion detection for a static camera is used to detect the initial location of the person face entering an area. For motion detection in the FOV of the camera, a background subtraction method is applied. Then to remove false positives, Bayesian skin classifier is applied on the detected motion region to discriminate skin regions from non skin regions. Face detection based on Viola and Jones face detector can be performed on the detected skin regions independently of their face size and position within the image

A new classification approach based on geometrical model for human detection in images

In recent years, object detection and classification has gained more attention, thus, there are several human object detection algorithms being used to locate and recognize human objects in images. The research of image processing and analysing based on human shape is a hot topic due to its wide applicability in real applications. In this research, we present a new shape-based classification approach to categorise the detected object as human or non-human in images. The classification in this approach is based on applying a geometrical model which contains a set of parameters related to the object’s upper portion. Based on the result of these geometric parameters, our approach can simply classify the detected object as human or non-human. In general, the classification process of this new approach is based on generating a geometrical model by observing unique geometrical relations between the upper portion shape points (neck, head, shoulders) of humans, this observation is based on analysis of the change in the histogram of the x values coordinates for human upper portion shape. To present the changing of X coordinate values we have used histograms with mathematical smoothing functions to avoid small angles, as the result we observed four parameters for human objects to be used in building the classifier, by applying the four parameters of the geometrical model and based on the four parameters results, our classification approach can classify the human object from another object. The proposed approach has been tested and compared with some of the machine learning approaches such as Artificial Neural Networks (ANN), Support Vector Machine (SVM) Model, and a famous type of decision tree called Random Forest, by using 358 different images for several objects obtained from INRIA dataset (set of human and non-human as an object in digital images). From the comparison and testing result between the proposed approach and the machine learning approaches in term of accuracy performance, we indicate that the proposed approach achieved the highest accuracy rate (93.85%), with the lowest miss detection rate (11.245%) and false discovery rate (9.34%). The result achieved from the testing and comparison shows the efficiency of this presented approach