A probabilistic integrated object recognition and tracking framework for video sequences

Recognition and tracking of multiple objects in video sequences is one of the main challenges in computer vision that currently deserves a lot of attention from researchers. Almost all the reported approaches are very application-dependent and there is a lack of a general methodology for dynamic object recognition and tracking that can be instantiated in particular cases. In this thesis, the work is oriented towards the definition and development of such a methodology which integrates object recognition and tracking from a general perspective using a probabilistic framework called PIORT (probabilistic integrated object recognition and tracking framework). It include some modules for which a variety of techniques and methods can be applied. Some of them are well-known but other methods have been designed, implemented and tested during the development of this thesis.The first step in the proposed framework is a static recognition module that provides class probabilities for each pixel of the image from a set of local features. These probabilities are updated dynamically and supplied to a tracking decision module capable of handling full and partial occlusions. The two specific methods presented use RGB colour features and differ in the classifier implemented: one is a Bayesian method based on maximum likelihood and the other one is based on a neural network. The experimental results obtained have shown that, on one hand, the neural net based approach performs similarly and sometimes better than the Bayesian approach when they are integrated within the tracking framework. And on the other hand, our PIORT methods have achieved better results when compared to other published tracking methods. All these methods have been tested experimentally in several test video sequences taken with still and moving cameras and including full and partial occlusions of the tracked object in indoor and outdoor scenarios in a variety of cases with different levels of task complexity. This allowed the evaluation of the general methodology and the alternative methods that compose these modules.A Probabilistic Integrated Object Recognition and Tracking Framework for Video SequencesEl reconocimiento y seguimiento de múltiples objetos en secuencias de vídeo es uno de los principales desafíos en visión por ordenador que actualmente merece mucha atención de los investigadores. Casi todos los enfoques reportados son muy dependientes de la aplicación y hay carencia de una metodología general para el reconocimiento y seguimiento dinámico de objetos, que pueda ser instanciada en casos particulares. En esta tesis, el trabajo esta orientado hacia la definición y desarrollo de tal metodología, la cual integra reconocimiento y seguimiento de objetos desde una perspectiva general usando un marco probabilístico de trabajo llamado PIORT (Probabilistic Integrated Object Recognition and Tracking). Este incluye algunos módulos para los que se puede aplicar una variedad de técnicas y métodos. Algunos de ellos son bien conocidos, pero otros métodos han sido diseñados, implementados y probados durante el desarrollo de esta tesis.El primer paso en el marco de trabajo propuesto es un módulo estático de reconocimiento que provee probabilidades de clase para cada píxel de la imagen desde un conjunto de características locales. Estas probabilidades son actualizadas dinámicamente y suministradas a un modulo decisión de seguimiento capaz de manejar oclusiones parciales o totales. Se presenta dos métodos específicos usando características de color RGB pero diferentes en la implementación del clasificador: uno es un método Bayesiano basado en la máxima verosimilitud y el otro método está basado en una red neuronal. Los resultados experimentales obtenidos han mostrado que, por una parte, el enfoque basado en la red neuronal funciona similarmente y algunas veces mejor que el enfoque bayesiano cuando son integrados dentro del marco probabilístico de seguimiento. Por otra parte, nuestro método PIORT ha alcanzado mejores resultados comparando con otros métodos de seguimiento publicados. Todos estos métodos han sido probados experimentalmente en varias secuencias de vídeo tomadas con cámaras fijas y móviles incluyendo oclusiones parciales y totales del objeto a seguir, en ambientes interiores y exteriores, en diferentes tareas y niveles de complejidad. Esto ha permitido evaluar tanto la metodología general como los métodos alternativos que componen sus módulos

Aplicação de técnicas de análise de imagem para sistemas de engenharia química

Orientador : Prof. Dr. Marcelo Kaminski LenziDissertação (mestrado) - Universidade Federal do Paraná, Setor de Tecnologia, Programa de Pós-Graduação em Engenharia Química. Defesa: Curitiba, 17/02/2011Bibliografia: fls.77-81Área de concentração: Modelagem, simulação, Otimização e Controle de ProcessosResumo: Este trabalho propõe a aplicação da análise de imagem a sistemas de engenharia química, visando o monitoramento de processos e controle de qualidade. O resultado do trabalho desenvolvido contempla um sistema de análise de imagem digital com três aplicações especificas, envolvendo técnicas de aquisição e processamento das imagens adquiridas. A aquisição de imagem foi realizada dinamicamente, utilizando uma simples webcam que interage com o programa, e este realiza o processamento das imagens por meio de algoritmos de comparação simples pixel a pixel, baseado no modelo RGB de cores. A primeira aplicação corresponde à determinação de viscosidade da glicerina pelo método de Stokes; a segunda ao controle do nível de líquido em tanques; e a terceira aplicação a expansão da espuma de poliuretano durante sua formação. Para cada aplicação são apresentados resultados satisfatórios, de forma a concluir que o sistema desenvolvido é uma eficaz ferramenta para o monitoramento e controle de qualidade.Abstract: This study proposes the application of the image analysis for chemical engineering systems, aimed at process monitoring and quality control. The result of the work system comprises a digital image analysis with three specific applications involving techniques of acquisition and processing of acquired images. The image acquisition was performed dynamically using a simple webcam that interacts with the program, and it performs the image processing algorithms through a simple comparison pixel by pixel, based on the RGB color model. The first application one is the determination of viscosity of the glycerin by the method of Stokes, the second to control the liquid level in tanks and the third application is the expansion of polyurethane foam during its formation. For each application the satisfactory results are presented, to conclude that the system is an effective tool for monitoring and quality control

Low and Variable Frame Rate Face Tracking Using an IP PTZ Camera

RÉSUMÉ En vision par ordinateur, le suivi d'objets avec des caméras PTZ a des applications dans divers domaines, tels que la surveillance vidéo, la surveillance du trafic, la surveillance de personnes et la reconnaissance de visage. Toutefois, un suivi plus précis, efficace, et fiable est requis pour une utilisation courante dans ces domaines. Dans cette thèse, le suivi est appliqué au haut du corps d'un humain, en incluant son visage. Le suivi du visage permet de déterminer son emplacement pour chaque trame d'une vidéo. Il peut être utilisé pour obtenir des images du visage d'un humain dans des poses différentes. Dans ce travail, nous proposons de suivre le visage d'un humain à l’aide d'une caméra IP PTZ (caméra réseau orientable). Une caméra IP PTZ répond à une commande via son serveur Web intégré et permet un accès distribué à partir d'Internet. Le suivi avec ce type de caméra inclut un bon nombre de défis, tels que des temps de réponse irrégulier aux commandes de contrôle, des taux de trame faibles et irréguliers, de grand mouvements de la cible entre deux trames, des occlusions, des modifications au champ de vue, des changements d'échelle, etc. Dans notre travail, nous souhaitons solutionner les problèmes des grands mouvements de la cible entre deux trames consécutives, du faible taux de trame, des modifications de l'arrière-plan, et du suivi avec divers changements d'échelle. En outre, l'algorithme de suivi doit prévoir les temps de réponse irréguliers de la caméra. Notre solution se compose d’une phase d’initialisation pour modéliser la cible (haut du corps), d’une adaptation du filtre de particules qui utilise le flux optique pour générer des échantillons à chaque trame (APF-OFS), et du contrôle de la caméra. Chaque composante exige des stratégies différentes. Lors de l'initialisation, on suppose que la caméra est statique. Ainsi, la détection du mouvement par soustraction d’arrière-plan est utilisée pour détecter l'emplacement initial de la personne. Ensuite, pour supprimer les faux positifs, un classificateur Bayesien est appliqué sur la région détectée afin de localiser les régions avec de la peau. Ensuite, une détection du visage basée sur la méthode de Viola et Jones est effectuée sur les régions de la peau. Si un visage est détecté, le suivi est lancé sur le haut du corps de la personne.----------ABSTRACT Object tracking with PTZ cameras has various applications in different computer vision topics such as video surveillance, traffic monitoring, people monitoring and face recognition. Accurate, efficient, and reliable tracking is required for this task. Here, object tracking is applied to human upper body tracking and face tracking. Face tracking determines the location of the human face for each input image of a video. It can be used to get images of the face of a human target under different poses. We propose to track the human face by means of an Internet Protocol (IP) Pan-Tilt-Zoom (PTZ) camera (i.e. a network-based camera that pans, tilts and zooms). An IP PTZ camera responds to command via its integrated web server. It allows a distributed access from Internet (access from everywhere, but with non-defined delay). Tracking with such camera includes many challenges such as irregular response times to camera control commands, low and irregular frame rate, large motions of the target between two frames, target occlusion, changing field of view (FOV), various scale changes, etc. In our work, we want to cope with the problem of large inter-frame motion of targets, low usable frame rate, background changes, and tracking with various scale changes. In addition, the tracking algorithm should handle the camera response time and zooming. Our solution consists of a system initialization phase which is the processing before camera motion and a tracker based on an Adaptive Particle Filter using Optical Flow based Sampling (APF-OFS) tracker, and camera control that are the processing after the motion of the camera. Each part requires different strategies. For initialization, when the camera is stationary, motion detection for a static camera is used to detect the initial location of the person face entering an area. For motion detection in the FOV of the camera, a background subtraction method is applied. Then to remove false positives, Bayesian skin classifier is applied on the detected motion region to discriminate skin regions from non skin regions. Face detection based on Viola and Jones face detector can be performed on the detected skin regions independently of their face size and position within the image

Cell-Based Deformation Monitoring via 3D Point Clouds

Deformation is one of the most important phenomena in environmental science and engineering. Deformation of artificial and natural objects happens worldwide, such as structural deformation, landslide, subsidence, erosion, and rockfall. Monitoring and assessment of such deformation process is not only scientifically interesting, but also beneficial to hazard/risk control and prediction. In addition, it is also useful for regional planning and development. Deformation monitoring was driven by geodetic observations in the field of traditional geodetic surveying, based on the measurement of sparse points in a control network. Recently, with the rapid development of terrestrial LiDAR techniques, millions of points with associated three-dimensional coordinates (known as "3D point clouds") can be promptly captured in a few minutes. Compared to traditional surveying, terrestrial LiDAR offers great potential for deformation monitoring, because of various advantages such as fast data capture, high data density, and precise 3D object representation. By analysing 3D point clouds, the objective of this thesis is to provide an effective and efficient approach for deformation monitoring. Towards this goal, this thesis designs a new concept of "deformation map" for deformation representation and a novel "cell-based approach" for deformation computation. The main outcome of this thesis is a novel and rich approach that is able to automatically and incrementally compute a deformation map that enables a better understanding of structural and natural hazards with heterogeneous deformation characteristics. This work includes several dedicated contributions as follows. Hybrid Deformation Modelling. This thesis firstly provides a comprehensive investigation on the modelling requirements of various deformation phenomena. The requirements concern three main aspects, i.e., what has deformation (deformation object), which type of deformation, and how to describe deformation. Based on this detailed requirement analysis, we propose a rich and hybrid deformation model. This model is composed of meta-deformation, sub-deformation and deformation map, corresponding to deformation for a small cell, for a partial area, and for the whole object, respectively. Cell-based Deformation Computation. In order to automatically and incrementally extract heterogeneous deformation of the whole monitored object, we bring the "cell" concept into deformation monitoring. This thesis builds a cell-based deformation computing framework, which consists of three key steps: split, detect, and merge. Split is to divide the space of the object into many cells (uniform or irregular); detect is to extract the meta-deformation for individual cells by analysing the inside point clouds at two epochs; and merge is to group adjacent cells with similar deformation together and to form a consistent sub-deformation. As the final result, an informative deformation map is computed for describing the deformation for the whole object. Evaluation of Cell-based Approach. To evaluate such hybrid modelling and cell-based deformation computation, this thesis extensively studies both synthetic and real-life point cloud datasets: (1) by imitating a landslide scenario, we generate synthetic data using Matlab programming and practical settings, and compare the cell-based approach with traditional non-cell based geodetic methods; (2) by analysing two real-life cases of deformation in Switzerland, we further validate our approach and compare the results with third party sources (e.g., results provided by a surveying company, results computed by using a commercial software like 3DReshaper). Extension of Cell-based Approach. At the last stages of this thesis work, we particularly focus on providing several technical extensions to enhance this cell-based deformation monitoring approach. The main extensions include: (1) supporting dynamic cells instead of uniform cells when splitting the entire object space, (2) finding cell correspondence for the deformation scenarios that have large deformation like rockfalls, (3) movement tracking with data-driven cells which have irregular cell shape that can be automatically determined by the deformation boundary itself, (4) designing an adaptive modelling strategy that is able to accordingly select a suitable model for detecting meta-deformation of cells, and (5) computing deformation evolution for a monitored object with more than two epochs of point cloud datasets