Augmented particle filtering for efficient visual tracking

Copyright © 2005 IEEEVisual tracking is one of the key tasks in computer vision. The particle filter algorithm has been extensively used to tackle this problem due to its flexibility. However the conventional particle filter uses system transition as the proposal distribution, frequently resulting in poor priors for the filtering step. The main reason is that it is difficult, if not impossible, to accurately model the target's motion. Such a proposal distribution does not take into account the current observations. It is not a trivial task to devise a satisfactory proposal distribution for the particle filter. In this paper we advance a general augmented particle filtering framework for designing the optimal proposal distribution. The essential idea is to augment a second filter's estimate into the proposal distribution design. We then show that several existing improved particle filters can be rationalised within this general framework. Based on this framework we further propose variant algorithms for robust and efficient visual tracking. Experiments indicate that the augmented particle filters are more efficient and robust than the conventional particle filter.Chunhua Shen Brooks, M.J. van den Hengel, A

Histograma de orientación de gradientes aplicado al seguimiento múltiple de personas basado en video

El seguimiento múltiple de personas en escenas reales es un tema muy importante en el campo de Visión Computacional dada sus múltiples aplicaciones en áreas como en los sistemas de vigilancia, robótica, seguridad peatonal, marketing, etc., además de los retos inherentes que representa la identificación de personas en escenas reales como son la complejidad de la escena misma, la concurrencia de personas y la presencia de oclusiones dentro del video debido a dicha concurrencia. Existen diversas técnicas que abordan el problema de la segmentación de imágenes y en particular la identificación de personas, desde diversas perspectivas; por su parte el presente trabajo tiene por finalidad desarrollar una propuesta basada en Histograma de Orientación de Gradientes (HOG) para el seguimiento múltiple de personas basado en video. El procedimiento propuesto se descompone en las siguientes etapas: Procesamiento de Video, este proceso consiste en la captura de los frames que componen la secuencia de video, para este propósito se usa la librería OpenCV de tal manera que se pueda capturar la secuencia desde cualquier fuente; la siguiente etapa es la Clasificación de Candidatos, esta etapa se agrupa el proceso de descripción de nuestro objeto, que para el caso de este trabajo son personas y la selección de los candidatos, para esto se hace uso de la implementación del algoritmo de HOG; por último la etapa final es el Seguimiento y Asociación, mediante el uso del algoritmo de Kalman Filter, permite determinar las asociaciones de las secuencias de objetos previamente detectados. La propuesta se aplicó sobre tres conjuntos de datos, tales son: TownCentre (960x540px), TownCentre (1920x1080px) y PETS 2009, obteniéndose los resultados para precisión: 94.47%, 90.63% y 97.30% respectivamente. Los resultados obtenidos durante las experimentaciones validan la propuesta del modelo haciendo de esta una herramienta que puede encontrar múltiples campos de aplicación, además de ser una propuesta innovadora a nivel nacional dentro del campo de Vision Computacional.Tesi

Particle Filters for Colour-Based Face Tracking Under Varying Illumination

Automatic human face tracking is the basis of robotic and active vision systems used for facial feature analysis, automatic surveillance, video conferencing, intelligent transportation, human-computer interaction and many other applications. Superior human face tracking will allow future safety surveillance systems which monitor drowsy drivers, or patients and elderly people at the risk of seizure or sudden falls and will perform with lower risk of failure in unexpected situations. This area has actively been researched in the current literature in an attempt to make automatic face trackers more stable in challenging real-world environments. To detect faces in video sequences, features like colour, texture, intensity, shape or motion is used. Among these feature colour has been the most popular, because of its insensitivity to orientation and size changes and fast process-ability. The challenge of colour-based face trackers, however, has been dealing with the instability of trackers in case of colour changes due to the drastic variation in environmental illumination. Probabilistic tracking and the employment of particle filters as powerful Bayesian stochastic estimators, on the other hand, is increasing in the visual tracking field thanks to their ability to handle multi-modal distributions in cluttered scenes. Traditional particle filters utilize transition prior as importance sampling function, but this can result in poor posterior sampling. The objective of this research is to investigate and propose stable face tracker capable of dealing with challenges like rapid and random motion of head, scale changes when people are moving closer or further from the camera, motion of multiple people with close skin tones in the vicinity of the model person, presence of clutter and occlusion of face. The main focus has been on investigating an efficient method to address the sensitivity of the colour-based trackers in case of gradual or drastic illumination variations. The particle filter is used to overcome the instability of face trackers due to nonlinear and random head motions. To increase the traditional particle filter\u27s sampling efficiency an improved version of the particle filter is introduced that considers the latest measurements. This improved particle filter employs a new colour-based bottom-up approach that leads particles to generate an effective proposal distribution. The colour-based bottom-up approach is a classification technique for fast skin colour segmentation. This method is independent to distribution shape and does not require excessive memory storage or exhaustive prior training. Finally, to address the adaptability of the colour-based face tracker to illumination changes, an original likelihood model is proposed based of spatial rank information that considers both the illumination invariant colour ordering of a face\u27s pixels in an image or video frame and the spatial interaction between them. The original contribution of this work lies in the unique mixture of existing and proposed components to improve colour-base recognition and tracking of faces in complex scenes, especially where drastic illumination changes occur. Experimental results of the final version of the proposed face tracker, which combines the methods developed, are provided in the last chapter of this manuscript

Estimating the location with angle measurements

Tässä diplomityössä käsitellään paikannusta saapumiskulmamittausten avulla. Työssä esitellään yleisimpiä suodattimia, robusteja suodattimia ulkolaisten tarkasteluun ja niiden algoritmit. Suodatuksen taustateorian lisäksi työssä esitellään erilaisia jakaumia mittausten ja kohinan mallintamista varten. Työ keskittyy kulmamittauksiin ja parhaan mahdollisen suodattimien ja jakaumien yhdistelmän löytämiseen sisätilapaikannuksessa. Suodatusta voidaan parantaa sovelluksilla, jotka huomioivat tilan rajoitteet, mittauksen signaalin voimakkuuden tai tilariippuvan virheen. Suodattimia testataan simuloiduilla ja todellisilla kulmamittauksilla. Tosidata on mitattu TTY:n CivitLab laboratoriossa OptiTrack sovelluksen avulla. Suodattimien paikannustuloksia verrataan keskenään. Suodattimien tilamalleina testataan vakionopeusmallia ja pelkkään paikkavektoriin perustuvaa mallia. Yleinen tapa esittää suuntamittaus on leveys- ja korkeussuunnan kulmamittaukset, joiden kohina on normaalijakautunut. Esitys ei useinkaan vastaa todellisuutta maapallon navoilla, jossa korkeusmittaus on nolla astetta. Monesti realistisempi mittausmalli saadaan, kun kohinaa kuvataan von Mises—Fisher-jakaumalla. Tällöin kulmamittauksiin liittyvä suunta ilmaistaan Eulerin kulmia käyttävällä von Mises—Fisher-jakauman yksikkövektorilla, jonka suunta perustuu mittausten keskiarvoiseen suuntaan. Kulmien kääntämisestä johtuen paras mahdollinen suodattimen ja jakauman yhdistelmä paikan estimointiin on suodatin, joka olettaa von Mises—Fisher-jakautuneen mittauskohinan sisältävän datan. Mittausdatan sisältäessä ulkolaismittauksia von Mises—Fisher-jakautuneen mittauskohinan olettavat robustit suodattimet antavat paremman paikkaestimaatin kuin ei-robustit suodattimet.In this thesis positioning is explored with angle of arrival measurements. Common filters in positioning, robust filters for detecting outliers and their algorithms are introduced. In addition to filtering theory different distributions to model measurements and noise presented. In this thesis, the focus is on angle of arrival measurements and finding the best combination of filter and distribution suitable for indoor positioning. Filtering can be improved with applications which take restrictions of the space, signal strength or space dependent error into account. Filters are tested and compared to each other with simulated and real angle measurements. Real data is measured with OptiTrack application in CivitLab laboratory in Tampere University of Technology. Positioning results filters produce are compared with each other. Filters are tested with different state models, the constant velocity model and model containing position vector. Common way to present directional measurement is to use azimuth and elevation angle measuments with normally distributed noise. This does not always correspond to reality when used to describe measurements in Poles where the elevation measurement is zero degrees. More realistic model is achieved with using von Mises—Fisher distribution to describe the noise. The direction related to angle of arrival measurements can be expressed with the unit vector of von Mises—Fisher distribution using Euler angles. Direction of the vector is based on the mean direction. Due to flipping the angles the best possible combination of filter and distribution to estimate the place is a filter which assumes measurement data containing von Mises—Fisher distributed measurement noise. Robust filters assuming von Mises—Fisher distributed measurement noise is the best choice when measurements contain outliers

Ανάδραση θέσης μικροβελόνας με χρήση βιντεομικροσκοπίου σε πραγματικό χρόνο

127 σ.Μεταπτυχιακή Εργασία -- Εθνικό Μετσόβιο Πολυτεχνείο. Διεπιστημονικό - Διατμηματικό Πρόγραμμα Μεταπτυχιακών Σπουδών "Μικροσυσήματα και Νανοδιατάξεις"Η παρούσα εργασία έγινε στα πλαίσια της ερευνητικής δραστηριότητας που αναπτύσσεται στο Εργαστήριο Αυτομάτου Ελέγχου της Σχολής Μηχανολόγων Μηχανικών του ΕΜΠ σχετικά με μικρορομποτικά συστήματα. Αναλύεται η ανάδραση θέσης μικροβελόνας η οποία είναι προσαρμοσμένη σε ένα μικρορομποτικό σύστημα. Η πληροφορία ανακτάται μέσω οπτικού μικροσκοπίου και βιντεοκάμερας τοποθετημένης επάνω από το χώρο δράσης του πειράματος. Η εργασία πραγματεύεται θέματα τεχνολογίας λήψης εικόνας με οπτικό μικροσκόπιο, τεχνολογίας οπτικών αισθητήρων, ψηφιακής επεξεργασίας εικόνας και λειτουργικών συστημάτων πραγματικού χρόνου. Το οπτικό μικροσκόπιο που χρησιμοποιείται έχει τη δυνατότητα μεταβολής οπτικής μεγέθυνσης (zoom) και εστίασης. Στη μία άκρη του οπτικού σωλήνα υπάρχει ισχυρός αντικειμενικός φακός ενώ στην άλλη προσαρμόστηκε η βιντεοκάμερα. Το μικροσκόπιο μεγεθύνει την άκρη της μικροβελόνας και στη συνέχεια συλλαμβάνεται η κίνησή της από τη βιντεοκάμερα. Τα στιγμιότυπα στέλνονται σε ηλεκτρονικό υπολογιστή όπου με κατάλληλους αλγόριθμους επεξεργασίας εικόνας προσδιορίζεται σε πραγματικό χρόνο η θέση και η διεύθυνση. Για να γίνει αυτό θα πρέπει η μικροβελόνα να είναι διακριτή σε σχέση με το υπόβαθρο, δηλαδή να υπάρχει εμφανής διαχωριστική γραμμή. Για να το επιτύχουμε αυτό χρωματίσαμε την μικροβελόνα με μαύρο χρώμα οπότε καταφέραμε να έχουμε μεγάλη αντίθεση ανάμεσα σε αυτή και στο φωτισμένο υπόβαθρο. Η μικροβελόνα απέχει από το επίπεδο κίνησης του μικρορομπότ περίπου 2 cm. Η απόσταση αυτή είναι ικανοποιητική έτσι ώστε το υπόβαθρο να παρουσιάζει ενιαία χαρακτηριστικά φωτός χωρίς να προσφέρει θόρυβο από τις ατέλειες της επιφάνειάς του. Για την εύρεση θέσης και διεύθυνσης της μικροβελόνας γίνεται επεξεργασία της εικόνας ανά στιγμιότυπο. Κομβικό σημείο στην επεξεργασία είναι η επιλογή κατάλληλης τεχνικής για την εξαγωγή της ζητούμενης πληροφορίας. Δοκιμάστηκαν διαφορετικές τεχνικές αλλά τελικά επιλέχθηκε ο μετασχηματισμός Hough (κυκλικός και γραμμικός) με την βοήθεια του οποίου ανιχνεύονται κύκλοι και γραμμές. Με κατάλληλη επεξεργασία των εξαγόμενων κύκλων και γραμμών υπολογίζουμε τη θέση και τη διεύθυνση της μικροβελόνας. Η εφαρμογή επιλέχθηκε να αναπτυχθεί σε περιβάλλον simulink, για την απλοποίησή της σχεδίασης, αλλά αυτό δημιούργησε προβλήματα σχετικά με την υλοποίηση της σε πραγματικό χρόνο, τα οποία αναλύονται στη συνέχεια. Η σχεδίαση του αλγόριθμου επεξεργασίας της εικόνας παράγει ικανοποιητικά αποτελέσματα (σε πραγματικό χρόνο).The present work became in the frames of inquiring activity that are developed in the Laboratory of Automatic Control of Faculty of Mechanical Engineering of NTUA with regard to microrobotic systems. Is analyzed the position and direction feedback of a micro-needle which is adapted in a microrobotic system. The information is recovered via optical microscope and video camera placed above the space of action of experiment. The work is discussed subjects of technology of reception of picture with optical microscope, technology of optical sensors, digital treatment of picture and real time operating systems. The optical microscope that is used has the possibility of change of optical enlargement (zoom) and focus. In one end of the optical pipe exists powerful objective lens while in the other was adapted the video camera. The microscope enlarges the edge of the micro-needle and afterwards is arrested her movement from the video camera. The snapshots are sent in computer where with suitable algorithms of treatment of picture is determined in real time the position and the direction. In order to becomes this it will be supposed that micro-needle is distinguishable in combination the background, that is to say exists obvious bisector line. In order to achieve this we colored micro-needle with black colour therefore we accomplished to have big opposition between this and in the litted up background. Micro-needle abstains from the level of movement of the microrobot roughly 2 cm. This distance is satisfactory so the background presents single characteristics of light without it offers noise from the imperfections of his surface. For the finding of position and direction of micro-needle becomes treatment of picture per snapshot. Nodal point in the treatment is the choice of suitable technique for the export of asked information. We're finally tried different techniques but were selected the Hough transformation (circular and linear) with the help of which are detected circles and lines. With suitable treatment of exported circles and lines we calculate the position and the direction of micro-needle. The application was selected is developed in environment simulink, for her simplification of designing, but this created problems with regard to her realization in real time, which are analyzed afterwards. The designing of algorithm of treatment of picture produces satisfactory results (in real time).Ιωάννης Δ. Τσουκαράκη

Hardware acceleration of the trace transform for vision applications

Computer Vision is a rapidly developing field in which machines process visual data to extract meaningful information. Digitised images in their pixels and bits serve no purpose of their own. It is only by interpreting the data, and extracting higher level information that a scene can be understood. The algorithms that enable this process are often complex, and data-intensive, limiting the processing rate when implemented in software. Hardware-accelerated implementations provide a significant performance boost that can enable real- time processing. The Trace Transform is a newly proposed algorithm that has been proven effective in image categorisation and recognition tasks. It is flexibly defined allowing the mathematical details to be tailored to the target application. However, it is highly computationally intensive, which limits its applications. Modern heterogeneous FPGAs provide an ideal platform for accelerating the Trace transform for real-time performance, while also allowing an element of flexibility, which highly suits the generality of the Trace transform. This thesis details the implementation of an extensible Trace transform architecture for vision applications, before extending this architecture to a full flexible platform suited to the exploration of Trace transform applications. As part of the work presented, a general set of architectures for large-windowed median and weighted median filters are presented as required for a number of Trace transform implementations. Finally an acceleration of Pseudo 2-Dimensional Hidden Markov Model decoding, usable in a person detection system, is presented. Such a system can be used to extract frames of interest from a video sequence, to be subsequently processed by the Trace transform. All these architectures emphasise the need for considered, platform-driven design in achieving maximum performance through hardware acceleration

Two Case Studies on Vision-based Moving Objects Measurement

In this thesis, we presented two case studies on vision-based moving objects measurement. In the first case, we used a monocular camera to perform ego-motion estimation for a robot in an urban area. We developed the algorithm based on vertical line features such as vertical edges of buildings and poles in an urban area, because vertical lines are easy to be extracted, insensitive to lighting conditions/shadows, and sensitive to camera/robot movements on the ground plane. We derived an incremental estimation algorithm based on the vertical line pairs. We analyzed how errors are introduced and propagated in the continuous estimation process by deriving the closed form representation of covariance matrix. Then, we formulated the minimum variance ego-motion estimation problem into a convex optimization problem, and solved the problem with the interior-point method. The algorithm was extensively tested in physical experiments and compared with two popular methods. Our estimation results consistently outperformed the two counterparts in robustness, speed, and accuracy. In the second case, we used a camera-mirror system to measure the swimming motion of a live fish and the extracted motion data was used to drive animation of fish behavior. The camera-mirror system captured three orthogonal views of the fish. We also built a virtual fish model to assist the measurement of the real fish. The fish model has a four-link spinal cord and meshes attached to the spinal cord. We projected the fish model into three orthogonal views and matched the projected views with the real views captured by the camera. Then, we maximized the overlapping area of the fish in the projected views and the real views. The maximization result gave us the position, orientation, and body bending angle for the fish model that was used for the fish movement measurement. Part of this algorithm is still under construction and will be updated in the future

Visual tracking: detecting and mapping occlusion and camouflage using process-behaviour charts

Visual tracking aims to identify a target object in each frame of an image sequence. It presents an important scientific problem since the human visual system is capable of tracking moving objects in a wide variety of situations. Artificial visual tracking systems also find practical application in areas such as visual surveillance, robotics, biomedical image analysis, medicine and the media. However, automatic visual tracking algorithms suffer from two common problems: occlusion and camouflage. Occlusion arises when another object, usually with different features, comes between the camera and the target. Camouflage occurs when an object with similar features lies behind the target and makes the target invisible from the camera’s point of view. Either of these disruptive events can cause a tracker to lose its target and fail. This thesis focuses on the detection of occlusion and camouflage in a particle-filter based tracking algorithm. Particle filters are commonly used in tracking. Each particle represents a single hypothesis as to the target’s state, with some probability of being correct. The collection of particles tracking a target in each frame of an image sequence is called a particle set. The configuration of that particle set provides vital information about the state of the tracker. The work detailed in this thesis presents three innovative approaches to detecting occlusion and/or camouflage during tracking by evaluating the fluctuating behaviours of the particle set and detecting anomalies using a graphical statistical tool called a process-behaviour chart. The information produced by the process-behaviour chart is then used to map out the boundary of the interfering object, providing valuable information about the viewed environment. A method based on the medial axis of a novel representation of particle distribution termed the Particle History Image was found to perform best over a set of real and artificial test sequences, detecting 90% of occlusion and 100% of camouflage events. Key advantages of the method over previous work in the area are: (1) it is less sensitive to false data and less likely to fire prematurely; (2) it provides a better representation of particle set behaviour by aggregating particles over a longer time period and (3) the use of a training set to parameterise the process-behaviour charts means that comparisons are being made between measurements that are both made over extended time periods, improving reliability