Robust automatic target tracking based on a Bayesian ego-motion compensation framework for airborne FLIR imagery

Automatic target tracking in airborne FLIR imagery is currently a challenge due to the camera ego-motion. This phenomenon distorts the spatio-temporal correlation of the video sequence, which dramatically reduces the tracking performance. Several works address this problem using ego-motion compensation strategies. They use a deterministic approach to compensate the camera motion assuming a specific model of geometric transformation. However, in real sequences a specific geometric transformation can not accurately describe the camera ego-motion for the whole sequence, and as consequence of this, the performance of the tracking stage can significantly decrease, even completely fail. The optimum transformation for each pair of consecutive frames depends on the relative depth of the elements that compose the scene, and their degree of texturization. In this work, a novel Particle Filter framework is proposed to efficiently manage several hypothesis of geometric transformations: Euclidean, affine, and projective. Each type of transformation is used to compute candidate locations of the object in the current frame. Then, each candidate is evaluated by the measurement model of the Particle Filter using the appearance information. This approach is able to adapt to different camera ego-motion conditions, and thus to satisfactorily perform the tracking. The proposed strategy has been tested on the AMCOM FLIR dataset, showing a high efficiency in the tracking of different types of targets in real working conditions

Robust Tracking in Aerial Imagery Based on an Ego-Motion Bayesian Model

A novel strategy for object tracking in aerial imagery is presented, which is able to deal with complex situations where the camera ego-motion cannot be reliably estimated due to the aperture problem (related to low structured scenes), the strong ego-motion, and/or the presence of independent moving objects. The proposed algorithm is based on a complex modeling of the dynamic information, which simulates both the object and the camera dynamics to predict the putative object locations. In this model, the camera dynamics is probabilistically formulated as a weighted set of affine transformations that represent possible camera ego-motions. This dynamic model is used in a Particle Filter framework to distinguish the actual object location among the multiple candidates, that result from complex cluttered backgrounds, and the presence of several moving objects. The proposed strategy has been tested with the aerial FLIR AMCOM dataset, and its performance has been also compared with other tracking techniques to demonstrate its efficiency

Face pose estimation in monocular images

People use orientation of their faces to convey rich, inter-personal information. For example, a person will direct his face to indicate who the intended target of the conversation is. Similarly in a conversation, face orientation is a non-verbal cue to listener when to switch role and start speaking, and a nod indicates that a person has understands, or agrees with, what is being said. Further more, face pose estimation plays an important role in human-computer interaction, virtual reality applications, human behaviour analysis, pose-independent face recognition, driver s vigilance assessment, gaze estimation, etc. Robust face recognition has been a focus of research in computer vision community for more than two decades. Although substantial research has been done and numerous methods have been proposed for face recognition, there remain challenges in this field. One of these is face recognition under varying poses and that is why face pose estimation is still an important research area. In computer vision, face pose estimation is the process of inferring the face orientation from digital imagery. It requires a serious of image processing steps to transform a pixel-based representation of a human face into a high-level concept of direction. An ideal face pose estimator should be invariant to a variety of image-changing factors such as camera distortion, lighting condition, skin colour, projective geometry, facial hairs, facial expressions, presence of accessories like glasses and hats, etc. Face pose estimation has been a focus of research for about two decades and numerous research contributions have been presented in this field. Face pose estimation techniques in literature have still some shortcomings and limitations in terms of accuracy, applicability to monocular images, being autonomous, identity and lighting variations, image resolution variations, range of face motion, computational expense, presence of facial hairs, presence of accessories like glasses and hats, etc. These shortcomings of existing face pose estimation techniques motivated the research work presented in this thesis. The main focus of this research is to design and develop novel face pose estimation algorithms that improve automatic face pose estimation in terms of processing time, computational expense, and invariance to different conditions

Design and performance assessment of correlation filters for the detection of objects in high clutter thermal imagery

The research reported in this thesis has examined means of enhancing the performance of the Optimal Trade-off Maximum Average Correlation Height (OT-MACH) filter for target detection in Forward Looking Infra-Red (FLIR) imagery acquired from a helicopter and border security FLIR camera in northern Kuwait. The data acquired with these FLIR sensors allows real-world evaluation of the comparative performance of the various filters that have been developed in the thesis. The results obtained have been quantified using well known performance measures such as Peak to Side-lobe Ratio (PSR) and Total Detection Error (TDE). The initial focus was to study the effect of modifying the OT-MACH parameters on the correlation metrics. A new optimisation technique has been presented, which computes statistically the filter alpha parameter associated with controlling the response of the filter to clutter noise. A further modification of the OT-MACH filter performance using the Difference of Gaussian bandpass filter (named the D-MACH filter) as a pre-processing stage has been described. The D-MACH has been applied to several test images containing single and multiple targets in the scene. Enhanced performance of the modified filter is demonstrated with improved metrics being obtained with less false side peaks in the correlation plane, especially when multiple targets are present in the test images. A further pre-processing technique was investigated using the Rayleigh distribution as a pre-processing filter (named the R-MACH filter). The R-MACH filter has been applied to multiple target types with tests conducted across various image data sets. The filter demonstrated an improvement over the Difference of Gaussian filter in terms of 6 reducing the number of parameters needing to be tuned whilst producing further enhanced correlation plane metrics. Finally, recommendations for future work has been made to improve the use of the OT-MACH filter in target detection and identification. A novel training image representation is proposed for further investigation, which will minimise the computational intensity of using the MACH filter for unconstrained object recognition

Multi-modal video analysis for early fire detection

In dit proefschrift worden verschillende aspecten van een intelligent videogebaseerd branddetectiesysteem onderzocht. In een eerste luik ligt de nadruk op de multimodale verwerking van visuele, infrarood en time-of-flight videobeelden, die de louter visuele detectie verbetert. Om de verwerkingskost zo minimaal mogelijk te houden, met het oog op real-time detectie, is er voor elk van het type sensoren een set ’low-cost’ brandkarakteristieken geselecteerd die vuur en vlammen uniek beschrijven. Door het samenvoegen van de verschillende typen informatie kunnen het aantal gemiste detecties en valse alarmen worden gereduceerd, wat resulteert in een significante verbetering van videogebaseerde branddetectie. Om de multimodale detectieresultaten te kunnen combineren, dienen de multimodale beelden wel geregistreerd (~gealigneerd) te zijn. Het tweede luik van dit proefschrift focust zich hoofdzakelijk op dit samenvoegen van multimodale data en behandelt een nieuwe silhouet gebaseerde registratiemethode. In het derde en tevens laatste luik van dit proefschrift worden methodes voorgesteld om videogebaseerde brandanalyse, en in een latere fase ook brandmodellering, uit te voeren. Elk van de voorgestelde technieken voor multimodale detectie en multi-view lokalisatie zijn uitvoerig getest in de praktijk. Zo werden onder andere succesvolle testen uitgevoerd voor de vroegtijdige detectie van wagenbranden in ondergrondse parkeergarages

Filtering of image sequences: on line edge detection and motion reconstruction

L'argomento della Tesi riguarda líelaborazione di sequenze di immagini, relative ad una scena in cui uno o pi˘ oggetti (possibilmente deformabili) si muovono e acquisite da un opportuno strumento di misura. A causa del processo di misura, le immagini sono corrotte da un livello di degradazione. Si riporta la formalizzazione matematica dellíinsieme delle immagini considerate, dellíinsieme dei moti ammissibili e della degradazione introdotta dallo strumento di misura. Ogni immagine della sequenza acquisita ha una relazione con tutte le altre, stabilita dalla legge del moto della scena. Líidea proposta in questa Tesi Ë quella di sfruttare questa relazione tra le diverse immagini della sequenza per ricostruire grandezze di interesse che caratterizzano la scena. Nel caso in cui si conosce il moto, líinteresse Ë quello di ricostruire i contorni dellíimmagine iniziale (che poi possono essere propagati attraverso la stessa legge del moto, in modo da ricostruire i contorni della generica immagine appartenente alla sequenza in esame), stimando líampiezza e del salto del livello di grigio e la relativa localizzazione. Nel caso duale si suppone invece di conoscere la disposizione dei contorni nellíimmagine iniziale e di avere un modello stocastico che descriva il moto; líobiettivo Ë quindi stimare i parametri che caratterizzano tale modello. Infine, si presentano i risultati dellíapplicazione delle due metodologie succitate a dati reali ottenuti in ambito biomedicale da uno strumento denominato pupillometro. Tali risultati sono di elevato interesse nellíottica di utilizzare il suddetto strumento a fini diagnostici

“Design, Development and Characterization of a Thermal Sensor Brick System for Modular Robotics

This thesis presents the work on thermal imaging sensor brick (TISB) system for modular robotics. The research demonstrates the design, development and characterization of the TISB system. The TISB system is based on the design philosophy of sensor bricks for modular robotics. In under vehicle surveillance for threat detection, which is a target application of this work we have demonstrated the advantages of the TISB system over purely vision-based systems. We have highlighted the advantages of the TISB system as an illumination invariant threat detection system for detecting hidden threat objects in the undercarriage of a car. We have compared the TISB system to the vision sensor brick system and the mirror on a stick. We have also illustrated the operational capability of the system on the SafeBot under vehicle robot to acquire and transmit the data wirelessly. The early designs of the TISB system, the evolution of the designs and the uniformity achieved while maintaining the modularity in building the different sensor bricks; the visual, the thermal and the range sensor brick is presented as part of this work. Each of these sensor brick systems designed and implemented at the Imaging Robotics and Intelligent Systems (IRIS) laboratory consist of four major blocks: Sensing and Image Acquisition Block, Pre-Processing and Fusion Block, Communication Block, and Power Block. The Sensing and Image Acquisition Block is to capture images or acquire data. The Pre-Processing and Fusion Block is to work on the acquired images or data. The Communication Block is for transferring data between the sensor brick and the remote host computer. The Power Block is to maintain power supply to the entire brick. The modular sensor bricks are self-sufficient plug and play systems. The SafeBot under vehicle robot designed and implemented at the IRIS laboratory has two tracked platforms one on each side with a payload bay area in the middle. Each of these tracked platforms is a mobility brick based on the same design philosophy as the modular sensor bricks. The robot can carry one brick at a time or even multiple bricks at the same time. The contributions of this thesis are: (1) designing and developing the hardware implementation of the TISB system, (2) designing and developing the software for the TISB system, and (3) characterizing the TISB system, where this characterization of the system is the major contribution of this thesis. The analysis of the thermal sensor brick system provides the user and future designers with sufficient information on parameters to be considered to make the right choice for future modifications, the kind of applications the TISB could handle and the load that the different blocks of the TISB system could manage. Under vehicle surveillance for threat detection, perimeter / area surveillance, scouting, and improvised explosive device (IED) detection using a car-mounted system are some of the applications that have been identified for this system

Pattern Recognition

A wealth of advanced pattern recognition algorithms are emerging from the interdiscipline between technologies of effective visual features and the human-brain cognition process. Effective visual features are made possible through the rapid developments in appropriate sensor equipments, novel filter designs, and viable information processing architectures. While the understanding of human-brain cognition process broadens the way in which the computer can perform pattern recognition tasks. The present book is intended to collect representative researches around the globe focusing on low-level vision, filter design, features and image descriptors, data mining and analysis, and biologically inspired algorithms. The 27 chapters coved in this book disclose recent advances and new ideas in promoting the techniques, technology and applications of pattern recognition