Advanced machine learning approaches for target detection, tracking and recognition

This dissertation addresses the key technical components of an Automatic Target Recognition (ATR) system namely: target detection, tracking, learning and recognition. Novel solutions are proposed for each component of the ATR system based on several new advances in the field of computer vision and machine learning. Firstly, we introduce a simple and elegant feature, RelCom, and a boosted feature selection method to achieve a very low computational complexity target detector. Secondly, we present a particle filter based target tracking algorithm that uses a quad histogram based appearance model along with online feature selection. Further, we improve the tracking performance by means of online appearance learning where appearance learning is cast as an Adaptive Kalman filtering (AKF) problem which we formulate using both covariance matching and, for the first time in a visual tracking application, the recent autocovariance least-squares (ALS) method. Then, we introduce an integrated tracking and recognition system that uses two generative models to accommodate the pose variations and maneuverability of different ground targets. Specifically, a tensor-based generative model is used for multi-view target representation that can synthesize unseen poses, and can be trained from a small set of signatures. In addition, a target-dependent kinematic model is invoked to characterize the target dynamics. Both generative models are integrated in a graphical framework for joint estimation of the target's kinematics, pose, and discrete valued identity. Finally, for target recognition we advocate the concept of a continuous identity manifold that captures both inter-class and intra-class shape variability among training targets. A hemispherical view manifold is used for modeling the view-dependent appearance. In addition to being able to deal with arbitrary view variations, this model can determine the target identity at both class and sub-class levels, for targets not present in the training data. The proposed components of the ATR system enable us to perform low computational complexity target detection with low false alarm rates, robust tracking of targets under challenging circumstances and recognition of target identities at both class and sub-class levels. Experiments on real and simulated data confirm the performance of the proposed components with promising results

Robust Tracking in Aerial Imagery Based on an Ego-Motion Bayesian Model

A novel strategy for object tracking in aerial imagery is presented, which is able to deal with complex situations where the camera ego-motion cannot be reliably estimated due to the aperture problem (related to low structured scenes), the strong ego-motion, and/or the presence of independent moving objects. The proposed algorithm is based on a complex modeling of the dynamic information, which simulates both the object and the camera dynamics to predict the putative object locations. In this model, the camera dynamics is probabilistically formulated as a weighted set of affine transformations that represent possible camera ego-motions. This dynamic model is used in a Particle Filter framework to distinguish the actual object location among the multiple candidates, that result from complex cluttered backgrounds, and the presence of several moving objects. The proposed strategy has been tested with the aerial FLIR AMCOM dataset, and its performance has been also compared with other tracking techniques to demonstrate its efficiency

Object Detection and Tracking in Wide Area Surveillance Using Thermal Imagery

The main objective behind this thesis is to examine how existing vision-based detection and tracking algorithms perform in thermal imagery-based video surveillance. While color-based surveillance has been extensively studied, these techniques can not be used during low illumination, at night, or with lighting changes and shadows which limits their applicability. The main contributions in this thesis are (1) the creation of a new color-thermal dataset, (2) a detailed performance comparison of different color-based detection and tracking algorithms on thermal data and (3) the proposal of an adaptive neural network for false detection rejection. Since there are not many publicly available datasets for thermal-video surveillance, a new UNLV Thermal Color Pedestrian Dataset was collected to evaluate the performance of popular color-based detection and tracking in thermal images. The dataset provides an overhead view of humans walking through a courtyard and is appropriate for aerial surveillance scenarios such as unmanned aerial systems (UAS). Three popular detection schemes are studied for thermal pedestrian detection: 1) Haar-like features, 2) local binary pattern (LBP) and 3) background subtraction motion detection. A i) Kalman filter predictor and ii) optical flow are used for tracking. Results show that combining Haar and LBP detections with a 50% overlap rule and tracking using Kalman filters can improve the true positive rate (TPR) of detection by 20%. However, motion-based methods are better at rejecting false positive in non-moving camera scenarios. The Kalman filter with LBP detection is the most efficient tracker but optical flow better rejects false noise detections. This thesis also presents a technique for learning and characterizing pedestrian detections with heat maps and an object-centric motion compensation method for UAS. Finally, an adaptive method to reject false detections using error back propagation using a neural network. The adaptive rejection scheme is able to successfully learn to identify static false detections for improved detection performance

Integration and Evaluation of a Video Surveillance System

Visual surveillance systems are getting a lot of attention over the last few years, due to a growing need for surveillance applications. In this thesis, we present a visual surveillance system that integrates modules for motion detection, tracking, and trajectory characterization to achieve robust monitoring of moving objects in scenes under surveillance. The system operates on video sequences acquired by stationary color and infra-red surveillance cameras. Motion detection is implemented using an algorithm that combines thresholding of temporal variance and background modeling. The tracking algorithm combines motion and appearance information into an appearance model and uses a particle filter framework for object tracking. The trajectory analysis module builds a model for a given normal activity using a factorization approach, and uses this model for the detection of any abnormal motion pattern. The system was tested on a large ground-truthed data set containing hundreds of color and FLIR image sequences. Results of performance evaluation using these sequences are reported in this thesis

A STATE VECTOR AUGMENTATION METHOD FOR INCLUDING VELOCITY INFORMATION IN THE LIKELIHOOD FUNCTION OF THE SIR VIDEO TARGET TRACKING FILTER

This thesis is focused on visual target tracking. Visual target tracking has been widely studied. The main idea is to be able to determine the target's location from a video sequence. Techniques such as the Kalman Filter and its variations have been proved to be the optimal solution when the system is linear or can be linearized, and Gaussianity can be assumed. But these conditions often do not hold in real world applications. Therefore, an alternative approach based on Sequential Monte-Carlo methods, also known as the Particle Filter, arose among others and has become a popular technique for target tracking recently. The particle filter is able to estimate the target state under nonlinear, non-Gaussian conditions. Different types of particle filters have been developed over the years, but one of the most popular is the sampling importance resampling (SIR) algorithm. However, in conditions of highly structured clutter and occlusion the filter's performance is decreased and the tracker can lock into the background and loose the target. Since motion information has been shown to be very important for the unmanned target tracking problem, in this thesis I introduce a new method to make the SIR filter more robust against these conditions by indirectly including velocity information in the likelihood function of the SIR filter. I propose augmenting the SIR filter state vector in order to use particle velocity information to prevent particles with poor motion estimates from obtaining large weights. The main original contributions of this thesis include the following: * I developed the theoretical formulation for the State Vector Augmented SIR filter algorithm. * I reformulated the normalized cross correlation used in the Likelihood function of the SIR filter to include the velocity information in it. * I developed an algorithm to generate synthetic data sequences with targets that can change both in magnification and rotation for testing the efficacy of tracking algorithms in a controlled environment. * I developed a simple template update strategy to deal with target appearance changes. * I prove the effectiveness of the proposed algorithm with tracking results obtained from two longwave infrared sequences and two synthetic data sequences. The results show that this new method can improve tracking performance for moving targets immersed in strong structured clutter