Face recognition with image sets using manifold density divergence

In many automatic face recognition applications, a set of a person\u27s face images is available rather than a single image. In this paper, we describe a novel method for face recognition using image sets. We propose a flexible, semi-parametric model for learning probability densities confined to highly non-linear but intrinsically low-dimensional manifolds. The model leads to a statistical formulation of the recognition problem in terms of minimizing the divergence between densities estimated on these manifolds. The proposed method is evaluated on a large data set, acquired in realistic imaging conditions with severe illumination variation. Our algorithm is shown to match the best and outperform other state-of-the-art algorithms in the literature, achieving 94% recognition rate on average

Bayesian multi-target tracking: application to total internal reflection fluorescence microscopy

This thesis focuses on the problem of automated tracking of tiny cellular and sub-cellular structures, known as particles, in the sequences acquired from total internal reflection fluorescence microscopy (TIRFM) imaging technique. Our primary biological motivation is to develop an automated system for tracking the sub-cellular structures involving exocytosis (an intracellular mechanism) which is helpful for studying the possible causes of the defects in diseases such as diabetes and obesity. However, all methods proposed in this thesis are generalized to be applicable for a wide range of particle tracking applications. A reliable multi-particle tracking method should be capable of tracking numerous similar objects in the presence of high levels of noise, high target density and complex motions and interactions. In this thesis, we choose the Bayesian filtering framework as our main approach to deal with this problem. We focus on the approaches that work based on detections. Therefore, in this thesis, we first propose a method that robustly detects the particles in the noisy TIRFM sequences with inhomogeneous and time-varying background. In order to evaluate our detection and tracking methods on the sequences with known and reliable ground truth, we also present a framework for generating realistic synthetic TIRFM data. To propose a reliable multi-particle tracking method for TIRFM sequences, we suggest a framework by combining two robust Bayesian filters, the interacting multiple model and joint probabilistic data association (IMM-JPDA) filters. The performance of our particle tracking method is compared against those of several popular and state-of-the art particle tracking approaches on both synthetic and real sequences. Although our approach performs well in tracking particles, it can be very computationally demanding for the applications with dense targets with poor detections. To propose a computationally cheap, but reliable, multi-particle tracking method, we investigate the performance of a recent multi-target Bayesian filter based on random finite theory, the probability hypothesis density (PHD) filter, on our application. To this end, we propose a general framework for tracking particles using this filter. Moreover, we assess the performance of our proposed PHD filter on both synthetic and real sequences with high level of noise and particle density. We compare its results from both aspects of accuracy and processing time against our IMM-JPDA filter. Finally, we suggest a framework for tracking particles in a challenging problem where the noise characteristic and the background intensity of sequences change during the acquisition process which make detection profile and clutter rate time-variant. To deal with this, we propose a bootstrap filter using another type of the random finite set based Bayesian filters, the cardinalized PHD (CPHD) filter, composed of an estimator and a tracker. The estimator adaptively estimates the required meta parameters for the tracker such as clutter rate and the detection probability while the tracker estimates the state of the targets. We evaluate the performance of our bootstrap on both synthetic and real sequences under these time-varying conditions. Moreover, its performance is compared against those of our other particle trackers as well as the state-of-the art particle tracking approaches

End-to-end anomaly detection in stream data

Nowadays, huge volumes of data are generated with increasing velocity through various systems, applications, and activities. This increases the demand for stream and time series analysis to react to changing conditions in real-time for enhanced efficiency and quality of service delivery as well as upgraded safety and security in private and public sectors. Despite its very rich history, time series anomaly detection is still one of the vital topics in machine learning research and is receiving increasing attention. Identifying hidden patterns and selecting an appropriate model that fits the observed data well and also carries over to unobserved data is not a trivial task. Due to the increasing diversity of data sources and associated stochastic processes, this pivotal data analysis topic is loaded with various challenges like complex latent patterns, concept drift, and overfitting that may mislead the model and cause a high false alarm rate. Handling these challenges leads the advanced anomaly detection methods to develop sophisticated decision logic, which turns them into mysterious and inexplicable black-boxes. Contrary to this trend, end-users expect transparency and verifiability to trust a model and the outcomes it produces. Also, pointing the users to the most anomalous/malicious areas of time series and causal features could save them time, energy, and money. For the mentioned reasons, this thesis is addressing the crucial challenges in an end-to-end pipeline of stream-based anomaly detection through the three essential phases of behavior prediction, inference, and interpretation. The first step is focused on devising a time series model that leads to high average accuracy as well as small error deviation. On this basis, we propose higher-quality anomaly detection and scoring techniques that utilize the related contexts to reclassify the observations and post-pruning the unjustified events. Last but not least, we make the predictive process transparent and verifiable by providing meaningful reasoning behind its generated results based on the understandable concepts by a human. The provided insight can pinpoint the anomalous regions of time series and explain why the current status of a system has been flagged as anomalous. Stream-based anomaly detection research is a principal area of innovation to support our economy, security, and even the safety and health of societies worldwide. We believe our proposed analysis techniques can contribute to building a situational awareness platform and open new perspectives in a variety of domains like cybersecurity, and health

Pedestrian localization, tracking and behavior analysis from multiple cameras

Video surveillance is currently undergoing a rapid growth. However, while thousands of cameras are being installed in public places all over the world, computer programs that could reliably detect and track people in order to analyze their behavior are not yet operational. Challenges are numerous, ranging from low image quality, suboptimal scene lighting, changing appearances of pedestrians, occlusions with environment and between people, complex interacting trajectories in crowds, etc. In this thesis, we propose a complete approach for detecting and tracking an unknown number of interacting people from multiple cameras located at eye level. Our system works reliably in spite of significant occlusions and delivers metrically accurate trajectories for each tracked individual. Furthermore, we develop a method for representing the most common types of motion in a specific environment and learning them automatically from image data. We demonstrate that a generative model for detection can effectively handle occlusions in each time frame independently, even when the only data available comes from the output of a simple background subtraction algorithm and when the number of individuals is unknown a priori. We then advocate that multi-people tracking can be achieved by detecting people in individual frames and then linking detections across frames. We formulate the linking step as a problem of finding the most probable state of a hidden Markov process given the set of images and frame-independent detections. We first propose to solve this problem by optimizing trajectories independently with Dynamic Programming. In a second step, we reformulate the problem as a constrained flow optimization resulting in a convex problem that can be solved using standard Linear Programming techniques and is far simpler formally and algorithmically than existing techniques. We show that the particular structure of this framework lets us solve it equivalently using the k-shortest paths algorithm, which leads to a much faster optimization. Finally, we introduce a novel behavioral model to describe pedestrians motions, which is able to capture sophisticated motion patterns resulting from the mixture of different categories of random trajectories. Due to its simplicity, this model can be learned from video sequences in a totally unsupervised manner through an Expectation-Maximization procedure. We show that this behavior model can be used to make tracking systems more robust in ambiguous situations. Moreover, we demonstrate its ability to characterize and detect atypical individual motions