Vehicle make and model recognition for intelligent transportation monitoring and surveillance.

Vehicle Make and Model Recognition (VMMR) has evolved into a significant subject of study due to its importance in numerous Intelligent Transportation Systems (ITS), such as autonomous navigation, traffic analysis, traffic surveillance and security systems. A highly accurate and real-time VMMR system significantly reduces the overhead cost of resources otherwise required. The VMMR problem is a multi-class classification task with a peculiar set of issues and challenges like multiplicity, inter- and intra-make ambiguity among various vehicles makes and models, which need to be solved in an efficient and reliable manner to achieve a highly robust VMMR system. In this dissertation, facing the growing importance of make and model recognition of vehicles, we present a VMMR system that provides very high accuracy rates and is robust to several challenges. We demonstrate that the VMMR problem can be addressed by locating discriminative parts where the most significant appearance variations occur in each category, and learning expressive appearance descriptors. Given these insights, we consider two data driven frameworks: a Multiple-Instance Learning-based (MIL) system using hand-crafted features and an extended application of deep neural networks using MIL. Our approach requires only image level class labels, and the discriminative parts of each target class are selected in a fully unsupervised manner without any use of part annotations or segmentation masks, which may be costly to obtain. This advantage makes our system more intelligent, scalable, and applicable to other fine-grained recognition tasks. We constructed a dataset with 291,752 images representing 9,170 different vehicles to validate and evaluate our approach. Experimental results demonstrate that the localization of parts and distinguishing their discriminative powers for categorization improve the performance of fine-grained categorization. Extensive experiments conducted using our approaches yield superior results for images that were occluded, under low illumination, partial camera views, or even non-frontal views, available in our real-world VMMR dataset. The approaches presented herewith provide a highly accurate VMMR system for rea-ltime applications in realistic environments.\\ We also validate our system with a significant application of VMMR to ITS that involves automated vehicular surveillance. We show that our application can provide law inforcement agencies with efficient tools to search for a specific vehicle type, make, or model, and to track the path of a given vehicle using the position of multiple cameras

Advanced Feature Learning and Representation in Image Processing for Anomaly Detection

Techniques for improving the information quality present in imagery for feature extraction are proposed in this thesis. Specifically, two methods are presented: soft feature extraction and improved Evolution-COnstructed (iECO) features. Soft features comprise the extraction of image-space knowledge by performing a per-pixel weighting based on an importance map. Through soft features, one is able to extract features relevant to identifying a given object versus its background. Next, the iECO features framework is presented. The iECO features framework uses evolutionary computation algorithms to learn an optimal series of image transforms, specific to a given feature descriptor, to best extract discriminative information. That is, a composition of image transforms are learned from training data to present a given feature descriptor with the best opportunity to extract its information for the application at hand. The proposed techniques are applied to an automatic explosive hazard detection application and significant results are achieved

Using Unsupervised Learning Methods to Analyse Magnetic Resonance Imaging (MRI) Scans for the Detection of Alzheimer’s Disease

Background: Alzheimer’s disease (AD) is the most common cause of dementia, characterised by behavioural and cognitive impairment. The manual diagnosis of AD by doctors is time-consuming and can be ineffective, so machine learning methods are increasingly being proposed to diagnose AD in many recent studies. Most research developing machine learning algorithms to diagnose AD use supervised learning to classify magnetic resonance imaging (MRI) scans. However, supervised learning requires a considerable volume of labelled data and MRI scans are difficult to label. The aim of this thesis was therefore to use unsupervised learning methods to differentiate between MRI scans from people who were cognitively normal (CN), people with mild cognitive impairment (MCI), and people with AD. Objectives: This study applied a statistical method and unsupervised learning methods to discriminate scans from (1) people with CN and with AD; (2) people with stable mild cognitive impairment (sMCI) and with progressive mild cognitive impairment (pMCI); (3) people with CN and with pMCI, using a limited number of labelled structural MRI scans. Methods: Two-sample t-tests were used to detect the regions of interest (ROIs) between each of the two groups (CN vs. AD; sMCI vs. pMCI; CN vs. pMCI), and then an unsupervised learning neural network was employed to extract features from the regions. Finally, a clustering algorithm was implemented to discriminate between each of the two groups based on the extracted features. The approach was tested on baseline brain structural MRI scans from 715 individuals from the Alzheimer’s Disease Neuroimaging Initiative (ADNI), of which 231 were CN, 198 had AD, 152 had sMCI, and 134 were pMCI. The results were evaluated by calculating the overall accuracy, the sensitivity, specificity, and positive and negative predictive values. Results: The abnormal regions around the lower parts of the limbic system were indicated as AD-relevant regions based on the two-sample t-test (p<0.001), and the proposed method yielded an overall accuracy of 0.842 for discriminating between CN and AD, an overall accuracy of 0.672 for discriminating between sMCI and pMCI, and an overall accuracy of 0.776 for discriminating between CN and pMCI. Conclusion: The study combined statistical and unsupervised learning methods to identify scans of people with different stages of AD. This method can detect AD-relevant regions and could be used to accurately diagnose stages of AD; it has the advantage that it does not require large amounts of labelled MRI scans. The performances of the three discriminations were all comparable to those of previous state-of-the-art studies. The research in this thesis could be implemented in the future to help in the automatic diagnosis of AD and provide a basis for diagnosing sMCI and pMCI

Advanced traffic video analytics for robust traffic accident detection

Automatic traffic accident detection is an important task in traffic video analysis due to its key applications in developing intelligent transportation systems. Reducing the time delay between the occurrence of an accident and the dispatch of the first responders to the scene may help lower the mortality rate and save lives. Since 1980, many approaches have been presented for the automatic detection of incidents in traffic videos. In this dissertation, some challenging problems for accident detection in traffic videos are discussed and a new framework is presented in order to automatically detect single-vehicle and intersection traffic accidents in real-time. First, a new foreground detection method is applied in order to detect the moving vehicles and subtract the ever-changing background in the traffic video frames captured by static or non-stationary cameras. For the traffic videos captured during day-time, the cast shadows degrade the performance of the foreground detection and road segmentation. A novel cast shadow detection method is therefore presented to detect and remove the shadows cast by moving vehicles and also the shadows cast by static objects on the road. Second, a new method is presented to detect the region of interest (ROI), which applies the location of the moving vehicles and the initial road samples and extracts the discriminating features to segment the road region. After detecting the ROI, the moving direction of the traffic is estimated based on the rationale that the crashed vehicles often make rapid change of direction. Lastly, single-vehicle traffic accidents and trajectory conflicts are detected using the first-order logic decision-making system. The experimental results using publicly available videos and a dataset provided by the New Jersey Department of Transportation (NJDOT) demonstrate the feasibility of the proposed methods. Additionally, the main challenges and future directions are discussed regarding (i) improving the performance of the foreground segmentation, (ii) reducing the computational complexity, and (iii) detecting other types of traffic accidents

Object detection, recognition and re-identification in video footage

There has been a significant number of security concerns in recent times; as a result, security cameras have been installed to monitor activities and to prevent crimes in most public places. These analysis are done either through video analytic or forensic analysis operations on human observations. To this end, within the research context of this thesis, a proactive machine vision based military recognition system has been developed to help monitor activities in the military environment. The proposed object detection, recognition and re-identification systems have been presented in this thesis. A novel technique for military personnel recognition is presented in this thesis. Initially the detected camouflaged personnel are segmented using a grabcut segmentation algorithm. Since in general a camouflaged personnel's uniform appears to be similar both at the top and the bottom of the body, an image patch is initially extracted from the segmented foreground image and used as the region of interest. Subsequently the colour and texture features are extracted from each patch and used for classification. A second approach for personnel recognition is proposed through the recognition of the badge on the cap of a military person. A feature matching metric based on the extracted Speed Up Robust Features (SURF) from the badge on a personnel's cap enabled the recognition of the personnel's arm of service. A state-of-the-art technique for recognising vehicle types irrespective of their view angle is also presented in this thesis. Vehicles are initially detected and segmented using a Gaussian Mixture Model (GMM) based foreground/background segmentation algorithm. A Canny Edge Detection (CED) stage, followed by morphological operations are used as pre-processing stage to help enhance foreground vehicular object detection and segmentation. Subsequently, Region, Histogram Oriented Gradient (HOG) and Local Binary Pattern (LBP) features are extracted from the refined foreground vehicle object and used as features for vehicle type recognition. Two different datasets with variant views of front/rear and angle are used and combined for testing the proposed technique. For night-time video analytics and forensics, the thesis presents a novel approach to pedestrian detection and vehicle type recognition. A novel feature acquisition technique named, CENTROG, is proposed for pedestrian detection and vehicle type recognition in this thesis. Thermal images containing pedestrians and vehicular objects are used to analyse the performance of the proposed algorithms. The video is initially segmented using a GMM based foreground object segmentation algorithm. A CED based pre-processing step is used to enhance segmentation accuracy prior using Census Transforms for initial feature extraction. HOG features are then extracted from the Census transformed images and used for detection and recognition respectively of human and vehicular objects in thermal images. Finally, a novel technique for people re-identification is proposed in this thesis based on using low-level colour features and mid-level attributes. The low-level colour histogram bin values were normalised to 0 and 1. A publicly available dataset (VIPeR) and a self constructed dataset have been used in the experiments conducted with 7 clothing attributes and low-level colour histogram features. These 7 attributes are detected using features extracted from 5 different regions of a detected human object using an SVM classifier. The low-level colour features were extracted from the regions of a detected human object. These 5 regions are obtained by human object segmentation and subsequent body part sub-division. People are re-identified by computing the Euclidean distance between a probe and the gallery image sets. The experiments conducted using SVM classifier and Euclidean distance has proven that the proposed techniques attained all of the aforementioned goals. The colour and texture features proposed for camouflage military personnel recognition surpasses the state-of-the-art methods. Similarly, experiments prove that combining features performed best when recognising vehicles in different views subsequent to initial training based on multi-views. In the same vein, the proposed CENTROG technique performed better than the state-of-the-art CENTRIST technique for both pedestrian detection and vehicle type recognition at night-time using thermal images. Finally, we show that the proposed 7 mid-level attributes and the low-level features results in improved performance accuracy for people re-identification

Learning Interpretable Features of Graphs and Time Series Data

Graphs and time series are two of the most ubiquitous representations of data of modern time. Representation learning of real-world graphs and time-series data is a key component for the downstream supervised and unsupervised machine learning tasks such as classification, clustering, and visualization. Because of the inherent high dimensionality, representation learning, i.e., low dimensional vector-based embedding of graphs and time-series data is very challenging. Learning interpretable features incorporates transparency of the feature roles, and facilitates downstream analytics tasks in addition to maximizing the performance of the downstream machine learning models. In this thesis, we leveraged tensor (multidimensional array) decomposition for generating interpretable and low dimensional feature space of graphs and time-series data found from three domains: social networks, neuroscience, and heliophysics. We present the theoretical models and empirical results on node embedding of social networks, biomarker embedding on fMRI-based brain networks, and prediction and visualization of multivariate time-series-based flaring and non-flaring solar events

Ubiquitous Technologies for Emotion Recognition

Emotions play a very important role in how we think and behave. As such, the emotions we feel every day can compel us to act and influence the decisions and plans we make about our lives. Being able to measure, analyze, and better comprehend how or why our emotions may change is thus of much relevance to understand human behavior and its consequences. Despite the great efforts made in the past in the study of human emotions, it is only now, with the advent of wearable, mobile, and ubiquitous technologies, that we can aim to sense and recognize emotions, continuously and in real time. This book brings together the latest experiences, findings, and developments regarding ubiquitous sensing, modeling, and the recognition of human emotions

Mobile Thermography-based Physiological Computing for Automatic Recognition of a Person’s Mental Stress

This thesis explores the use of Mobile Thermography1, a significantly less investigated sensing capability, with the aim of reliably extracting a person’s multiple physiological signatures and recognising mental stress in an automatic, contactless manner. Mobile thermography has greater potentials for real-world applications because of its light-weight, low computation-cost characteristics. In addition, thermography itself does not necessarily require the sensors to be worn directly on the skin. It raises less privacy concerns and is less sensitive to ambient lighting conditions. The work presented in this thesis is structured through a three-stage approach that aims to address the following challenges: i) thermal image processing for mobile thermography in variable thermal range scenes; ii) creation of rich and robust physiology measurements; and iii) automated stress recognition based on such measurements. Through the first stage (Chapter 4), this thesis contributes new processing techniques to address negative effects of environmental temperature changes upon automatic tracking of regions-of-interest and measuring of surface temperature patterns. In the second stage (Chapters 5,6,7), the main contributions are: robustness in tracking respiratory and cardiovascular thermal signatures both in constrained and unconstrained settings (e.g. respiration: strong correlation with ground truth, r=0.9987), and investigation of novel cortical thermal signatures associated with mental stress. The final stage (Chapters 8,9) contributes automatic stress inference systems that focus on capturing richer dynamic information of physiological variability: firstly, a novel respiration representation-based system (which has achieved state-of-the-art performance: 84.59% accuracy, two stress levels), and secondly, a novel cardiovascular representation-based system using short-term measurements of nasal thermal variability and heartrate variability from another sensing channel (78.33% accuracy achieved from 20seconds measurements). Finally, this thesis contributes software libraries and incrementally built labelled datasets of thermal images in both constrained and everyday ubiquitous settings. These are used to evaluate performance of our proposed computational methods across the three-stages