Agent-based framework for person re-identification

In computer based human object re-identification, a detected human is recognised to a level sufficient to re-identify a tracked person in either a different camera capturing the same individual, often at a different angle, or the same camera at a different time and/or the person approaching the camera at a different angle. Instead of relying on face recognition technology such systems study the clothing of the individuals being monitored and/or objects being carried to establish correspondence and hence re-identify the human object. Unfortunately present human-object re-identification systems consider the entire human object as one connected region in making the decisions about similarity of two objects being matched. This assumption has a major drawback in that when a person is partially occluded, a part of the occluding foreground will be picked up and used in matching. Our research revealed that when a human observer carries out a manual human-object re-identification task, the attention is often taken over by some parts of the human figure/body, more than the others, e.g. face, brightly colour shirt, presence of texture patterns in clothing etc., and occluding parts are ignored. In this thesis, a novel multi-agent based framework is proposed for the design of a human object re-identification system. Initially a HOG based feature extraction is used in a SVM based classification of a human object as a human of a full-body or of half body nature. Subsequently the relative visual significance of the top and the bottom parts of the human, in re-identification is quantified by the analysis of Gray Level Co-occurrence based texture features and colour histograms obtained in the HSV colour space. Accordingly different weights are assigned to the top and bottom of the human body using a novel probabilistic approach. The weights are then used to modify the Hybrid Spatiogram and Covariance Descriptor (HSCD) feature based re-identification algorithm adopted. A significant novelty of the human object re-identification systems proposed in this thesis is the agent based design procedure adopted that separates the use of computer vision algorithms for feature extraction, comparison etc., from the decision making process of re-identification. Multiple agents are assigned to execute different algorithmic tasks and the agents communicate to make the required logical decisions. Detailed experimental results are provided to prove that the proposed multi agent based framework for human object re-identification performs significantly better than the state of-the-art algorithms. Further it is shown that the design flexibilities and scalabilities of the proposed system allows it to be effectively utilised in more complex computer vision based video analytic/forensic tasks often conducted within distributed, multi-camera systems

Deep Representation Learning for Vehicle Re-Identification

With the widespread use of surveillance cameras in cities and on motorways, computer vision based intelligent systems are becoming a standard in the industry. Vehicle related problems such as Automatic License Plate Recognition have been addressed by computer vision systems, albeit in controlled settings (e.g.cameras installed at toll gates). Due to the freely available research data becoming available in the last few years, surveillance footage analysis for vehicle related problems are being studied with a computer vision focus. In this thesis, vision-based approaches for the problem of vehicle re-identification are investigated and original approaches are presented for various challenges of the problem. Computer vision based systems have advanced considerably in the last decade due to rapid improvements in machine learning with the advent of deep learning and convolutional neural networks (CNNs). At the core of the paradigm shift that has arrived with deep learning in machine learning is feature learning by multiple stacked neural network layers. Compared to traditional machine learning methods that utilise hand-crafted feature extraction and shallow model learning, deep neural networks can learn hierarchical feature representations as input data transform from low-level to high-level representation through consecutive neural network layers. Furthermore, machine learning tasks are trained in an end-to-end fashion that integrates feature extraction and machine learning methods into a combined framework using neural networks. This thesis focuses on visual feature learning with deep convolutional neural networks for the vehicle re-identification problem. The problem of re-identification has attracted attention from the computer vision community, especially for the person re-identification domain, whereas vehicle re-identification is relatively understudied. Re-identification is the problem of matching identities of subjects in images. The images come from non-overlapping viewing angles captured at varying locations, illuminations, etc. Compared to person re-identification, vehicle reidentification is particularly challenging as vehicles are manufactured to have the same visual appearance and shape that makes different instances visually indistinguishable. This thesis investigates solutions for the aforementioned challenges and makes the following contributions, improving accuracy and robustness of recent approaches. The contributions are the following: (1) Exploring the man-made nature of vehicles, that is, their hierarchical categories such as type (e.g.sedan, SUV) and model (e.g.Audi-2011-A4) and its usefulness in identity matching when identity pairwise labelling is not present (2) A new vehicle re-identification benchmark, Vehicle Re-Identification in Context (VRIC), is introduced to enable the design and evaluation of vehicle re-id methods to more closely reflect real-world application conditions compared to existing benchmarks. VRIC is uniquely characterised by unconstrained vehicle images in low resolution; from wide field of view traffic scene videos exhibiting variations of illumination, motion blur,and occlusion. (3) We evaluate the advantages of Multi-Scale Visual Representation (MSVR) in multi-scale cross-camera matching performance by training a multi-branch CNN model for vehicle re-identification enabled by the availability of low resolution images in VRIC. Experimental results indicate that this approach is useful in real-world settings where image resolution is low and varying across cameras. (4) With Multi-Task Mutual Learning (MTML) we propose a multi-modal learning representation e.g.using orientation as well as identity labels in training. We utilise deep convolutional neural networks with multiple branches to facilitate the learning of multi-modal and multi-scale deep features that increase re-identification performance, as well as orientation invariant feature learning