Automatic object classification for surveillance videos.

PhDThe recent popularity of surveillance video systems, specially located in urban scenarios, demands the development of visual techniques for monitoring purposes. A primary step towards intelligent surveillance video systems consists on automatic object classification, which still remains an open research problem and the keystone for the development of more specific applications. Typically, object representation is based on the inherent visual features. However, psychological studies have demonstrated that human beings can routinely categorise objects according to their behaviour. The existing gap in the understanding between the features automatically extracted by a computer, such as appearance-based features, and the concepts unconsciously perceived by human beings but unattainable for machines, or the behaviour features, is most commonly known as semantic gap. Consequently, this thesis proposes to narrow the semantic gap and bring together machine and human understanding towards object classification. Thus, a Surveillance Media Management is proposed to automatically detect and classify objects by analysing the physical properties inherent in their appearance (machine understanding) and the behaviour patterns which require a higher level of understanding (human understanding). Finally, a probabilistic multimodal fusion algorithm bridges the gap performing an automatic classification considering both machine and human understanding. The performance of the proposed Surveillance Media Management framework has been thoroughly evaluated on outdoor surveillance datasets. The experiments conducted demonstrated that the combination of machine and human understanding substantially enhanced the object classification performance. Finally, the inclusion of human reasoning and understanding provides the essential information to bridge the semantic gap towards smart surveillance video systems

Active object recognition for 2D and 3D applications

Includes bibliographical referencesActive object recognition provides a mechanism for selecting informative viewpoints to complete recognition tasks as quickly and accurately as possible. One can manipulate the position of the camera or the object of interest to obtain more useful information. This approach can improve the computational efficiency of the recognition task by only processing viewpoints selected based on the amount of relevant information they contain. Active object recognition methods are based around how to select the next best viewpoint and the integration of the extracted information. Most active recognition methods do not use local interest points which have been shown to work well in other recognition tasks and are tested on images containing a single object with no occlusions or clutter. In this thesis we investigate using local interest points (SIFT) in probabilistic and non-probabilistic settings for active single and multiple object and viewpoint/pose recognition. Test images used contain objects that are occluded and occur in significant clutter. Visually similar objects are also included in our dataset. Initially we introduce a non-probabilistic 3D active object recognition system which consists of a mechanism for selecting the next best viewpoint and an integration strategy to provide feedback to the system. A novel approach to weighting the uniqueness of features extracted is presented, using a vocabulary tree data structure. This process is then used to determine the next best viewpoint by selecting the one with the highest number of unique features. A Bayesian framework uses the modified statistics from the vocabulary structure to update the system's confidence in the identity of the object. New test images are only captured when the belief hypothesis is below a predefined threshold. This vocabulary tree method is tested against randomly selecting the next viewpoint and a state-of-the-art active object recognition method by Kootstra et al.. Our approach outperforms both methods by correctly recognizing more objects with less computational expense. This vocabulary tree method is extended for use in a probabilistic setting to improve the object recognition accuracy. We introduce Bayesian approaches for object recognition and object and pose recognition. Three likelihood models are introduced which incorporate various parameters and levels of complexity. The occlusion model, which includes geometric information and variables that cater for the background distribution and occlusion, correctly recognizes all objects on our challenging database. This probabilistic approach is further extended for recognizing multiple objects and poses in a test images. We show through experiments that this model can recognize multiple objects which occur in close proximity to distractor objects. Our viewpoint selection strategy is also extended to the multiple object application and performs well when compared to randomly selecting the next viewpoint, the activation model and mutual information. We also study the impact of using active vision for shape recognition. Fourier descriptors are used as input to our shape recognition system with mutual information as the active vision component. We build multinomial and Gaussian distributions using this information, which correctly recognizes a sequence of objects. We demonstrate the effectiveness of active vision in object recognition systems. We show that even in different recognition applications using different low level inputs, incorporating active vision improves the overall accuracy and decreases the computational expense of object recognition systems

Multi-camera cooperative scene interpretation

In our society, video processing has become a convenient and widely used tool to assist, protect and simplify the daily life of people in areas such as surveillance and video conferencing. The growing number of cameras, the handling and analysis of these vast amounts of video data enable the development of multi-camera applications that cooperatively use multiple sensors. In many applications, bandwidth constraints, privacy issues, and difficulties in storing and analyzing large amounts of video data make applications costly and technically challenging. In this thesis, we deploy techniques ranging from low-level to high-level approaches, specifically designed for multi-camera networks. As a low-level approach, we designed a novel low-level foreground detection algorithm for real-time tracking applications, concentrating on difficult and changing illumination conditions. The main part of this dissertation focuses on a detailed analysis of two novel state-of-the-art real-time tracking approaches: a multi-camera tracking approach based on occupancy maps and a distributed multi-camera tracking approach with a feedback loop. As a high-level application we propose an approach to understand the dynamics in meetings - so called, smart meetings - using a multi-camera setup, consisting of fixed ambient and portable close-up cameras. For all method, we provided qualitative and quantitative results on several experiments, compared to state-of-the-art methods

Advances and Applications of DSmT for Information Fusion. Collected Works, Volume 5

This fifth volume on Advances and Applications of DSmT for Information Fusion collects theoretical and applied contributions of researchers working in different fields of applications and in mathematics, and is available in open-access. The collected contributions of this volume have either been published or presented after disseminating the fourth volume in 2015 in international conferences, seminars, workshops and journals, or they are new. The contributions of each part of this volume are chronologically ordered. First Part of this book presents some theoretical advances on DSmT, dealing mainly with modified Proportional Conflict Redistribution Rules (PCR) of combination with degree of intersection, coarsening techniques, interval calculus for PCR thanks to set inversion via interval analysis (SIVIA), rough set classifiers, canonical decomposition of dichotomous belief functions, fast PCR fusion, fast inter-criteria analysis with PCR, and improved PCR5 and PCR6 rules preserving the (quasi-)neutrality of (quasi-)vacuous belief assignment in the fusion of sources of evidence with their Matlab codes. Because more applications of DSmT have emerged in the past years since the apparition of the fourth book of DSmT in 2015, the second part of this volume is about selected applications of DSmT mainly in building change detection, object recognition, quality of data association in tracking, perception in robotics, risk assessment for torrent protection and multi-criteria decision-making, multi-modal image fusion, coarsening techniques, recommender system, levee characterization and assessment, human heading perception, trust assessment, robotics, biometrics, failure detection, GPS systems, inter-criteria analysis, group decision, human activity recognition, storm prediction, data association for autonomous vehicles, identification of maritime vessels, fusion of support vector machines (SVM), Silx-Furtif RUST code library for information fusion including PCR rules, and network for ship classification. Finally, the third part presents interesting contributions related to belief functions in general published or presented along the years since 2015. These contributions are related with decision-making under uncertainty, belief approximations, probability transformations, new distances between belief functions, non-classical multi-criteria decision-making problems with belief functions, generalization of Bayes theorem, image processing, data association, entropy and cross-entropy measures, fuzzy evidence numbers, negator of belief mass, human activity recognition, information fusion for breast cancer therapy, imbalanced data classification, and hybrid techniques mixing deep learning with belief functions as well

Development of an unsupervised remote sensing methodology of detect surface leakage from terrestrial CO2 storage sites

Imperial Users onl

Advances in Detection and Classification of Underwater Targets using Synthetic Aperture Sonar Imagery

In this PhD thesis, the problem of underwater mine detection and classification using synthetic aperture sonar (SAS) imagery is considered. The automatic detection and automatic classification (ADAC) system is applied to images obtained by SAS systems. The ADAC system contains four steps, namely mine-like object (MLO) detection, image segmentation, feature extraction, and mine type classification. This thesis focuses on the last three steps. In the mine-like object detection step, a template-matching technique based on the a priori knowledge of mine shapes is applied to scan the sonar imagery for the detection of MLOs. Regions containing MLOs are called regions of interest (ROI). They are extracted and forwarded to the subsequent steps, i.e. image segmentation and feature extraction. In the image segmentation step, a modified expectation-maximization (EM) approach is proposed. For the sake of acquiring the shape information of the MLO in the ROI, the SAS images are segmented into highlights, shadows, and backgrounds. A generalized mixture model is adopted to approximate the statistics of the image data. In addition, a Dempster-Shafer theory-based clustering technique is used to consider the spatial correlation between pixels so that the clutters in background regions can be removed. Optimal parameter settings for the proposed EM approach are found with the help of quantitative numerical studies. In the feature extraction step, features are extracted and will be used as the inputs for the mine type classification step. Both the geometrical features and the texture features are applied. However, there are numerous features proposed to describe the object shape and the texture in the literature. Due to the curse of dimensionality, it is indispensable to do the feature selection during the design of an ADAC system. A sophisticated filter method is developed to choose optimal features for the classification purpose. This filter method utilizes a novel feature relevance measure that is a combination of the mutual information, the modified Relief weight, and the Shannon entropy. The selected features demonstrate a higher generalizability. Compared with other filter methods, the features selected by our method can lead to superior classification accuracy, and their performance variation over different classifiers is decreased. In the mine type classification step, the prediction of the types of MLO is considered. In order to take advantage of the complementary information among different classifiers, a classifier combination scheme is developed in the framework of the Dempster-Shafer theory. The outputs of individual classifiers are combined according to this classifier combination scheme. The resulting classification accuracy is better than those of individual classifiers. All of the proposed methods are evaluated using SAS data. Finally, conclusions are drawn, and some suggestions about future works are proposed as well

Representing archaeological uncertainty in cultural informatics

This thesis sets out to explore, describe, quantify, and visualise uncertainty in a cultural informatics context, with a focus on archaeological reconstructions. For quite some time, archaeologists and heritage experts have been criticising the often toorealistic appearance of three-dimensional reconstructions. They have been highlighting one of the unique features of archaeology: the information we have on our heritage will always be incomplete. This incompleteness should be reflected in digitised reconstructions of the past. This criticism is the driving force behind this thesis. The research examines archaeological theory and inferential process and provides insight into computer visualisation. It describes how these two areas, of archaeology and computer graphics, have formed a useful, but often tumultuous, relationship through the years. By examining the uncertainty background of disciplines such as GIS, medicine, and law, the thesis postulates that archaeological visualisation, in order to mature, must move towards archaeological knowledge visualisation. Three sequential areas are proposed through this thesis for the initial exploration of archaeological uncertainty: identification, quantification and modelling. The main contributions of the thesis lie in those three areas. Firstly, through the innovative design, distribution, and analysis of a questionnaire, the thesis identifies the importance of uncertainty in archaeological interpretation and discovers potential preferences among different evidence types. Secondly, the thesis uniquely analyses and evaluates, in relation to archaeological uncertainty, three different belief quantification models. The varying ways that these mathematical models work, are also evaluated through simulated experiments. Comparison of results indicates significant convergence between the models. Thirdly, a novel approach to archaeological uncertainty and evidence conflict visualisation is presented, influenced by information visualisation schemes. Lastly, suggestions for future semantic extensions to this research are presented through the design and development of new plugins to a search engine

Proceedings of the 2019 Joint Workshop of Fraunhofer IOSB and Institute for Anthropomatics, Vision and Fusion Laboratory

In 2019 again, the annual joint workshop of the Fraunhofer IOSB and the Vision and Fusion Laboratory of the Karlsruhe Institute of Technology took place. The doctoral students of both institutions presented extensive reports on the status of their research and discussed topics ranging from computer vision and optical metrology to network security, usage control and machine learning. The results and ideas presented at the workshop are collected in this book in the form of technical reports