Target classification in multimodal video

The presented thesis focuses on enhancing scene segmentation and target recognition methodologies via the mobilisation of contextual information. The algorithms developed to achieve this goal utilise multi-modal sensor information collected across varying scenarios, from controlled indoor sequences to challenging rural locations. Sensors are chiefly colour band and long wave infrared (LWIR), enabling persistent surveillance capabilities across all environments. In the drive to develop effectual algorithms towards the outlined goals, key obstacles are identified and examined: the recovery of background scene structure from foreground object ’clutter’, employing contextual foreground knowledge to circumvent training a classifier when labeled data is not readily available, creating a labeled LWIR dataset to train a convolutional neural network (CNN) based object classifier and the viability of spatial context to address long range target classification when big data solutions are not enough. For an environment displaying frequent foreground clutter, such as a busy train station, we propose an algorithm exploiting foreground object presence to segment underlying scene structure that is not often visible. If such a location is outdoors and surveyed by an infra-red (IR) and visible band camera set-up, scene context and contextual knowledge transfer allows reasonable class predictions for thermal signatures within the scene to be determined. Furthermore, a labeled LWIR image corpus is created to train an infrared object classifier, using a CNN approach. The trained network demonstrates effective classification accuracy of 95% over 6 object classes. However, performance is not sustainable for IR targets acquired at long range due to low signal quality and classification accuracy drops. This is addressed by mobilising spatial context to affect network class scores, restoring robust classification capability

Hybrid intelligent decision support system for distributed detection based on ad hoc integrated WSN & RFID

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonThe real time monitoring of environment context aware activities, based on distributed detection, is becoming a standard in public safety and service delivery in a wide range of domains (child and elderly care and supervision, logistics, circulation, and other). The safety of people, goods and premises depends on the prompt immediate reaction to potential hazards identified in real time, at an early stage to engage appropriate control actions. Effective emergency response can be supported only by available and acquired expertise or elaborate collaborative knowledge in the domain of distributed detection that include indoor sensing, tracking and localizing. This research proposes a hybrid conceptual multi-agent framework for the acquisition of collaborative knowledge in dynamic complex context aware environments for distributed detection. This framework has been applied for the design and development of a hybrid intelligent multi-agent decision system (HIDSS) that supports a decentralized active sensing, tracking and localizing strategy, and the deployment and configuration of smart detection devices associated to active sensor nodes wirelessly connected in a network topology to configure, deploy and control ad hoc wireless sensor networks (WSNs). This system, which is based on the interactive use of data, models and knowledge base, has been implemented to support fire detection and control access fusion functions aimed at elaborating: An integrated data model, grouping the building information data and WSN-RFID database, composed of the network configuration and captured data, A virtual layout configuration of the controlled premises, based on using a building information model, A knowledge-based support for the design of generic detection devices, A multi-criteria decision making model for generic detection devices distribution, ad hoc WSNs configuration, clustering and deployment, and Predictive data models for evacuation planning, and fire and evacuation simulation. An evaluation of the system prototype has been carried out to enrich information and knowledge fusion requirements and show the scope of the concepts used in data and process modelling. It has shown the practicability of hybrid solutions grouping generic homogeneous smart detection devices enhanced by heterogeneous support devices in their deployment, forming ad hoc networks that integrate WSNs and radio frequency identification (RFID) technology. The novelty in this work is the web-based support system architecture proposed in this framework that is based on the use of intelligent agent modelling and multi-agent systems, and the decoupling of the processes supporting the multi-sensor data fusion from those supporting different context applications. Although this decoupling is essential to appropriately distribute the different fusion functions, the integration of several dimensions of policy settings for the modelling of knowledge processes, and intelligent and pro-active decision making activities, requires the organisation of interactive fusion functions deployed upstream to a safety and emergency response.Saudi government, represented by the Ministry of Interior and General Directorate of Civil Defenc

Generative Models for Novelty Detection Applications in abnormal event and situational changedetection from data series

Novelty detection is a process for distinguishing the observations that differ in some respect from the observations that the model is trained on. Novelty detection is one of the fundamental requirements of a good classification or identification system since sometimes the test data contains observations that were not known at the training time. In other words, the novelty class is often is not presented during the training phase or not well defined. In light of the above, one-class classifiers and generative methods can efficiently model such problems. However, due to the unavailability of data from the novelty class, training an end-to-end model is a challenging task itself. Therefore, detecting the Novel classes in unsupervised and semi-supervised settings is a crucial step in such tasks. In this thesis, we propose several methods to model the novelty detection problem in unsupervised and semi-supervised fashion. The proposed frameworks applied to different related applications of anomaly and outlier detection tasks. The results show the superior of our proposed methods in compare to the baselines and state-of-the-art methods

Intelligent Sensor Networks

In the last decade, wireless or wired sensor networks have attracted much attention. However, most designs target general sensor network issues including protocol stack (routing, MAC, etc.) and security issues. This book focuses on the close integration of sensing, networking, and smart signal processing via machine learning. Based on their world-class research, the authors present the fundamentals of intelligent sensor networks. They cover sensing and sampling, distributed signal processing, and intelligent signal learning. In addition, they present cutting-edge research results from leading experts