Novelty detection in video surveillance using hierarchical neural networks

Abstract. A hierarchical self-organising neural network is described for the detection of unusual pedestrian behaviour in video-based surveillance systems. The system is trained on a normal data set, with no prior information about the scene under surveillance, thereby requiring minimal user input. Nodes use a trace activation rule and feedforward connections, modified so that higher layer nodes are sensitive to trajectory segments traced across the previous layer. Top layer nodes have binary lateral connections and corresponding “novelty accumulator” nodes. Lateral connections are set between co-occurring nodes, generating a signal to prevent accumulation of the novelty measure along normal sequences. In abnormal sequences the novelty accumulator nodes are allowed to increase their activity, generating an alarm state

Hierarchical self organizing map and focusing inspection strategy for mobile robot novelty detection

Novelty detection is a process of recognizing changes based on learned knowledge. In this research, a novelty detection system was implemented on a mobile robot with an array of sonar sensors for surveillance application. In order to perform novelty detection, a map that stores normal information with respect to any particular robot pose in an environment is required. The map is needed to detect changes and determine the position of novel event. The challenges of mobile novelty detection system are that the false positive rate is usually high whereas the true positive rate is usually low due to mapping and monitoring problems. During mapping, errors due to robot localization and sensor measurement can reduce the quality of the map built. However, available methods in mapping assume perfect localization, hence error in localization is not taken into account in the process of mapping. During monitoring, inspection interval that is too small will consume a lot of time and energy but if the interval is too big, novelty could be missed, hence lower the true positive detection. On top of that, low true positive detection is also caused by the low reliability of sonar sensor measurement. Thus, the objective of this thesis is to utilize mobile novelty detection system by developing a mapping and monitoring strategy that has low false positive detection, high true positive detection and able to estimate the position of a novelty. This thesis proposed two methods regarding to mapping and monitoring process; a hierarchical Self Organizing Map (SOM) and a Focusing Inspection Strategy (FIS). Unlike other mapping methods, hierarchical SOM also consider localization error when associating the normal information with respect to the robot pose. FIS is a multi resolution monitoring strategy which works by changing the frequency of measurement depending on the detection of anomaly. In this thesis, two models were considered; a step (FS) and linear (FL) resolution models. The hierarchical SOM was validated by using simulation and experimentation of the inspection in environment with normal and novel event. False positive rate is measured to determine the map performance. The results show that hierarchical SOM is able to map the normal condition of the environment very well. The inspection results show the false positive rate occurred less than 0.1 at the higher sensitivity setting of 0.9 in either normal or novel condition. The performance of FIS was investigated by using experimentation of the inspection of novel objects of different sizes. The results show that by changing the frequency of measurement using the FS and FL models, the number of true positive detection increases up to 80% when compared to inspection with fix measurement frequency. FIS also reduced the error of position estimation by about 8.8% and 10.9% each for FS and FL and maintained the false positive rate lower than 0.1

A Neural System for Automated CCTV Surveillance

This paper overviews a new system, the “Owens Tracker,” for automated identification of suspicious pedestrian activity in a car-park. Centralized CCTV systems relay multiple video streams to a central point for monitoring by an operator. The operator receives a continuous stream of information, mostly related to normal activity, making it difficult to maintain concentration at a sufficiently high level. While it is difficult to place quantitative boundaries on the number of scenes and time period over which effective monitoring can be performed, Wallace and Diffley [1] give some guidance, based on empirical and anecdotal evidence, suggesting that the number of cameras monitored by an operator be no greater than 16, and that the period of effective monitoring may be as low as 30 minutes before recuperation is required. An intelligent video surveillance system should therefore act as a filter, censuring inactive scenes and scenes showing normal activity. By presenting the operator only with unusual activity his/her attention is effectively focussed, and the ratio of cameras to operators can be increased. The Owens Tracker learns to recognize environmentspecific normal behaviour, and refers sequences of unusual behaviour for operator attention. The system was developed using standard low-resolution CCTV cameras operating in the car-parks of Doxford Park Industrial Estate (Sunderland, Tyne and Wear), and targets unusual pedestrian behaviour. The modus operandi of the system is to highlight excursions from a learned model of normal behaviour in the monitored scene. The system tracks objects and extracts their centroids; behaviour is defined as the trajectory traced by an object centroid; normality as the trajectories typically encountered in the scene. The essential stages in the system are: segmentation of objects of interest; disambiguation and tracking of multiple contacts, including the handling of occlusion and noise, and successful tracking of objects that “merge” during motion; identification of unusual trajectories. These three stages are discussed in more detail in the following sections, and the system performance is then evaluated

A survey of outlier detection methodologies

Outlier detection has been used for centuries to detect and, where appropriate, remove anomalous observations from data. Outliers arise due to mechanical faults, changes in system behaviour, fraudulent behaviour, human error, instrument error or simply through natural deviations in populations. Their detection can identify system faults and fraud before they escalate with potentially catastrophic consequences. It can identify errors and remove their contaminating effect on the data set and as such to purify the data for processing. The original outlier detection methods were arbitrary but now, principled and systematic techniques are used, drawn from the full gamut of Computer Science and Statistics. In this paper, we introduce a survey of contemporary techniques for outlier detection. We identify their respective motivations and distinguish their advantages and disadvantages in a comparative review

Applied Sensor Fault Detection, Identification and Data Reconstruction

Sensor fault detection and identification (SFD/I) has attracted considerable attention in military applications, especially when safety- or mission-critical issues are of paramount importance. Here, two readily implementable approaches for SFD/I are proposed through hierarchical clustering and self-organizing map neural networks. The proposed methodologies are capable of detecting sensor faults from a large group of sensors measuring different physical quantities and achieve SFD/I in a single stage. Furthermore, it is possible to reconstruct the measurements expected from the faulted sensor and thereby facilitate improved unit availability. The efficacy of the proposed approaches is demonstrated through the use of measurements from experimental trials on a gas turbine. Ultimately, the underlying principles are readily transferable to other complex industrial and military systems

Hybrid HC-PAA-G3K for novelty detection on industrial systems

Piecewise aggregate approximation (PAA) provides a powerful yet computationally efficient tool for dimensionality reduction and feature extraction. A new distance-based hierarchical clustering (HC) is now proposed to adjust the PAA segment frame sizes. The proposed hybrid HC-PAA is validated by a generic clustering method ‘G3Kmeans’ (G3K). The efficacy of the hybrid HC-PAA-G3K methodology is demonstrated using an application case study based on novelty detection on industrial gas turbines. Results show the hybrid HC-PAA provides improved performance with regard to cluster separation, compared to traditional PAA. The proposed method therefore provides a robust algorithm for feature extraction and novelty detection. There are two main contributions of the paper: 1) application of HC to modify conventional PAA segment frame size; 2) introduction of ‘G3Kmeans’ to improve the performance of the traditional K-means clustering methods