Generative Models for Novelty Detection Applications in abnormal event and situational changedetection from data series

Novelty detection is a process for distinguishing the observations that differ in some respect from the observations that the model is trained on. Novelty detection is one of the fundamental requirements of a good classification or identification system since sometimes the test data contains observations that were not known at the training time. In other words, the novelty class is often is not presented during the training phase or not well defined. In light of the above, one-class classifiers and generative methods can efficiently model such problems. However, due to the unavailability of data from the novelty class, training an end-to-end model is a challenging task itself. Therefore, detecting the Novel classes in unsupervised and semi-supervised settings is a crucial step in such tasks. In this thesis, we propose several methods to model the novelty detection problem in unsupervised and semi-supervised fashion. The proposed frameworks applied to different related applications of anomaly and outlier detection tasks. The results show the superior of our proposed methods in compare to the baselines and state-of-the-art methods

Towards Intelligent Crowd Behavior Understanding through the STFD Descriptor Exploration

Realizing the automated and online detection of crowd anomalies from surveillance CCTVs is a research-intensive and application-demanding task. This research proposes a novel technique for detecting crowd abnormalities through analyzing the spatial and temporal features of input video signals. This integrated solution defines an image descriptor (named spatio-temporal feature descriptor - STFD) that reflects the global motion information of crowds over time. A CNN has then been adopted to classify dominant or large-scale crowd abnormal behaviors. The work reported has focused on: 1) detecting moving objects in online (or near real-time) manner through spatio-temporal segmentations of crowds that is defined by the similarity of group trajectory structures in temporal space and the foreground blocks based on Gaussian Mixture Model (GMM) in spatial space; 2) dividing multiple clustered groups based on the spectral clustering method by considering image pixels from spatio-temporal segmentation regions as dynamic particles; 3) generating the STFD descriptor instances by calculating the attributes (i.e., collectiveness, stability, conflict and crowd density) of particles in the corresponding groups; 4) inputting generated STFD descriptor instances into the devised convolutional neural network (CNN) to detect suspicious crowd behaviors. The test and evaluation of the devised models and techniques have selected the PETS database as the primary experimental data sets. Results against benchmarking models and systems have shown promising advancements of this novel approach in terms of accuracy and efficiency for detecting crowd anomalies

Towards Intelligent Crowd Behavior Understanding through the STFD Descriptor Exploration

Realizing the automated and online detection of crowd anomalies from surveillance CCTVs is a research-intensive and application-demanding task. This research proposes a novel technique for detecting crowd abnormalities through analyzing the spatial and temporal features of input video signals. This integrated solution defines an image descriptor (named spatio-temporal feature descriptor - STFD) that reflects the global motion information of crowds over time. A CNN has then been adopted to classify dominant or large-scale crowd abnormal behaviors. The work reported has focused on: 1) detecting moving objects in online (or near real-time) manner through spatio-temporal segmentations of crowds that is defined by the similarity of group trajectory structures in temporal space and the foreground blocks based on Gaussian Mixture Model (GMM) in spatial space; 2) dividing multiple clustered groups based on the spectral clustering method by considering image pixels from spatio-temporal segmentation regions as dynamic particles; 3) generating the STFD descriptor instances by calculating the attributes (i.e., collectiveness, stability, conflict and crowd density) of particles in the corresponding groups; 4) inputting generated STFD descriptor instances into the devised convolutional neural network (CNN) to detect suspicious crowd behaviors. The test and evaluation of the devised models and techniques have selected the PETS database as the primary experimental data sets. Results against benchmarking models and systems have shown promising advancements of this novel approach in terms of accuracy and efficiency for detecting crowd anomalies

CENTRIST3D : um descritor espaço-temporal para detecção de anomalias em vídeos de multidões

Orientador: Hélio PedriniDissertação (mestrado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: O campo de estudo da detecção de anomalias em multidões possui uma vasta gama de aplicações, podendo-se destacar o monitoramento e vigilância de áreas de interesse, tais como aeroportos, bancos, parques, estádios e estações de trens, como uma das mais importantes. Em geral, sistemas de vigilância requerem prossionais qualicados para assistir longas gravações à procura de alguma anomalia, o que demanda alta concentração e dedicação. Essa abordagem tende a ser ineciente, pois os seres humanos estão sujeitos a falhas sob condições de fadiga e repetição devido aos seus próprios limites quanto à capacidade de observação e seu desempenho está diretamente ligado a fatores físicos e psicológicos, os quais podem impactar negativamente na qualidade de reconhecimento. Multidões tendem a se comportar de maneira complexa, possivelmente mudando de orientação e velocidade rapidamente, bem como devido à oclusão parcial ou total. Consequentemente, técnicas baseadas em rastreamento de pedestres ou que dependam de segmentação de fundo geralmente apresentam maiores taxas de erros. O conceito de anomalia é subjetivo e está sujeito a diferentes interpretações, dependendo do contexto da aplicação. Neste trabalho, duas contribuições são apresentadas. Inicialmente, avaliamos a ecácia do descritor CENsus TRansform hISTogram (CENTRIST), originalmente utilizado para categorização de cenas, no contexto de detecção de anomalias em multidões. Em seguida, propusemos o CENTRIST3D, uma versão modicada do CENTRIST que se utiliza de informações espaço-temporais para melhorar a discriminação dos eventos anômalos. Nosso método cria histogramas de características espaço-temporais de quadros de vídeos sucessivos, os quais foram divididos hierarquicamente utilizando um algoritmo modicado da correspondência em pirâmide espacial. Os resultados foram validados em três bases de dados públicas: University of California San Diego (UCSD) Anomaly Detection Dataset, Violent Flows Dataset e University of Minesota (UMN) Dataset. Comparado com outros trabalhos da literatura, CENTRIST3D obteve resultados satisfatórios nas bases Violent Flows e UMN, mas um desempenho abaixo do esperado na base UCSD, indicando que nosso método é mais adequado para cenas com mudanças abruptas em movimento e textura. Por m, mostramos que há evidências de que o CENTRIST3D é um descritor eciente de ser computado, sendo facilmente paralelizável e obtendo uma taxa de quadros por segundo suciente para ser utilizado em aplicações de tempo realAbstract: Crowd abnormality detection is a eld of study with a wide range of applications, where surveillance of interest areas, such as airports, banks, parks, stadiums and subways, is one of the most important purposes. In general, surveillance systems require well-trained personnel to watch video footages in order to search for abnormal events. Moreover, they usually are dependent on human operators, who are susceptible to failure under stressful and repetitive conditions. This tends to be an ineective approach since humans have their own natural limits of observation and their performance is tightly related to their physical and mental state, which might aect the quality of surveillance. Crowds tend to be complex, subject to subtle changes in motion and to partial or total occlusion. Consequently, approaches based on individual pedestrian tracking and background segmentation may suer in quality due to the aforementioned problems. Anomaly itself is a subjective concept, since it depends on the context of the application. Two main contributions are presented in this work. We rst evaluate the eectiveness of the CENsus TRansform hISTogram (CENTRIST) descriptor, initially designed for scene categorization, in crowd abnormality detection. Then, we propose the CENTRIST3D descriptor, a spatio-temporal variation of CENTRIST. Our method creates a histogram of spatiotemporal features from successive frames by extracting histograms of Volumetric Census Transform from a spatial representation using a modied Spatial Pyramid Matching algorithm. Additionally, we test both descriptors in three public data collections: UCSD Anomaly Detection Dataset, Violent Flows Dataset, and UMN Datasets. Compared to other works of the literature, CENTRIST3D achieved satisfactory accuracy rates on both Violent Flows and UMN Datasets, but poor performance on the UCSD Dataset, indicating that our method is more suitable to scenes with fast changes in motion and texture. Finally, we provide evidence that CENTRIST3D is an ecient descriptor to be computed, since it requires little computational time, is easily parallelizable and achieves suitable frame-per-second rates to be used in real-time applicationsMestradoCiência da ComputaçãoMestre em Ciência da Computação1406874159166/2015-2CAPESCNP

Human Tracking and Activity Recognition for Surveillence Applications

Tracking and study of behavioural changes of human beings through vision is a challenging task. For surveillance, automated systems are important which can observe the traffic and can detect the abnormality. For tracking human or any kind of object, colour feature based mean shift technique is widely used. This technique uses Bhattacharya coefficient to locate the object based on the maximisation of the similarity function between object model and candidate model. Traditional mean shift algorithm fails when the object having large motion, occlusion, corrupted frames etc. In addition to that, the technique is not automatic to initiate the tracking. To overcome all these problems, this thesis work proposed a technique which uses three additional modules to the traditional method to make it more efficient. The proposed modules used human detection by modelling through star skeletonization, followed by block search algorithm and occlusion handling. Block search algorithm helped to supply an overlapping area to candidate model to continue the track when tracking fails due to fast motion. Occlusion handling helped in initiating the tracking after prolonged period of occlusion. The proposed method has been tested on real time data and it outperforms the conventional method effectively to overcome the mentioned problems up to large extent. Human activity recognition is a hierarchical procedure which confirms abnormality step by step. Low level activity recognition is a trajectory based application in which trajectory of tracks of a human being helps to detect the abnormal events like person fell down, illegal entry, abnormal loitering, line formation etc. At high level, human pose will be detected by the help of shape based human pose detection. The main aim of the system is to make a person independent real-time human activity recognition with decreased false alarm rates