Abnormal crowd behavior detection using novel optical flow-based features

In this paper, we propose a novel optical flow based features for abnormal crowd behaviour detection. The proposed feature is mainly based on the angle difference computed between the optical flow vectors in the current frame and in the previous frame at each pixel location. The angle difference information is also combined with the optical flow magnitude to produce new, effective and direction invariant event features. A one-class SVM is utilized to learn normal crowd behavior. If a test sample deviates significantly from the normal behavior, it is detected as abnormal crowd behavior. Although there are many optical flow based features for crowd behaviour analysis, this is the first time the angle difference between optical flow vectors in the current frame and in the previous frame is considered as a anomaly feature. Evaluations on UMN and PETS2009 datasets show that the proposed method performs competitive results compared to the state-of-the-art methods

A Survey on Unusual Event Detection in Videos

As the usage of CCTV cameras in outdoor and indoor locations has increased significantly, one needs to design a system to detect the unusual events, at the time of its occurrence. Computer vision is used for Human Action recognition, which has been widely implemented in the systems, but unusual event detection is lately entering into the limelight. In order to detect the unusual events, supervised techniques, semi-supervised techniques and unsupervised techniques have been adopted. Social force model (SFM) and Force field are used to model the interaction among crowds. Only normal events training samples is not sufficient for detection of unusual events. Double sparse representation has been used as a solution to this, which includes normal and abnormal training data. To develop an intelligent video surveillance system, behavioural representation and behavioural modelling techniques are used. Various machine learning techniques to identify unusual events include: Graph modelling and matching, object trajectory based, object silhouettes based and pixel based approaches. Kullback–Leibler (KL) divergence, Quaternion Discrete Cosine Transformation (QDCT) analysis, hidden Markov model (HMM) and histogram of oriented contextual gradient (HOCG) descriptor are some of the models used are used for detecting unusual events. This paper briefly discusses the above mentioned strategies and pay attention on their pros and cons

An Approach to detecting crowd anomalies for entrance and checkpoint security

This thesis develops an approach for detecting behavioral anomalies using tracks of pedestrians, including specified threat tracks. The application area is installation security with focus on monitoring the entrances of these installations. The approach specifically allows operator interaction to specify threats and to interactively adjust the system parameters depending on the context of the situation. This research has discovered physically meaningful features that are developed and organized in a manner so that features can be systematically added or deleted depending on the situation and operator preference. The features can be used with standard classifiers such as the one class support vector machine that is used in this research. The one class support vector machine is very stable for this application and provides significant insight into the nature of its decision boundary. Its stability and ease of system use stems from a unique automatic tuning approach that is computationally efficient and compares favorable with competing approaches. This automatic tuning approach is believed to be novel and was developed as part of this research. Results are provided using both measured and synthetic data

Detecção de eventos complexos em vídeos baseada em ritmos visuais

Orientador: Hélio PedriniDissertação (mestrado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: O reconhecimento de eventos complexos em vídeos possui várias aplicações práticas relevantes, alavancadas pela grande disponibilidade de câmeras digitais instaladas em aeroportos, estações de ônibus e trens, centros de compras, estádios, hospitais, escolas, prédios, estradas, entre vários outros locais. Avanços na tecnologia digital têm aumentado as capacidades dos sistemas em reconhecer eventos em vídeos por meio do desenvolvimento de dispositivos com alta resolução, dimensões físicas pequenas e altas taxas de amostragem. Muitos trabalhos disponíveis na literatura têm explorado o tema a partir de diferentes pontos de vista. Este trabalho apresenta e avalia uma metodologia para extrair características dos ritmos visuais no contexto de detecção de eventos em vídeos. Um ritmo visual pode ser visto com a projeção de um vídeo em uma imagem, tal que a tarefa de análise de vídeos é reduzida a um problema de análise de imagens, beneficiando-se de seu baixo custo de processamento em termos de tempo e complexidade. Para demonstrar o potencial do ritmo visual na análise de vídeos complexos, três problemas da área de visão computacional são selecionados: detecção de eventos anômalos, classificação de ações humanas e reconhecimento de gestos. No primeiro problema, um modelo e? aprendido com situações de normalidade a partir dos rastros deixados pelas pessoas ao andar, enquanto padro?es representativos das ações são extraídos nos outros dois problemas. Nossa hipo?tese e? de que vídeos similares produzem padro?es semelhantes, tal que o problema de classificação de ações pode ser reduzido a uma tarefa de classificação de imagens. Experimentos realizados em bases públicas de dados demonstram que o método proposto produz resultados promissores com baixo custo de processamento, tornando-o possível aplicar em tempo real. Embora os padro?es dos ritmos visuais sejam extrai?dos como histograma de gradientes, algumas tentativas para adicionar características do fluxo o?tico são discutidas, além de estratégias para obter ritmos visuais alternativosAbstract: The recognition of complex events in videos has currently several important applications, particularly due to the wide availability of digital cameras in environments such as airports, train and bus stations, shopping centers, stadiums, hospitals, schools, buildings, roads, among others. Moreover, advances in digital technology have enhanced the capabilities for detection of video events through the development of devices with high resolution, small physical size, and high sampling rates. Many works available in the literature have explored the subject from different perspectives. This work presents and evaluates a methodology for extracting a feature descriptor from visual rhythms of video sequences in order to address the video event detection problem. A visual rhythm can be seen as the projection of a video onto an image, such that the video analysis task can be reduced into an image analysis problem, benefiting from its low processing cost in terms of time and complexity. To demonstrate the potential of the visual rhythm in the analysis of complex videos, three computer vision problems are selected in this work: abnormal event detection, human action classification, and gesture recognition. The former problem learns a normalcy model from the traces that people leave when they walk, whereas the other two problems extract representative patterns from actions. Our hypothesis is that similar videos produce similar patterns, therefore, the action classification problem is reduced into an image classification task. Experiments conducted on well-known public datasets demonstrate that the method produces promising results at high processing rates, making it possible to work in real time. Even though the visual rhythm features are mainly extracted as histogram of gradients, some attempts for adding optical flow features are discussed, as well as strategies for obtaining alternative visual rhythmsMestradoCiência da ComputaçãoMestre em Ciência da Computação1570507, 1406910, 1374943CAPE

Audio-visual multi-modality driven hybrid feature learning model for crowd analysis and classification

The high pace emergence in advanced software systems, low-cost hardware and decentralized cloud computing technologies have broadened the horizon for vision-based surveillance, monitoring and control. However, complex and inferior feature learning over visual artefacts or video streams, especially under extreme conditions confine majority of the at-hand vision-based crowd analysis and classification systems. Retrieving event-sensitive or crowd-type sensitive spatio-temporal features for the different crowd types under extreme conditions is a highly complex task. Consequently, it results in lower accuracy and hence low reliability that confines existing methods for real-time crowd analysis. Despite numerous efforts in vision-based approaches, the lack of acoustic cues often creates ambiguity in crowd classification. On the other hand, the strategic amalgamation of audio-visual features can enable accurate and reliable crowd analysis and classification. Considering it as motivation, in this research a novel audio-visual multi-modality driven hybrid feature learning model is developed for crowd analysis and classification. In this work, a hybrid feature extraction model was applied to extract deep spatio-temporal features by using Gray-Level Co-occurrence Metrics (GLCM) and AlexNet transferrable learning model. Once extracting the different GLCM features and AlexNet deep features, horizontal concatenation was done to fuse the different feature sets. Similarly, for acoustic feature extraction, the audio samples (from the input video) were processed for static (fixed size) sampling, pre-emphasis, block framing and Hann windowing, followed by acoustic feature extraction like GTCC, GTCC-Delta, GTCC-Delta-Delta, MFCC, Spectral Entropy, Spectral Flux, Spectral Slope and Harmonics to Noise Ratio (HNR). Finally, the extracted audio-visual features were fused to yield a composite multi-modal feature set, which is processed for classification using the random forest ensemble classifier. The multi-class classification yields a crowd-classification accurac12529y of (98.26%), precision (98.89%), sensitivity (94.82%), specificity (95.57%), and F-Measure of 98.84%. The robustness of the proposed multi-modality-based crowd analysis model confirms its suitability towards real-world crowd detection and classification tasks

Physical crowds and psychological crowds: applying self-categorization theory to computer simulation of collective behaviour

Computer models are used to simulate pedestrian behaviour for safety at mass events. Previous research has indicated differences between physical crowds of co-present individuals, and psychological crowds who mobilise collective behaviour through a shared social identity. This thesis aimed to examine the assumptions models use about crowds, conduct two studies of crowd movement to ascertain the behavioural signatures of psychological crowds, and implement these into a theoretically-driven model of crowd behaviour. A systematic review of crowd modelling literature is presented which explores the assumptions about crowd behaviour being used in current models. This review demonstrates that models portray the crowd as either an identical mass with no inter-personal connections, unique individuals with no connections to others, or as small groups within a crowd. Thus, no models have incorporated the role of self-categorisation theory needed to simulate collective behaviour. The empirical research in this thesis aimed to determine the behavioural effects of self-categorisation on pedestrian movement. Findings from a first study illustrate that, in comparison to a physical crowd, perception of shared social identities in the psychological crowd motivated participants to maintain close proximity with ingroup members through regulation of their speed and distance walked. A second study showed that collective self-organisation seemed to be increased by the presence of an outgroup, causing ingroup members to tighten formation to avoid splitting up. Finally, a computer model is presented which implements the quantified behavioural effects of self-categorisation found in the behavioural studies. A self-categorisation parameter is introduced to simulate ingroup members self-organising to remain together. This is compared to a physical crowd simulation with group identities absent. The results demonstrate that the self-categorisation parameter provides more accurate simulation of psychological crowd behaviour. Thus, it is argued that models should implement self-categorisation into simulations of psychological crowds to increase safety at mass events