Human and Group Activity Recognition from Video Sequences

A good solution to human activity recognition enables the creation of a wide variety of useful applications such as applications in visual surveillance, vision-based Human-Computer-Interaction (HCI) and gesture recognition. In this thesis, a graph based approach to human activity recognition is proposed which models spatio-temporal features as contextual space-time graphs. In this method, spatio-temporal gradient cuboids were extracted at significant regions of activity, and feature graphs (gradient, space-time, local neighbours, immediate neighbours) are constructed using the similarity matrix. The Laplacian representation of the graph is utilised to reduce the computational complexity and to allow the use of traditional statistical classifiers. A second methodology is proposed to detect and localise abnormal activities in crowded scenes. This approach has two stages: training and identification. During the training stage, specific human activities are identified and characterised by employing modelling of medium-term movement flow through streaklines. Each streakline is formed by multiple optical flow vectors that represent and track locally the movement in the scene. A dictionary of activities is recorded for a given scene during the training stage. During the testing stage, the consistency of each observed activity with those from the dictionary is verified using the Kullback-Leibler (KL) divergence. The anomaly detection of the proposed methodology is compared to state of the art, producing state of the art results for localising anomalous activities. Finally, we propose an automatic group activity recognition approach by modelling the interdependencies of group activity features over time. We propose to model the group interdependences in both motion and location spaces. These spaces are extended to time-space and time-movement spaces and modelled using Kernel Density Estimation (KDE). The recognition performance of the proposed methodology shows an improvement in recognition performance over state of the art results on group activity datasets

Algorithms for anomaly detection in video sequences through discriminative models

Monitoring public areas with pedestrians is a task that has to be frequently accomplished by means of security systems. Nevertheless, manual detection of these anomalies is a tough task and it is easy to lose interesting events when many areas have to be attended. This is the main reason why the automated detection of these anomalies and interesting events in general has become an important source of research in the past years, specially in the eld of computer vision. Automated anomaly detection is still an open task even though that many methods have been proposed. One of the reasons is that a successful and accurate anomaly detection algorithm strongly depends on the context and the de nition of the anomalies to detect and the objects that produce them. The state of the art included in this work has been developed to make a complete study of all these aspects in detail, as well as a study of advantages and drawbacks of the main methods of the literature, helping to choose the best techniques and strategies for speci c surveillance scenarios. Since there is a great di culty to model every anomaly, we have decided to fashion the normality by means of Gaussian mixture models, which are relatively simple methods compared to others in the literature such as [1, 2], but that have shown potential at detecting anomalies. This can be observed on the methods proposed in [3] and [4]. We have decided to work at pixel level. Thus, to feed the model, discriminative descriptors are built based on a robust optical ow method, that has become the main source of motion and textural information of the scene. This fact makes this work di erent to other state-of-the-art approaches that work at pixel-level, whose optical ow is not capable to give such a detailed information of the scene. Finally, the evaluation of the nal algorithm is performed exhaustively from a baseline method, whose descriptor grows depending on the best results so far on a publicly available dataset. Detection results are compared with the state-of-the-art methods, concluding that our method is at the same level of the methods proposed in the literature

Video based detection of normal and anomalous behaviour of individuals

This PhD research has proposed novel computer vision and machine learning algorithms for the problem of video based anomalous event detection of individuals. Varieties of Hidden Markov Models were designed to model the temporal and spatial causalities of crowd behaviour. A Markov Random Field on top of a Gaussian Mixture Model is proposed to incorporate spatial context information during classification. Discriminative conditional random field methods are also proposed. Novel features are proposed to extract motion and appearance information. Most of the proposed approaches comprehensively outperform other techniques on publicly available datasets during the time of publications originating from the results