An effective video processing pipeline for crowd pattern analysis

With the purpose of automatic detection of crowd patterns including abrupt and abnormal changes, a novel approach for extracting motion “textures” from dynamic Spatio-Temporal Volume (STV) blocks formulated by live video streams has been proposed. This paper starts from introducing the common approach for STV construction and corresponding Spatio-Temporal Texture (STT) extraction techniques. Next the crowd motion information contained within the random STT slices are evaluated based on the information entropy theory to cull the static background and noises occupying most of the STV spaces. A preprocessing step using Gabor filtering for improving the STT sampling efficiency and motion fidelity has been devised and tested. The technique has been applied on benchmarking video databases for proof-of-concept and performance evaluation. Preliminary results have shown encouraging outcomes and promising potentials for its real-world crowd monitoring and control applications

STV-based Video Feature Processing for Action Recognition

In comparison to still image-based processes, video features can provide rich and intuitive information about dynamic events occurred over a period of time, such as human actions, crowd behaviours, and other subject pattern changes. Although substantial progresses have been made in the last decade on image processing and seen its successful applications in face matching and object recognition, video-based event detection still remains one of the most difficult challenges in computer vision research due to its complex continuous or discrete input signals, arbitrary dynamic feature definitions, and the often ambiguous analytical methods. In this paper, a Spatio-Temporal Volume (STV) and region intersection (RI) based 3D shape-matching method has been proposed to facilitate the definition and recognition of human actions recorded in videos. The distinctive characteristics and the performance gain of the devised approach stemmed from a coefficient factor-boosted 3D region intersection and matching mechanism developed in this research. This paper also reported the investigation into techniques for efficient STV data filtering to reduce the amount of voxels (volumetric-pixels) that need to be processed in each operational cycle in the implemented system. The encouraging features and improvements on the operational performance registered in the experiments have been discussed at the end

Crowd Abnormal Behaviour Detection and Analysis

The analysis and understanding of abnormal behaviours in human crowds is a challenging task in pattern recognition and computer vision. First of all, the semantic definition of the term “crowd” is ambiguous. Secondly, the taxonomy of crowd behaviours is usually rudimentary and intrinsically complicated. How to identify and construct effective features for crowd behaviour classification is a prominent challenge. Thirdly, the acquisition of suitable video for crowd analysis is another critical problem. In order to address those issues, a categorization model for abnormal behaviour types is defined according to the state-of-the-art. In the novel taxonomy of crowd behaviour, eight types of crowd behaviours are defined based on the key visual patterns. An enhanced social force-based model is proposed to achieve the visual realism in crowd simulation, hence to generate customizable videos for crowd analysis. The proposed model consists of a long-term behavior control model based on A-star path finding algorithm and a short-term interaction handling model based on the enhanced social force. The proposed simulation approach produced all the crowd behaviours in the new taxonomy for the training and testing of the detection procedure. On the aspect of feature engineering, an innovative signature is devised for assisting the segmentation of crowd in both low and high density. The signature is modelled with derived features from Grey-Level Co-occurrence Matrix. Another major breakthrough is an effective approach for efficiently extracting spatial temporal information based on the information entropy theory and Gabor background subtraction. The extraction approach is capable of obtaining the texture with most motion information, which could help the detection approach to achieve the real-time processing. Overall, these contributions have supported the crucial components in a pipeline of abnormal crowd behaviour detecting process. This process is consisted of crowd behaviour taxonomy, crowd video generation, crowd segmentation and crowd abnormal behaviour detection. Experiments for each component show promising results, and proved the accessibility of the proposed approaches

Markerless Motion Capture in the Crowd

This work uses crowdsourcing to obtain motion capture data from video recordings. The data is obtained by information workers who click repeatedly to indicate body configurations in the frames of a video, resulting in a model of 2D structure over time. We discuss techniques to optimize the tracking task and strategies for maximizing accuracy and efficiency. We show visualizations of a variety of motions captured with our pipeline then apply reconstruction techniques to derive 3D structure.Comment: Presented at Collective Intelligence conference, 2012 (arXiv:1204.2991

Crowdsourcing in Computer Vision

Computer vision systems require large amounts of manually annotated data to properly learn challenging visual concepts. Crowdsourcing platforms offer an inexpensive method to capture human knowledge and understanding, for a vast number of visual perception tasks. In this survey, we describe the types of annotations computer vision researchers have collected using crowdsourcing, and how they have ensured that this data is of high quality while annotation effort is minimized. We begin by discussing data collection on both classic (e.g., object recognition) and recent (e.g., visual story-telling) vision tasks. We then summarize key design decisions for creating effective data collection interfaces and workflows, and present strategies for intelligently selecting the most important data instances to annotate. Finally, we conclude with some thoughts on the future of crowdsourcing in computer vision.Comment: A 69-page meta review of the field, Foundations and Trends in Computer Graphics and Vision, 201