Visual Analysis of Videos of Crowded Scenes

Automatic, vision-based analysis of crowds has implications in a number of fields, but faces unique challenges due to the large number of pedestrians within the scenes. The movement of each pedestrian contributes to the overall crowd motion (i.e., the collective motions of the scene's constituents) that varies spatially across the frame and temporally over the video. This thesis explores how to model the dynamically varying crowd motion, and how to leverage it to perform vision-based analysis on videos of crowded scenes. The crowd motion serves as a scene-centric constraint (i.e., representing the motion in the entire video), compared with conventional objectcentric methods that build on individual constituents. By exploring what information the crowd motion can represent, we demonstrate the impact of leveraging our model on three problems facing video analysis of crowded scenes. First, we represent the crowd motion using a novel statistical model of local motion patterns (i.e., the motion in local space-time areas). By doing so, we may learn the spatially and temporally varying underlying structure of the crowd motion from an example video of crowd behavior. Second, we use our model to represent the typical crowd activity (i.e., the crowd's steady-state) and detect unusual events in local areas of the video. Specifically, we identify local motion patterns that statistically deviate from our learned model. Our space-time model enables detection and isolation of unusual events that are specific to the scene and the location within the video. Next, we use the crowd motion as an indicator of an individual's motion to perform tracking. Specifically, we predict the local motion patterns at different space-time locations of the video and use them as a prior to track individuals in a Bayesian framework. Leveraging the crowd motion provides an accurate prior that dynamically adapts to the space-time variations of the crowd. Finally, we explore how to measure how much individual pedestrians conform to the movement of the crowd. To achieve this, we use our crowd model to indicate the future locations of pedestrians, and compare the direction they would move to their instantaneous optical flow. By identifying deviations from the crowd, we identify global unusual events and augment our tracking method to model the individuality of each target. We compare with conventional object-centric methods and those that do not encode the space-time varying motion of the crowd. We demonstrate that our scene-centric approach (i.e, one that starts with the crowd motion) advances video analysis closer to the robustness and dependability needed for real-world video analysis of scenes containing a large number of pedestrians.Ph.D., Computer Science -- Drexel University, 201

Coherent Filtering: Detecting Coherent Motions from Crowd Clutters

Abstract. Coherent motions, which describe the collective movements of indi-viduals in crowd, widely exist in physical and biological systems. Understand-ing their underlying priors and detecting various coherent motion patterns from background clutters have both scientific values and a wide range of practical ap-plications, especially for crowd motion analysis. In this paper, we propose and study a prior of coherent motion called Coherent Neighbor Invariance, which characterizes the local spatiotemporal relationships of individuals in coherent mo-tion. Based on the coherent neighbor invariance, a general technique of detecting coherent motion patterns from noisy time-series data called Coherent Filtering is proposed. It can be effectively applied to data with different distributions at different scales in various real-world problems, where the environments could be sparse or extremely crowded with heavy noise. Experimental evaluation and comparison on synthetic and real data show the existence of Coherence Neighbor Invariance and the effectiveness of our Coherent Filtering.1

Human Motion Analysis and Synthesis in Computer Graphics

This thesis focuses on solving a challenging problem in the field of computer graphics, namely to model and understand 3D human motion efficiently and meaningfully. This is vital to achieve the analysis (health & sports science), synthesis (character animation) and control (video game) of human movements. Though numerous studies have focused on improving the results of motion analysis, motion synthesis and motion control, only a few of these studies solved the problems from the fundamental part owing to the lack of information encoded in motion data. In my works, the motion of human was divided into the three types, namely single human motion, multi-people interactions and crowd movement. Subsequently, I solved the problems from motion analysis to motion control in different types of motion. In the single human motion, two types of motion graphs on the motion sequence were proposed using Markov Process. The human motion is represented as the directed graphs, which suggests the number of action patterns and transitions among them. By analyzing the graphs topologies, the richness, transitions flexibility and unpredictability among different action patterns inside the human motion sequence can be easily verified. The framework here is capable of visualizing and analyzing the human motion on the high level of action preference, intention and diversity. For the two people interaction motion, the use of 3D volumetric meshes on the interacting people was proposed to model their movement and spatial relationship among them. The semantic meanings of the motions were defined by such relationship. A customized Earth Movers Distance was proposed to assess the topological and geometric difference between two groups of meshes. The above assessment captured the semantic similarities among different two-people interactions, which is consistent with what humans perceive. With this interaction motion representation, the multi-people interactions in semantic level can be retrieved and analyzed, and such complex movements can be easily adapted and synthesized with low computational costs. In the crowd movement, a data-driven gesture-based crowd control system was proposed, in which the control scheme was learned from example gestures provided by different users. The users gestures and corresponding crowd motions, representable to the crowd motions properties and irrelevant to style variations of gestures and crowd motions, were modelled into a compact low dimensional space. With this representation, the proposed framework can take an arbitrary users input gesture and generate appropriate crowd motion in real time. This thesis shows the advantages of higher-level human motion modelling in different scenarios and solves different challenging tasks of computer graphics. The unified framework summarizes the knowledge to analyze, synthesize and control the movement of human