Visual Analysis of Extremely Dense Crowded Scenes

Visual analysis of dense crowds is particularly challenging due to large number of individuals, occlusions, clutter, and fewer pixels per person which rarely occur in ordinary surveillance scenarios. This dissertation aims to address these challenges in images and videos of extremely dense crowds containing hundreds to thousands of humans. The goal is to tackle the fundamental problems of counting, detecting and tracking people in such images and videos using visual and contextual cues that are automatically derived from the crowded scenes. For counting in an image of extremely dense crowd, we propose to leverage multiple sources of information to compute an estimate of the number of individuals present in the image. Our approach relies on sources such as low confidence head detections, repetition of texture elements (using SIFT), and frequency-domain analysis to estimate counts, along with confidence associated with observing individuals, in an image region. Furthermore, we employ a global consistency constraint on counts using Markov Random Field which caters for disparity in counts in local neighborhoods and across scales. We tested this approach on crowd images with the head counts ranging from 94 to 4543 and obtained encouraging results. Through this approach, we are able to count people in images of high-density crowds unlike previous methods which are only applicable to videos of low to medium density crowded scenes. However, the counting procedure just outputs a single number for a large patch or an entire image. With just the counts, it becomes difficult to measure the counting error for a query image with unknown number of people. For this, we propose to localize humans by finding repetitive patterns in the crowd image. Starting with detections from an underlying head detector, we correlate them within the image after their selection through several criteria: in a pre-defined grid, locally, or at multiple scales by automatically finding the patches that are most representative of recurring patterns in the crowd image. Finally, the set of generated hypotheses is selected using binary integer quadratic programming with Special Ordered Set (SOS) Type 1 constraints. Human Detection is another important problem in the analysis of crowded scenes where the goal is to place a bounding box on visible parts of individuals. Primarily applicable to images depicting medium to high density crowds containing several hundred humans, it is a crucial pre-requisite for many other visual tasks, such as tracking, action recognition or detection of anomalous behaviors, exhibited by individuals in a dense crowd. For detecting humans, we explore context in dense crowds in the form of locally-consistent scale prior which captures the similarity in scale in local neighborhoods with smooth variation over the image. Using the scale and confidence of detections obtained from an underlying human detector, we infer scale and confidence priors using Markov Random Field. In an iterative mechanism, the confidences of detections are modified to reflect consistency with the inferred priors, and the priors are updated based on the new detections. The final set of detections obtained are then reasoned for occlusion using Binary Integer Programming where overlaps and relations between parts of individuals are encoded as linear constraints. Both human detection and occlusion reasoning in this approach are solved with local neighbor-dependent constraints, thereby respecting the inter-dependence between individuals characteristic to dense crowd analysis. In addition, we propose a mechanism to detect different combinations of body parts without requiring annotations for individual combinations. Once human detection and localization is performed, we then use it for tracking people in dense crowds. Similar to the use of context as scale prior for human detection, we exploit it in the form of motion concurrence for tracking individuals in dense crowds. The proposed method for tracking provides an alternative and complementary approach to methods that require modeling of crowd flow. Simultaneously, it is less likely to fail in the case of dynamic crowd flows and anomalies by minimally relying on previous frames. The approach begins with the automatic identification of prominent individuals from the crowd that are easy to track. Then, we use Neighborhood Motion Concurrence to model the behavior of individuals in a dense crowd, this predicts the position of an individual based on the motion of its neighbors. When the individual moves with the crowd flow, we use Neighborhood Motion Concurrence to predict motion while leveraging five-frame instantaneous flow in case of dynamically changing flow and anomalies. All these aspects are then embedded in a framework which imposes hierarchy on the order in which positions of individuals are updated. The results are reported on eight sequences of medium to high density crowds and our approach performs on par with existing approaches without learning or modeling patterns of crowd flow. We experimentally demonstrate the efficacy and reliability of our algorithms by quantifying the performance of counting, localization, as well as human detection and tracking on new and challenging datasets containing hundreds to thousands of humans in a given scene

How do we solve wicked problems? Effective crowd management

This paper introduces concepts that may improve our understanding of crowd behaviour and new tools which may help to improve the management of crowds. A multidisciplinary approach taken, drawing on psychology, sociology, mathematics and computer science, amongst other disciplines, and incorporating general management theory, complexity theory and concepts of emergent behaviour and complex adaptive systems. The new tools available for crowd management have been made possible due to advances in digital technology and artificial intelligence

Enhancing camera surveillance using computer vision: a research note

$\mathbf{Purpose}$ - The growth of police operated surveillance cameras has out-paced the ability of humans to monitor them effectively. Computer vision is a possible solution. An ongoing research project on the application of computer vision within a municipal police department is described. The paper aims to discuss these issues. $\mathbf{Design/methodology/approach}$ - Following the demystification of computer vision technology, its potential for police agencies is developed within a focus on computer vision as a solution for two common surveillance camera tasks (live monitoring of multiple surveillance cameras and summarizing archived video files). Three unaddressed research questions (can specialized computer vision applications for law enforcement be developed at this time, how will computer vision be utilized within existing public safety camera monitoring rooms, and what are the system-wide impacts of a computer vision capability on local criminal justice systems) are considered. $\mathbf{Findings}$ - Despite computer vision becoming accessible to law enforcement agencies the impact of computer vision has not been discussed or adequately researched. There is little knowledge of computer vision or its potential in the field. $\mathbf{Originality/value}$ - This paper introduces and discusses computer vision from a law enforcement perspective and will be valuable to police personnel tasked with monitoring large camera networks and considering computer vision as a system upgrade

Physical crowds and psychological crowds: applying self-categorization theory to computer simulation of collective behaviour

Computer models are used to simulate pedestrian behaviour for safety at mass events. Previous research has indicated differences between physical crowds of co-present individuals, and psychological crowds who mobilise collective behaviour through a shared social identity. This thesis aimed to examine the assumptions models use about crowds, conduct two studies of crowd movement to ascertain the behavioural signatures of psychological crowds, and implement these into a theoretically-driven model of crowd behaviour. A systematic review of crowd modelling literature is presented which explores the assumptions about crowd behaviour being used in current models. This review demonstrates that models portray the crowd as either an identical mass with no inter-personal connections, unique individuals with no connections to others, or as small groups within a crowd. Thus, no models have incorporated the role of self-categorisation theory needed to simulate collective behaviour. The empirical research in this thesis aimed to determine the behavioural effects of self-categorisation on pedestrian movement. Findings from a first study illustrate that, in comparison to a physical crowd, perception of shared social identities in the psychological crowd motivated participants to maintain close proximity with ingroup members through regulation of their speed and distance walked. A second study showed that collective self-organisation seemed to be increased by the presence of an outgroup, causing ingroup members to tighten formation to avoid splitting up. Finally, a computer model is presented which implements the quantified behavioural effects of self-categorisation found in the behavioural studies. A self-categorisation parameter is introduced to simulate ingroup members self-organising to remain together. This is compared to a physical crowd simulation with group identities absent. The results demonstrate that the self-categorisation parameter provides more accurate simulation of psychological crowd behaviour. Thus, it is argued that models should implement self-categorisation into simulations of psychological crowds to increase safety at mass events

Single to multiple target, multiple type visual tracking

Visual tracking is a key task in applications such as intelligent surveillance, humancomputer interaction (HCI), human-robot interaction (HRI), augmented reality (AR), driver assistance systems, and medical applications. In this thesis, we make three main novel contributions for target tracking in video sequences. First, we develop a long-term model-free single target tracking by learning discriminative correlation filters and an online classifier that can track a target of interest in both sparse and crowded scenes. In this case, we learn two different correlation filters, translation and scale correlation filters, using different visual features. We also include a re-detection module that can re-initialize the tracker in case of tracking failures due to long-term occlusions. Second, a multiple target, multiple type filtering algorithm is developed using Random Finite Set (RFS) theory. In particular, we extend the standard Probability Hypothesis Density (PHD) filter for multiple type of targets, each with distinct detection properties, to develop multiple target, multiple type filtering, N-type PHD filter, where N ≥ 2, for handling confusions that can occur among target types at the measurements level. This method takes into account not only background false positives (clutter), but also confusions between target detections, which are in general different in character from background clutter. Then, under the assumptions of Gaussianity and linearity, we extend Gaussian mixture (GM) implementation of the standard PHD filter for the proposed N-type PHD filter termed as N-type GM-PHD filter. Third, we apply this N-type GM-PHD filter to real video sequences by integrating object detectors’ information into this filter for two scenarios. In the first scenario, a tri-GM-PHD filter is applied to real video sequences containing three types of multiple targets in the same scene, two football teams and a referee, using separate but confused detections. In the second scenario, we use a dual GM-PHD filter for tracking pedestrians and vehicles in the same scene handling their detectors’ confusions. For both cases, Munkres’s variant of the Hungarian assignment algorithm is used to associate tracked target identities between frames. We make extensive evaluations of these developed algorithms and find out that our methods outperform their corresponding state-of-the-art approaches by a large margin.EPSR

Sonic Urbanities: Undoing the Soundscape and Aural History in Kingston, NY

Senior Project submitted to The Division of Social Studies of Bard College

Tracking In Dense Crowds Using Prominence And Neighborhood Motion Concurrence

Methods designed for tracking in dense crowds typically employ prior knowledge to make this difficult problem tractable. In this paper, we show that it is possible to handle this problem, without any priors, by utilizing the visual and contextual information already available in such scenes. We propose a novel tracking method tailored to dense crowds which provides an alternative and complementary approach to methods that require modeling of crowd flow and, simultaneously, is less likely to fail in the case of dynamic crowd flows and anomalies by minimally relying on previous frames. Our method begins with the automatic identification of prominent individuals from the crowd that are easy to track. Then, we use Neighborhood Motion Concurrence to model the behavior of individuals in a dense crowd, this predicts the position of an individual based on the motion of its neighbors. When the individual moves with the crowd flow, we use Neighborhood Motion Concurrence to predict motion while leveraging five-frame instantaneous flow in case of dynamically changing flow and anomalies. All these aspects are then embedded in a framework which imposes hierarchy on the order in which positions of individuals are updated. Experiments on a number of sequences show that the proposed solution can track individuals in dense crowds without requiring any pre-processing, making it a suitable online tracking algorithm for dense crowds. © 2013 Elsevier B.V

Tracking in dense crowds using prominence and neighborhood motion concurrence

Methods designed for tracking in dense crowds typically employ prior knowledge to make this difficult problem tractable. In this paper, we show that it is possible to handle this problem, without any priors, by utilizing the visual and contextual information already available in such scenes. We propose a novel tracking method tailored to dense crowds which provides an alternative and complementary approach to methods that require modeling of crowd flow and, simultaneously, is less likely to fail in the case of dynamic crowd flows and anomalies by minimally relying on previous frames. Our method begins with the automatic identification of prominent individuals from the crowd that are easy to track. Then, we use Neighborhood Motion Concurrence to model the behavior of individuals in a dense crowd, this predicts the position of an individual based on the motion of its neighbors. When the individual moves with the crowd flow, we use Neighborhood Motion Concurrence to predict motion while leveraging five-frame instantaneous flow in case of dynamically changing flow and anomalies. All these aspects are then embedded in a framework which imposes hierarchy on the order in which positions of individuals are updated. Experiments on a number of sequences show that the proposed solution can track individuals in dense crowds without requiring any pre-processing, making it a suitable online tracking algorithm for dense crowds. (C) 2013 Elsevier B.V. All rights reserved

Alfred E. Smith and Transitional Progressivism: The Revolution before the New Deal

In New York State in the 1910s and 1920s, two groups of political actors--largely female social work reformers from the settlement house tradition, and legislators from urban ethnic political machines--coalesced to develop a unique political amalgam: transitional progressivism. Transitional progressivism brought together the common interests of these two groups, forging an agenda that sought to expand the role of the state in protecting industrial laborers, ensuring social welfare, and promoting cultural pluralism. Through a complex process, this agenda became Democratic partisan dogma--first in New York and then nationally; and during both the implementation of this program and the articulation of the broader ideology of the transitional progressives in the context of state and national campaigns, transitional progressivism became the political platform of America's urban ethnic working-class voters. Through these voters and their political representatives, many priorities from the transitional progressive tradition became important facets of New Deal liberalism. Thus, by way of transitional progressivism, key elements of Progressive Era reform evolved into hallmarks of the New Deal. The foremost practitioner of this unique progressivism was Alfred E. Smith, a Democrat who served four terms as governor of New York and ran unsuccessfully for president in 1928. Part I explores the rise of transitional progressivism and its implementation during the Smith governorship. Part II presents a revisionist interpretation of the 1928 presidential contest. The conclusion follows the developments of 1928 into the 1930s, suggesting ways in which transitional progressivism exerted an important influence on the development of the New Deal