Visual Human Tracking and Group Activity Analysis: A Video Mining System for Retail Marketing

Thesis (PhD) - Indiana University, Computer Sciences, 2007In this thesis we present a system for automatic human tracking and activity recognition from video sequences. The problem of automated analysis of visual information in order to derive descriptors of high level human activities has intrigued computer vision community for decades and is considered to be largely unsolved. A part of this interest is derived from the vast range of applications in which such a solution may be useful. We attempt to find efficient formulations of these tasks as applied to the extracting customer behavior information in a retail marketing context. Based on these formulations, we present a system that visually tracks customers in a retail store and performs a number of activity analysis tasks based on the output from the tracker. In tracking we introduce new techniques for pedestrian detection, initialization of the body model and a formulation of the temporal tracking as a global trans-dimensional optimization problem. Initial human detection is addressed by a novel method for head detection, which incorporates the knowledge of the camera projection model.The initialization of the human body model is addressed by newly developed shape and appearance descriptors. Temporal tracking of customer trajectories is performed by employing a human body tracking system designed as a Bayesian jump-diffusion filter. This approach demonstrates the ability to overcome model dimensionality ambiguities as people are leaving and entering the scene. Following the tracking, we developed a two-stage group activity formulation based upon the ideas from swarming research. For modeling purposes, all moving actors in the scene are viewed here as simplistic agents in the swarm. This allows to effectively define a set of inter-agent interactions, which combine to derive a distance metric used in further swarm clustering. This way, in the first stage the shoppers that belong to the same group are identified by deterministically clustering bodies to detect short term events and in the second stage events are post-processed to form clusters of group activities with fuzzy memberships. Quantitative analysis of the tracking subsystem shows an improvement over the state of the art methods, if used under similar conditions. Finally, based on the output from the tracker, the activity recognition procedure achieves over 80% correct shopper group detection, as validated by the human generated ground truth results

Real time tracking using nature-inspired algorithms

This thesis investigates the core difficulties in the tracking field of computer vision. The aim is to develop a suitable tuning free optimisation strategy so that a real time tracking could be achieved. The population and multi-solution based approaches have been applied first to analyse the convergence behaviours in the evolutionary test cases. The aim is to identify the core misconceptions in the manner the search characteristics of particles are defined in the literature. A general perception in the scientific community is that the particle based methods are not suitable for the real time applications. This thesis improves the convergence properties of particles by a novel scale free correlation approach. By altering the fundamental definition of a particle and by avoiding the nostalgic operations the tracking was expedited to a rate of 250 FPS. There is a reasonable amount of similarity between the tracking landscapes and the ones generated by three dimensional evolutionary test cases. Several experimental studies are conducted that compares the performances of the novel optimisation to the ones observed with the swarming methods. It is therefore concluded that the modified particle behaviour outclassed the traditional approaches by huge margins in almost every test scenario

Swarming Reconnaissance Using Unmanned Aerial Vehicles in a Parallel Discrete Event Simulation

Current military affairs indicate that future military warfare requires safer, more accurate, and more fault-tolerant weapons systems. Unmanned Aerial Vehicles (UAV) are one answer to this military requirement. Technology in the UAV arena is moving toward smaller and more capable systems and is becoming available at a fraction of the cost. Exploiting the advances in these miniaturized flying vehicles is the aim of this research. How are the UAVs employed for the future military? The concept of operations for a micro-UAV system is adopted from nature from the appearance of flocking birds, movement of a school of fish, and swarming bees among others. All of these natural phenomena have a common thread: a global action resulting from many small individual actions. This emergent behavior is the aggregate result of many simple interactions occurring within the flock, school, or swarm. In a similar manner, a more robust weapon system uses emergent behavior resulting in no weakest link because the system itself is made up of simple interactions by hundreds or thousands of homogeneous UAVs. The global system in this research is referred to as a swarm. Losing one or a few individual unmanned vehicles would not dramatically impact the swarms ability to complete the mission or cause harm to any human operator. Swarming reconnaissance is the emergent behavior of swarms to perform a reconnaissance operation. An in-depth look at the design of a reconnaissance swarming mission is studied. A taxonomy of passive reconnaissance applications is developed to address feasibility. Evaluation of algorithms for swarm movement, communication, sensor input/analysis, targeting, and network topology result in priorities of each model\u27s desired features. After a thorough selection process of available implementations, a subset of those models are integrated and built upon resulting in a simulation that explores the innovations of swarming UAVs

Microscale arrays for the profiling of start and stop signals coordinating human-neutrophil swarming.

Neutrophil swarms protect healthy tissues by sealing off sites of infection. In the absence of swarming, microbial invasion of surrounding tissues can result in severe infections. Recent observations in animal models have shown that swarming requires rapid neutrophil responses and well-choreographed neutrophil migration patterns. However, in animal models physical access to the molecular signals coordinating neutrophil activities during swarming is limited. Here, we report the development and validation of large microscale arrays of zymosan-particle clusters for the study of human neutrophils during swarming ex vivo. We characterized the synchronized swarming of human neutrophils under the guidance of neutrophil-released chemokines, and measured the mediators released at different phases of human-neutrophil swarming against targets simulating infections. We found that the network of mediators coordinating human-neutrophil swarming includes start and stop signals, proteolytic enzymes and enzyme inhibitors, as well as modulators of activation of other immune and non-immune cells. We also show that the swarming behavior of neutrophils from patients following major trauma is deficient and gives rise to smaller swarms than those of neutrophils from healthy individuals

GUARDIANS final report

Emergencies in industrial warehouses are a major concern for firefghters. The large dimensions together with the development of dense smoke that drastically reduces visibility, represent major challenges. The Guardians robot swarm is designed to assist fire fighters in searching a large warehouse. In this report we discuss the technology developed for a swarm of robots searching and assisting fire fighters. We explain the swarming algorithms which provide the functionality by which the robots react to and follow humans while no communication is required. Next we discuss the wireless communication system, which is a so-called mobile ad-hoc network. The communication network provides also one of the means to locate the robots and humans. Thus the robot swarm is able to locate itself and provide guidance information to the humans. Together with the re ghters we explored how the robot swarm should feed information back to the human fire fighter. We have designed and experimented with interfaces for presenting swarm based information to human beings

Active Brownian Particles. From Individual to Collective Stochastic Dynamics

We review theoretical models of individual motility as well as collective dynamics and pattern formation of active particles. We focus on simple models of active dynamics with a particular emphasis on nonlinear and stochastic dynamics of such self-propelled entities in the framework of statistical mechanics. Examples of such active units in complex physico-chemical and biological systems are chemically powered nano-rods, localized patterns in reaction-diffusion system, motile cells or macroscopic animals. Based on the description of individual motion of point-like active particles by stochastic differential equations, we discuss different velocity-dependent friction functions, the impact of various types of fluctuations and calculate characteristic observables such as stationary velocity distributions or diffusion coefficients. Finally, we consider not only the free and confined individual active dynamics but also different types of interaction between active particles. The resulting collective dynamical behavior of large assemblies and aggregates of active units is discussed and an overview over some recent results on spatiotemporal pattern formation in such systems is given.Comment: 161 pages, Review, Eur Phys J Special-Topics, accepte

GUARDIANS final report part 1 (draft): a robot swarm assisting a human fire fighter

Emergencies in industrial warehouses are a major concern for fire fighters. The large dimensions together with the development of dense smoke that drastically reduces visibility, represent major challenges. The Guardians robot swarm is designed to assist re ghters in searching a large warehouse. In this paper we discuss the technology developed for a swarm of robots assisting re ghters. We explain the swarming algorithms which provide the functionality by which the robots react to and follow humans while no communication is required. Next we discuss the wireless communication system, which is a so-called mobile ad-hoc network. The communication network provides also the means to locate the robots and humans. Thus the robot swarm is able to provide guidance information to the humans. Together with the fire fighters we explored how the robot swarm should feed information back to the human fire fighter. We have designed and experimented with interfaces for presenting swarm based information to human beings

Shared behavioral mechanisms underlie <i>C. elegans</i> aggregation and swarming

In complex biological systems, simple individual-level behavioral rules can give rise to emergent group-level behavior. While collective behavior has been well studied in cells and larger organisms, the mesoscopic scale is less understood, as it is unclear which sensory inputs and physical processes matter a priori. Here, we investigate collective feeding in the roundworm C. elegans at this intermediate scale, using quantitative phenotyping and agent-based modeling to identify behavioral rules underlying both aggregation and swarming—a dynamic phenotype only observed at longer timescales. Using fluorescence multi-worm tracking, we quantify aggregation in terms of individual dynamics and population-level statistics. Then we use agent-based simulations and approximate Bayesian inference to identify three key behavioral rules for aggregation: cluster-edge reversals, a density-dependent switch between crawling speeds, and taxis towards neighboring worms. Our simulations suggest that swarming is simply driven by local food depletion but otherwise employs the same behavioral mechanisms as the initial aggregation

Collective motion of cells: from experiments to models

Swarming or collective motion of living entities is one of the most common and spectacular manifestations of living systems having been extensively studied in recent years. A number of general principles have been established. The interactions at the level of cells are quite different from those among individual animals therefore the study of collective motion of cells is likely to reveal some specific important features which are overviewed in this paper. In addition to presenting the most appealing results from the quickly growing related literature we also deliver a critical discussion of the emerging picture and summarize our present understanding of collective motion at the cellular level. Collective motion of cells plays an essential role in a number of experimental and real-life situations. In most cases the coordinated motion is a helpful aspect of the given phenomenon and results in making a related process more efficient (e.g., embryogenesis or wound healing), while in the case of tumor cell invasion it appears to speed up the progression of the disease. In these mechanisms cells both have to be motile and adhere to one another, the adherence feature being the most specific to this sort of collective behavior. One of the central aims of this review is both presenting the related experimental observations and treating them in the light of a few basic computational models so as to make an interpretation of the phenomena at a quantitative level as well.Comment: 24 pages, 25 figures, 13 reference video link