Model-based automatic tracking of articulated human movement

This study applied a vision-based tracking approach to the analysis of articulated, three-dimensional (3D) whole-body human movements. A 3D computer graphics model of the human body was constructed from ellipsoid solids and customized to two gymnasts for size and colour. The model was used in the generation of model images from multiple camera views with simulated environments based on measurements taken on each of three synchronized video cameras and the lighting sources present in the original recording environment. A hierarchical procedure was used whereby the torso was tracked initially to establish whole-body position and orientation and subsequently body segments were added successively to the model to establish body configuration. An iterative procedure was used at each stage to optimize each new set of variables using a score based on the RGB colour difference between the model images and video images at each stage. Tracking experiments were carried out on movement sequences using both synthetic and video image data. Promising qualitative results were obtained with consistent model matching in all sequences, including sequences involving whole-body rotational movements. Accurate tracking results were obtained for the synthetic image sequences. Automatic tracking results for the video images were also compared with kinematic estimates obtained via manual digitization and favourable comparisons were obtained. It is concluded that with further development this model-based approach using colour matching should provide the basis of a robust and accurate tracking system applicable to data collection for biomechanics studies

Visual Human Tracking and Group Activity Analysis: A Video Mining System for Retail Marketing

Thesis (PhD) - Indiana University, Computer Sciences, 2007In this thesis we present a system for automatic human tracking and activity recognition from video sequences. The problem of automated analysis of visual information in order to derive descriptors of high level human activities has intrigued computer vision community for decades and is considered to be largely unsolved. A part of this interest is derived from the vast range of applications in which such a solution may be useful. We attempt to find efficient formulations of these tasks as applied to the extracting customer behavior information in a retail marketing context. Based on these formulations, we present a system that visually tracks customers in a retail store and performs a number of activity analysis tasks based on the output from the tracker. In tracking we introduce new techniques for pedestrian detection, initialization of the body model and a formulation of the temporal tracking as a global trans-dimensional optimization problem. Initial human detection is addressed by a novel method for head detection, which incorporates the knowledge of the camera projection model.The initialization of the human body model is addressed by newly developed shape and appearance descriptors. Temporal tracking of customer trajectories is performed by employing a human body tracking system designed as a Bayesian jump-diffusion filter. This approach demonstrates the ability to overcome model dimensionality ambiguities as people are leaving and entering the scene. Following the tracking, we developed a two-stage group activity formulation based upon the ideas from swarming research. For modeling purposes, all moving actors in the scene are viewed here as simplistic agents in the swarm. This allows to effectively define a set of inter-agent interactions, which combine to derive a distance metric used in further swarm clustering. This way, in the first stage the shoppers that belong to the same group are identified by deterministically clustering bodies to detect short term events and in the second stage events are post-processed to form clusters of group activities with fuzzy memberships. Quantitative analysis of the tracking subsystem shows an improvement over the state of the art methods, if used under similar conditions. Finally, based on the output from the tracker, the activity recognition procedure achieves over 80% correct shopper group detection, as validated by the human generated ground truth results

Video based system monitoring

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 2006.Includes bibliographical references (p. 207-216).In this work we develop new algorithms for video comparison, for video alignment, and for determining the similarity between entire video clips or detecting similarities between sub-videos. The intent of this work is to develop video-based techniques for autonomous monitoring of systems in industrial, manufacturing, and research environments. We develop an algorithm, Dynamic Time and Space Warping, to determine a model-free similarity between an example and an unknown video. The algorithm optimally shifts space and warps time according to local measures of video similarity. The resulting similarity measure is an optimal path of similarity versus space and time used to optimally align or compare the two video. We demonstrate the applicability of such similarity measures to industrial wear monitoring, failure prediction, and assembly-line feedback control and to non-industrial settings with examples in sports and entertainment. We extend the similarity machinery and introduce a new technique for video event-detection. The local similarity is integrated along the optimal space-time path in order to determine a cumulative similarity.(cont.) We demonstrate the applicability to content query and surveillance; we identify the temporal and spatial location inside of a large video stream which is similar to a query, or template, video. We explore applications in video classification. We investigate the performance degradation and robustness of the algorithms to noise via distortion of real examples and simulation. We develop techniques to aid engineers in the selection of a video template that is relevant to their monitoring application and locally robust to noise. We explore the structure and computational complexity of the algorithms. We demonstrate that the algorithms are highly-parallelizable and that the fast processing rates necessary for many industrial monitoring applications are achievable.by Brian W. Anthony.Ph.D