Intelligent video object tracking in large public environments

This Dissertation addresses the problem of video object tracking in large public environments, and was developed within the context of a partnership between ISCTE-IUL and THALES1 object. This partnership aimed at developing a new approach to video tracking, based on a simple tracking algorithm aided by object position estimations to deal with the harder cases of video object tracking. This proposed approach has been applied successfully in the TRAPLE2 project developed at THALES where the main focus is the real-time monitoring of public spaces and the tracking of moving objects (i.e., persons). The proposed low-processing tracking solution woks as follows: after the detection step, the various objects in the visual scene are tracked through their centres of mass (centroids) that, typically, exhibit little variations along close apart video frames. After this step, some heuristics are applied to the results to maintain coherent the identification of the video objects and estimate their positions in cases of uncertainties, e.g., occlusions, which is one of the major novelties proposed in this Dissertation. The proposed approach was tested with relevant test video sequences representing real video monitoring scenes and the obtained results showed that this approach is able to track multiple persons in real-time with reasonable computational power.Esta dissertação aborda o problema do seguimento de objectos vídeo em ambientes públicos de grande dimensão e foi desenvolvida no contexto de uma parceria entre o ISCTE-IUL e a THALES. Esta parceria visou o desenvolvimento de uma nova abordagem ao seguimento de objectos de vídeo baseada num processamento de vídeo simples em conjunto com a estimação da posição dos objectos nos casos mais difíceis de efectuar o seguimento. Esta abordagem foi aplicada com sucesso no âmbito do projecto TRAPLE desenvolvido pela THALES onde um dos principais enfoques é o seguimento de múltiplos objectos de vídeo em tempo real em espaços públicos, tendo como objectivo o seguimento de pessoas que se movam ao longo desse espaço. A solução de baixo nível de processamento proposta funciona do seguinte modo: após o passo de detecção de objectos, os diversos objectos detectados na cena são seguidos através dos seus centros de massa que, normalmente, apresentam poucas variações ao longo de imagens consecutivas de vídeo. Após este passo, algumas heurísticas são aplicadas aos resultados mantendo a identificação dos objectos de vídeo coerente e estimando as suas posições em casos de incertezas (e.g., oclusões) que é uma das principais novidades propostas nesta dissertação. A abordagem proposta foi testada com várias sequências de vídeo de teste representando cenas reais de videovigilância e os resultados obtidos mostraram que esta abordagem é capaz de seguir várias pessoas em tempo real com um nível de processamento moderado

Real-time Aerial Vehicle Detection and Tracking using a Multi-modal Optical Sensor

Vehicle tracking from an aerial platform poses a number of unique challenges including the small number of pixels representing a vehicle, large camera motion, and parallax error. For these reasons, it is accepted to be a more challenging task than traditional object tracking and it is generally tackled through a number of different sensor modalities. Recently, the Wide Area Motion Imagery sensor platform has received reasonable attention as it can provide higher resolution single band imagery in addition to its large area coverage. However, still, richer sensory information is required to persistently track vehicles or more research on the application of WAMI for tracking is required. With the advancements in sensor technology, hyperspectral data acquisition at video frame rates become possible as it can be cruical in identifying objects even in low resolution scenes. For this reason, in this thesis, a multi-modal optical sensor concept is considered to improve tracking in adverse scenes. The Rochester Institute of Technology Multi-object Spectrometer is capable of collecting limited hyperspectral data at desired locations in addition to full-frame single band imagery. By acquiring hyperspectral data quickly, tracking can be achieved at reasonableframe rates which turns out to be crucial in tracking. On the other hand, the relatively high cost of hyperspectral data acquisition and transmission need to be taken into account to design a realistic tracking. By inserting extended data of the pixels of interest we can address or avoid the unique challenges posed by aerial tracking. In this direction, we integrate limited hyperspectral data to improve measurement-to-track association. Also, a hyperspectral data based target detection method is presented to avoid the parallax effect and reduce the clutter density. Finally, the proposed system is evaluated on realistic, synthetic scenarios generated by the Digital Image and Remote Sensing software

Recent Developments in Video Surveillance

With surveillance cameras installed everywhere and continuously streaming thousands of hours of video, how can that huge amount of data be analyzed or even be useful? Is it possible to search those countless hours of videos for subjects or events of interest? Shouldn’t the presence of a car stopped at a railroad crossing trigger an alarm system to prevent a potential accident? In the chapters selected for this book, experts in video surveillance provide answers to these questions and other interesting problems, skillfully blending research experience with practical real life applications. Academic researchers will find a reliable compilation of relevant literature in addition to pointers to current advances in the field. Industry practitioners will find useful hints about state-of-the-art applications. The book also provides directions for open problems where further advances can be pursued

Automated shark detection using computer vision

With the technological advancements of UAVs, researchers are finding more ways to harness their capabilities to reduce expenses in everyday society. Machine vision is at the forefront of this research and in particular image recognition. Training a machine to identify objects and di↵erentiate them from others plays an integral role in the advancement of artificial intelligence. This project aims to design an algorithm capable of automatically detecting sharks from a UAV. Testing is performed by post-processing aerial footage of sharks taken from helicopters and drones, and analysing the reliability of the algorithm. Initially this research project involved analysing aerial photography of sharks, dissecting the images into the individual colour channels that made up the RGB and HSV colour spaces and identifying methods to detect the shark blobs. Once an adaptive threshold of the brightness channel was designed, filters were curated specific to the environments presented in the obtained aerial footage to reject false positives. These methods were considerably successful in both rejecting false positives and consistently detecting the sharks in the video feed. The methods produced in this dissertation leave room for future work in the shark detection field. By acquiring more reliable data, improvements such as using a kalman filter to detect and track moving blobs could be implemented to produce a robust shark detection and tracking system

An attention model and its application in man-made scene interpretation

The ultimate aim of research into computer vision is designing a system which interprets its surrounding environment in a similar way the human can do effortlessly. However, the state of technology is far from achieving such a goal. In this thesis different components of a computer vision system that are designed for the task of interpreting man-made scenes, in particular images of buildings, are described. The flow of information in the proposed system is bottom-up i.e., the image is first segmented into its meaningful components and subsequently the regions are labelled using a contextual classifier. Starting from simple observations concerning the human vision system and the gestalt laws of human perception, like the law of “good (simple) shape” and “perceptual grouping”, a blob detector is developed, that identifies components in a 2D image. These components are convex regions of interest, with interest being defined as significant gradient magnitude content. An eye tracking experiment is conducted, which shows that the regions identified by the blob detector, correlate significantly with the regions which drive the attention of viewers. Having identified these blobs, it is postulated that a blob represents an object, linguistically identified with its own semantic name. In other words, a blob may contain a window a door or a chimney in a building. These regions are used to identify and segment higher order structures in a building, like facade, window array and also environmental regions like sky and ground. Because of inconsistency in the unary features of buildings, a contextual learning algorithm is used to classify the segmented regions. A model which learns spatial and topological relationships between different objects from a set of hand-labelled data, is used. This model utilises this information in a MRF to achieve consistent labellings of new scenes

Video content analysis for automated detection and tracking of humans in CCTV surveillance applications

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.The problems of achieving high detection rate with low false alarm rate for human detection and tracking in video sequence, performance scalability, and improving response time are addressed in this thesis. The underlying causes are the effect of scene complexity, human-to-human interactions, scale changes, and scene background-human interactions. A two-stage processing solution, namely, human detection, and human tracking with two novel pattern classifiers is presented. Scale independent human detection is achieved by processing in the wavelet domain using square wavelet features. These features used to characterise human silhouettes at different scales are similar to rectangular features used in [Viola 2001]. At the detection stage two detectors are combined to improve detection rate. The first detector is based on shape-outline of humans extracted from the scene using a reduced complexity outline extraction algorithm. A Shape mismatch measure is used to differentiate between the human and the background class. The second detector uses rectangular features as primitives for silhouette description in the wavelet domain. The marginal distribution of features collocated at a particular position on a candidate human (a patch of the image) is used to describe statistically the silhouette. Two similarity measures are computed between a candidate human and the model histograms of human and non human classes. The similarity measure is used to discriminate between the human and the non human class. At the tracking stage, a tracker based on joint probabilistic data association filter (JPDAF) for data association, and motion correspondence is presented. Track clustering is used to reduce hypothesis enumeration complexity. Towards improving response time with increase in frame dimension, scene complexity, and number of channels; a scalable algorithmic architecture and operating accuracy prediction technique is presented. A scheduling strategy for improving the response time and throughput by parallel processing is also presented

Visual Tracking by Affine Kernel Fitting Using Color and Object Boundary

Kernel-based trackers aggregate image features within the support of a kernel (a mask) regardless of their spatial structure. These trackers spatially fit the kernel (usually in location and in scale) such that a function of the aggregate is optimized. We propose a kernel-based visual tracker that exploits the constancy of color and the presence of color edges along the target boundary. The tracker estimates the best affinity of a spatially aligned pair of kernels, one of which is color-related and the other of which is object boundary-related. In a sense, this work extends previous kernel-based track-ers by incorporating the object boundary cue into the track-ing process and by allowing the kernels to be affinely trans-formed instead of only translated and isotropically scaled. These two extensions make for more precise target local-ization. Moreover, a more accurately localized target facil-itates safer updating of its reference color model, further enhancing the tracker’s robustness. The improved tracking is demonstrated for several challenging image sequences. 1

Visual Human Tracking and Group Activity Analysis: A Video Mining System for Retail Marketing

Thesis (PhD) - Indiana University, Computer Sciences, 2007In this thesis we present a system for automatic human tracking and activity recognition from video sequences. The problem of automated analysis of visual information in order to derive descriptors of high level human activities has intrigued computer vision community for decades and is considered to be largely unsolved. A part of this interest is derived from the vast range of applications in which such a solution may be useful. We attempt to find efficient formulations of these tasks as applied to the extracting customer behavior information in a retail marketing context. Based on these formulations, we present a system that visually tracks customers in a retail store and performs a number of activity analysis tasks based on the output from the tracker. In tracking we introduce new techniques for pedestrian detection, initialization of the body model and a formulation of the temporal tracking as a global trans-dimensional optimization problem. Initial human detection is addressed by a novel method for head detection, which incorporates the knowledge of the camera projection model.The initialization of the human body model is addressed by newly developed shape and appearance descriptors. Temporal tracking of customer trajectories is performed by employing a human body tracking system designed as a Bayesian jump-diffusion filter. This approach demonstrates the ability to overcome model dimensionality ambiguities as people are leaving and entering the scene. Following the tracking, we developed a two-stage group activity formulation based upon the ideas from swarming research. For modeling purposes, all moving actors in the scene are viewed here as simplistic agents in the swarm. This allows to effectively define a set of inter-agent interactions, which combine to derive a distance metric used in further swarm clustering. This way, in the first stage the shoppers that belong to the same group are identified by deterministically clustering bodies to detect short term events and in the second stage events are post-processed to form clusters of group activities with fuzzy memberships. Quantitative analysis of the tracking subsystem shows an improvement over the state of the art methods, if used under similar conditions. Finally, based on the output from the tracker, the activity recognition procedure achieves over 80% correct shopper group detection, as validated by the human generated ground truth results

Video processing road safety system

The purpose of this report was to investigate the possible applications of computer vision processing in roadway safety. With the huge expansion of computer vision software as well as affordable processing hardware opportunity to apply established methods to new areas has arisen. A literature review was undertaken investigating and comparing algorithms in computer vision and their possible application for roadway safety. Through the literature review it was decided the most effective application for pedestrian tracking would be a combination of Gaussian Mixture Modeling for background subtraction and a Kalman filter for tracking. This in conjunction with a range of morphological filters was the final design decision. This design was cemented as a Simulink model and was run against a series of scenarios. The most important results were testing against a dynamic roadway showing extremely promising outcomes. The model was able to detect and track pedestrians over a range of designated areas. This in addition to its ability to differentiate between cars and pedestrians and the location of those pedestrians in relation to the road made the system a success. With simple counting and regions of interest changes the model can be applied to range of different environments and scenarios. However problems did emerge in the implementation of the Kalman filter causing issues in its application and track association. This report was dedicated to investigating and applying current methods and techniques in computer vision that could benefit pedestrian safety, in this regard it has successfully represented the possible uses and application of such a model