Robust object detection for video surveillance using stereo vision and Gaussian mixture model

In this paper, a novel approach is presented for intrusion detection in the field of wide-area outdoor surveillance such as construction site monitoring, using a rotatable stereo camera system combined with a multi-pose object segmentation process. In many current surveillance applications, monocular cameras are used which are sensitive to illumination changes or shadow casts. Additionally, the object classification, spatial measurement and localization using the 2D projection of a 3D world is ambiguous. Hence, a stereo camera is used to calculate a 3D point cloud of the scenery which is nearly unaffected by illumination changes, therefore enabling robust object detection and localization in the 3D space. The limited viewing range of the stereo camera is expanded by mounting it onto a rotatable tripod. To detect objects in different poses of the camera, pose specific Gaussian Mixture Models (GMM) are used. However, changing illumination outside the current field of view of the camera or spontaneously changing lighting conditions caused by e.g. lights controlled by motion sensors, would lead to false-positives in the segmentation process if using the brightness values. Hence, segmentation is performed using the calculated point cloud which is demonstrated to be robust against changing illumination and shadow casts by comparing the results of the proposed method with other state of the art segmentation methods using a database of self-captured images of a public outdoor area

Control of a PTZ camera in a hybrid vision system

In this paper, we propose a new approach to steer a PTZ camera in the direction of a detected object visible from another fixed camera equipped with a fisheye lens. This heterogeneous association of two cameras having different characteristics is called a hybrid stereo-vision system. The presented method employs epipolar geometry in a smart way in order to reduce the range of search of the desired region of interest. Furthermore, we proposed a target recognition method designed to cope with the illumination problems, the distortion of the omnidirectional image and the inherent dissimilarity of resolution and color responses between both cameras. Experimental results with synthetic and real images show the robustness of the proposed method

Towards A Self-calibrating Video Camera Network For Content Analysis And Forensics

Due to growing security concerns, video surveillance and monitoring has received an immense attention from both federal agencies and private firms. The main concern is that a single camera, even if allowed to rotate or translate, is not sufficient to cover a large area for video surveillance. A more general solution with wide range of applications is to allow the deployed cameras to have a non-overlapping field of view (FoV) and to, if possible, allow these cameras to move freely in 3D space. This thesis addresses the issue of how cameras in such a network can be calibrated and how the network as a whole can be calibrated, such that each camera as a unit in the network is aware of its orientation with respect to all the other cameras in the network. Different types of cameras might be present in a multiple camera network and novel techniques are presented for efficient calibration of these cameras. Specifically: (i) For a stationary camera, we derive new constraints on the Image of the Absolute Conic (IAC). These new constraints are shown to be intrinsic to IAC; (ii) For a scene where object shadows are cast on a ground plane, we track the shadows on the ground plane cast by at least two unknown stationary points, and utilize the tracked shadow positions to compute the horizon line and hence compute the camera intrinsic and extrinsic parameters; (iii) A novel solution to a scenario where a camera is observing pedestrians is presented. The uniqueness of formulation lies in recognizing two harmonic homologies present in the geometry obtained by observing pedestrians; (iv) For a freely moving camera, a novel practical method is proposed for its self-calibration which even allows it to change its internal parameters by zooming; and (v) due to the increased application of the pan-tilt-zoom (PTZ) cameras, a technique is presented that uses only two images to estimate five camera parameters. For an automatically configurable multi-camera network, having non-overlapping field of view and possibly containing moving cameras, a practical framework is proposed that determines the geometry of such a dynamic camera network. It is shown that only one automatically computed vanishing point and a line lying on any plane orthogonal to the vertical direction is sufficient to infer the geometry of a dynamic network. Our method generalizes previous work which considers restricted camera motions. Using minimal assumptions, we are able to successfully demonstrate promising results on synthetic as well as on real data. Applications to path modeling, GPS coordinate estimation, and configuring mixed-reality environment are explored

Implementazione di un sistema multi-camera per il pattugliamanto perimetrale coordinato

Questo lavoro di tesi nasce dalla attiva collaborazione tra l'Università di Padova e la Videotec S.p.A.. La tesi ha lo scopo di implementare l'algoritmo di pattugliamento perimetrale proposto dal Dipartimento di Ingegneria dell'Informazione sul testbed fornito dall'azienda. Il pattugliamento perimetrale rientra tra le caratteristiche che un sistema di videosorveglianza deve avere per essere completo ed interamente automatizzato. Nel nostro caso il sistema proposto ha un approccio distribuito che fornirebbe degli spunti interessanti per quanto riguarda la gestione delle risorse e delle telecamere in caso di guasti. Le principali problematiche che si sono trattate sono, in primo luogo, lo studio dell'algoritmo proposto e le eventuali estensioni da adottare per estenderlo dal semplice pattugliamento 1-D in un ambiente 3-D. In particolare è stato suggerito un approccio per il controllo dei parametri della telecamera PTZ. Un altro importante contributo che è stato proposto è una possibile architettura distribuita per il patrolling perimetrale che consenta, in uno sviluppo futuro, di integrare attività di trackingope

An Opportunistic Service Oriented Approach for Robot Search

Health care for the elderly poses a major challenge as the baby boomer generation ages. Part of the solution is to develop technology using sensor networks and service robotics to increase the length of time that an elder can remain at home. Since moderate immobility and memory impairment are common as people age, a major problem for the elderly is locating and retrieving frequently used common objects such as keys, cellphones, books, etc. However, for robots to assist people while they search for objects, they must possess the ability to interact with the human client, complex client-side environments and heterogeneous sensorimotor resources. Given this complexity, the traditional approach of developing particular control strategies in a top-down manner is not suitable. In this dissertation an opportunistic service-oriented approach is presented to address the robot search problem in residential eldercare. With the presented approach, a hierarchy of search strategies is developed in a bottom-up manner from passive object detection and retrieval performed by embedded camera sensors to context-aware cooperative search performed by a human-robot team. By opportunistically employing available sensorimotor resources, the robotic application achieves increased search performance, and has the flexibility to balance between performance goals and resource constraints. To evaluate the proposed approach, I describe several experiments with a robot-sensor network that includes the UMass uBot-5, Pan-Tilt-Zoom cameras and wireless sensors. The results of these experiments suggest that the robot search application based on the proposed approach can lead to efficient search performance and great flexibility in resource-constrained environments

Cross-layer Optimized Wireless Video Surveillance

A wireless video surveillance system contains three major components, the video capture and preprocessing, the video compression and transmission over wireless sensor networks (WSNs), and the video analysis at the receiving end. The coordination of different components is important for improving the end-to-end video quality, especially under the communication resource constraint. Cross-layer control proves to be an efficient measure for optimal system configuration. In this dissertation, we address the problem of implementing cross-layer optimization in the wireless video surveillance system. The thesis work is based on three research projects. In the first project, a single PTU (pan-tilt-unit) camera is used for video object tracking. The problem studied is how to improve the quality of the received video by jointly considering the coding and transmission process. The cross-layer controller determines the optimal coding and transmission parameters, according to the dynamic channel condition and the transmission delay. Multiple error concealment strategies are developed utilizing the special property of the PTU camera motion. In the second project, the binocular PTU camera is adopted for video object tracking. The presented work studied the fast disparity estimation algorithm and the 3D video transcoding over the WSN for real-time applications. The disparity/depth information is estimated in a coarse-to-fine manner using both local and global methods. The transcoding is coordinated by the cross-layer controller based on the channel condition and the data rate constraint, in order to achieve the best view synthesis quality. The third project is applied for multi-camera motion capture in remote healthcare monitoring. The challenge is the resource allocation for multiple video sequences. The presented cross-layer design incorporates the delay sensitive, content-aware video coding and transmission, and the adaptive video coding and transmission to ensure the optimal and balanced quality for the multi-view videos. In these projects, interdisciplinary study is conducted to synergize the surveillance system under the cross-layer optimization framework. Experimental results demonstrate the efficiency of the proposed schemes. The challenges of cross-layer design in existing wireless video surveillance systems are also analyzed to enlighten the future work. Adviser: Song C