Object Tracking and Mensuration in Surveillance Videos

This thesis focuses on tracking and mensuration in surveillance videos. The first part of the thesis discusses several object tracking approaches based on the different properties of tracking targets. For airborne videos, where the targets are usually small and with low resolutions, an approach of building motion models for foreground/background proposed in which the foreground target is simplified as a rigid object. For relatively high resolution targets, the non-rigid models are applied. An active contour-based algorithm has been introduced. The algorithm is based on decomposing the tracking into three parts: estimate the affine transform parameters between successive frames using particle filters; detect the contour deformation using a probabilistic deformation map, and regulate the deformation by projecting the updated model onto a trained shape subspace. The active appearance Markov chain (AAMC). It integrates a statistical model of shape, appearance and motion. In the AAMC model, a Markov chain represents the switching of motion phases (poses), and several pairwise active appearance model (P-AAM) components characterize the shape, appearance and motion information for different motion phases. The second part of the thesis covers video mensuration, in which we have proposed a heightmeasuring algorithm with less human supervision, more flexibility and improved robustness. From videos acquired by an uncalibrated stationary camera, we first recover the vanishing line and the vertical point of the scene. We then apply a single view mensuration algorithm to each of the frames to obtain height measurements. Finally, using the LMedS as the cost function and the Robbins-Monro stochastic approximation (RMSA) technique to obtain the optimal estimate

Joint acoustic-video fingerprinting of vehicles, part II

In this second paper, we first show how to estimate the wheelbase length of a vehicle using line metrology in video. We then address the vehicle fingerprinting problem using vehicle silhouettes and color invariants. We combine the acoustic metrology and classification results discussed in Part I with the video results to improve estimation performance and robustness. The acoustic video fusion is achieved in a Bayesian framework by assuming conditional independence of the observations of each modality. For the metrology density functions, Laplacian approximations are used for computational efficiency. Experimental results are given using field data

Multiple Targets Geolocation Using SIFT and Stereo Vision on Airborne Video Sequences

We propose a robust and accurate method for multi-target geo-localization from airborne video. The difference between our approach and other approaches in the literature is fourfold: 1) it does not require gimbal control of the camera or any particular path planning control for the UAV; 2) it can instantaneously geolocate multiple targets even if they were not previously observed by the camera; 3) it does not require a georeferenced terrain database nor an altimeter for estimating the UAV's and the target's altitudes; and 4) it requires only one camera, but it employs a multi-stereo technique using the image sequence for increased accuracy in target geo-location. The only requirements for our approach are: that the intrinsic parameters of the camera be known; that the on board camera be equipped with global positioning system (GPS) and inertial measurement unit (IMU); and that enough feature points can be extracted from the surroundings of the target. Since the first two constraints are easily satisfied, the only real requirement is regarding the feature points. However, as we explain later, this last constraint can also be alleviated if the ground is approximately planar. The result is a method that can reach a few meters of accuracy for an UAV flying at a few hundred meters above the ground. Such performance is demonstrated by computer simulation, in-scale data using a model city, and real airborne video with ground truth

Object reconstruction using close-range all-round digital photogrammetry for applications in industry

Bibliography: p. 66-68.Photogrammetry has many inherent advantages in engineering and industrial applications, which include the ability to obtain accurate, non-contact measurements from data rapidly acquired with the object in situ. Along with these advantages, digital photogrammetry offers the potential for the automation or semi-automation of many of the conventional photogrammetric procedures, leading to real-time or near real-time measurement capabilities. However, all-round surface measurement of an object usually benefits less from the above advantages of photogrammetry. To obtain the necessary imagery from all sides of the measurement object, real-time processing is nearly impossible, and it becomes difficult to avoid moving the object, thus precluding in situ measurement. However, all-round digital photogrammetry and, in particular, the procedure presented here, still offer advantages over other methods of full surface measurement, including rapid, non-contact data acquisition along with the ability to store and reprocess data at a later date. Conventional or topographic photogrammetry is well-established as a tool for mapping simple terrain surfaces and for acquiring accurate 3-D point data. The complexities of all-round photogrammetry make many of the standard photogrammetric methods all but redundant. The work presented in this thesis was aimed at the development of a reliable method of obtaining complete surface data of an object with non-topographic, all-round, close-range digital photogrammetry. A method was developed to improve the integrity of the data, and possibilities for the presentation and visualisation of the data were explored. The potential for automation was considered important, as was the need to keep the overall time required to a minimum. A measurement system was developed to take as input an object, and produce as output an accurate, representative point cloud, allowing for the reconstruction of the surface. This system included the following procedures: ■ a novel technique of achieving high-accuracy system pre-calibration using a cubic control frame and fixed camera stations, ■ separate image capture for the control frame and the object, ■ surface sub-division and all-round step-wise image matching to produce a comprehensive 3-D data set, ■ point cloud refinement, and ■ surface reconstruction by separate surface generation. The development and reliability of these new approaches is discussed and investigated; and the results of various test procedures are presented. The technique of system pre-calibration involved the use of a mechanical device - a rotary table - to impart precisely repeatable rotations to the control frame and, separately, the object. The actual repeatability precision was tested and excellent results achieved, with standard deviations for the resected camera station coordinates of between 0.05 and 0.5 mm. In a detailed test case, actual rotations differed from the desired rotations by an average of 0.7" with a standard deviation of less than 2'. The image matching for the test case, from a set of forty-eight images, achieved a satisfactory final accuracy, comparable to that achieved in other similar work. The meaningful reconstruction of surfaces presented problems, although an acceptable rendering was achieved, and a thorough survey of current commercially available software failed to produce a package capable of all-round modelling from random 3-D data. The final analysis of the results indicated that digital photogrammetry, and this method in particular, are highly suited to accurate all-round surface measurement. The potential for automation - and, therefore, for near real-time results - of the method in the stages of image acquisition and processing, calibration, image matching and data visualisation is great. The method thus lends itself to industrial applications. However, the need for a robust and rapid method of surface reconstruction needs to be fulfilled

Joint acoustic-video fingerprinting of vehicles, part I

We address vehicle classification and mensuration problems using acoustic and video sensors. In this paper, we show how to estimate a vehicle's speed, width, and length by jointly estimating its acoustic wave-pattern using a single passive acoustic sensor that records the vehicle's drive-by noise. The acoustic wave-pattern is approximated using three envelope shape (ES) components, which approximate the shape of the received signal's power envelope. We incorporate the parameters of the ES components along with estimates of the vehicle engine RPM and number of cylinders to create a vehicle profile vector that forms an intuitive discriminatory feature space. In the companion paper, we discuss vehicle classification and mensuration based on silhouette extraction and wheel detection, using a video sensor. Vehicle speed estimation and classification results are provided using field data

Calibration and Metrology Using Still and Video Images

Metrology, the measurement of real world metrics, has been investigated extensively in computer vision for many applications. The prevalence of video cameras and sequences has led to the demand for fully automated systems. Most of the existing video metrology methods are simple extensions of still-image algorithms, which have certain limitations, requiring constraints such as parallelism of lines. New techniques are needed in order to achieve accurate results for broader applications. An important preprocessing step and a closely related topic to metrology is calibration using planar patterns. Existing approaches lack exibility and robustness when extended to video sequences. This dissertation advances the state of the art in calibration and video metrology in three directions: (1) the concept of partial rectification is proposed along with new calibration techniques using a circle with diverse types of constraints; (2) new calibration methods for video sequences using planar patterns undergoing planar motion are proposed; and (3) new algorithms to extend video metrology to a wide range of applications are presented. A fully automated system using the new technique has been built for measuring the wheelbases of vehicles

Acoustic detection and quantification of benthic egg beds of the squid Loligo opalescens in Monterey Bay, California

Author Posting. © Acoustical Society of America, 2006. This article is posted here by permission of Acoustical Society of America for personal use, not for redistribution. The definitive version was published in Journal of the Acoustical Society of America 119 (2006): 844-856, doi:10.1121/1.2149840.The squid Loligo opalescens is a key species in the nearshore pelagic community of California, supporting the most valuable state marine fishery, yet the stock biomass is unknown. In southern Monterey Bay, extensive beds occur on a flat, sandy bottom, water depths 20–60 m, thus sidescan sonar is a prima-facie candidate for use in rapid, synoptic, and noninvasive surveying. The present study describes development of an acoustic method to detect, identify, and quantify squid egg beds by means of high-frequency sidescan-sonar imagery. Verification of the method has been undertaken with a video camera carried on a remotely operated vehicle. It has been established that sidescan sonar images can be used to predict the presence or absence of squid egg beds. The lower size limit of detectability of an isolated egg bed is about 0.5 m with a 400-kHz sidescan sonar used with a 50-m range when towed at 3 knots. It is possible to estimate the abundance of eggs in a region of interest by computing the cumulative area covered by the egg beds according to the sidescan sonar image. In a selected quadrat one arc second on each side, the estimated number of eggs was 36.5 million.funding from the National Sea Grant, Essential Fish Habitat Program, Sea Grant Project No. NA16RG2273

Vision-based legged robot navigation: localisation, local planning, learning

The recent advances in legged locomotion control have made legged robots walk up staircases, go deep into underground caves, and walk in the forest. Nevertheless, autonomously achieving this task is still a challenge. Navigating and acomplishing missions in the wild relies not only on robust low-level controllers but also higher-level representations and perceptual systems that are aware of the robot's capabilities. This thesis addresses the navigation problem for legged robots. The contributions are four systems designed to exploit unique characteristics of these platforms, from the sensing setup to their advanced mobility skills over different terrain. The systems address localisation, scene understanding, and local planning, and advance the capabilities of legged robots in challenging environments. The first contribution tackles localisation with multi-camera setups available on legged platforms. It proposes a strategy to actively switch between the cameras and stay localised while operating in a visual teach and repeat context---in spite of transient changes in the environment. The second contribution focuses on local planning, effectively adding a safety layer for robot navigation. The approach uses a local map built on-the-fly to generate efficient vector field representations that enable fast and reactive navigation. The third contribution demonstrates how to improve local planning in natural environments by learning robot-specific traversability from demonstrations. The approach leverages classical and learning-based methods to enable online, onboard traversability learning. These systems are demonstrated via different robot deployments on industrial facilities, underground mines, and parklands. The thesis concludes by presenting a real-world application: an autonomous forest inventory system with legged robots. This last contribution presents a mission planning system for autonomous surveying as well as a data analysis pipeline to extract forestry attributes. The approach was experimentally validated in a field campaign in Finland, evidencing the potential that legged platforms offer for future applications in the wild