Generic, deformable models for 3-d vehicle surveillance

Abstract

Vehicle surveillance is the task of measuring moving road vehicles to automatically obtain information about vehicle shape, appearance, identity, path of motion, and, ultimately, driver behavior. While various vehicle sensors exist, none are as versatile as the surveillance camera. Computer vision algorithms can interpret digital images to make a wide variety of vehicle measurements using a single sensor. An ideal algorithm would reconstruct a detailed three-dimensional (3-d) representation of the dynamic traffic scene complete with 3-d vehicle surfaces, trajectories of motion, and identities. Unfortunately, much of the 3-d information is lost during the projection of the world into a 2-d image. As a result, the reconstruction problem is ill-posed. Several researchers have addressed this problem by incorporating prior knowledge about the world to rule out implausible reconstructions. Specifically, in the case of vehicle surveillance, a prior model of 3-d vehicle shape is often used. A constrained alignment of the model to images allows for 3-d shape recovery, tracking, and recognition. Previous 3-d vehicle models are either generic but overly simple or rigid and overly complex. Rigid models represent exactly one vehicle design, so a large collection is needed. A single generic model can deform to a wide variety of shapes, but those shapes have been far too primitive. This thesis presents a new generic 3-d vehicle model that deforms to match a wide variety of passenger vehicles. It is adjustable in complexity between the two extremes. The model is aligned to images by predicting and matching image intensity edges. Novel algorithms are presented for fitting models to images, tracking in video, and learning shape deformation from a collection of detailed rigid models. Experiments compare the proposed model to simple generic models in accuracy and reliability of 3-d shape recovery from images and tracking in video. Standard techniques for recognition are also used to compare the models. The proposed model out performs the existing simple models at each task. Yet, there is still much room for improvement, especially since training data is limited