Vehicle detection and tracking using homography-based plane rectification and particle filtering

This paper presents a full system for vehicle detection and tracking in non-stationary settings based on computer vision. The method proposed for vehicle detection exploits the geometrical relations between the elements in the scene so that moving objects (i.e., vehicles) can be detected by analyzing motion parallax. Namely, the homography of the road plane between successive images is computed. Most remarkably, a novel probabilistic framework based on Kalman filtering is presented for reliable and accurate homography estimation. The estimated homography is used for image alignment, which in turn allows to detect the moving vehicles in the image. Tracking of vehicles is performed on the basis of a multidimensional particle filter, which also manages the exit and entries of objects. The filter involves a mixture likelihood model that allows a better adaptation of the particles to the observed measurements. The system is specially designed for highway environments, where it has been proven to yield excellent results

Multiple object tracking using an automatic veriable-dimension particle filter

Object tracking through particle filtering has been widely addressed in recent years. However, most works assume a constant number of objects or utilize an external detector that monitors the entry or exit of objects in the scene. In this work, a novel tracking method based on particle filtering that is able to automatically track a variable number of objects is presented. As opposed to classical prior data assignment approaches, adaptation of tracks to the measurements is managed globally. Additionally, the designed particle filter is able to generate hypotheses on the presence of new objects in the scene, and to confirm or dismiss them by gradually adapting to the global observation. The method is especially suited for environments where traditional object detectors render noisy measurements and frequent artifacts, such as that given by a camera mounted on a vehicle, where it is proven to yield excellent results

Video analysis based vehicle detection and tracking using an MCMC sampling framework

This article presents a probabilistic method for vehicle detection and tracking through the analysis of monocular images obtained from a vehicle-mounted camera. The method is designed to address the main shortcomings of traditional particle filtering approaches, namely Bayesian methods based on importance sampling, for use in traffic environments. These methods do not scale well when the dimensionality of the feature space grows, which creates significant limitations when tracking multiple objects. Alternatively, the proposed method is based on a Markov chain Monte Carlo (MCMC) approach, which allows efficient sampling of the feature space. The method involves important contributions in both the motion and the observation models of the tracker. Indeed, as opposed to particle filter-based tracking methods in the literature, which typically resort to observation models based on appearance or template matching, in this study a likelihood model that combines appearance analysis with information from motion parallax is introduced. Regarding the motion model, a new interaction treatment is defined based on Markov random fields (MRF) that allows for the handling of possible inter-dependencies in vehicle trajectories. As for vehicle detection, the method relies on a supervised classification stage using support vector machines (SVM). The contribution in this field is twofold. First, a new descriptor based on the analysis of gradient orientations in concentric rectangles is dened. This descriptor involves a much smaller feature space compared to traditional descriptors, which are too costly for real-time applications. Second, a new vehicle image database is generated to train the SVM and made public. The proposed vehicle detection and tracking method is proven to outperform existing methods and to successfully handle challenging situations in the test sequences

Real-time robust estimation of vanishing points through nonlinear optimization

Vanishing points are elements of great interest in the computer vision field, since they are the main source of information about the geometry of the scene and the projection process associated to the camera. They have been studied and applied during decades for plane rectification, 3D reconstruction, and mainly auto-calibration tasks. Nevertheless, the literature lacks accurate online solutions for multiple vanishing point estimation. Most strategies focalize on the accuracy, using highly computational demanding iterative procedures. We propose a novel strategy for multiple vanishing point estimation that finds a trade-off between accuracy and efficiency, being able to operate in real time for video sequences. This strategy takes advantage of the temporal coherence of the images of the sequences to reduce the computational load of the processing algorithms while keeping a high level of accuracy due to an optimization process. The key element of the approach is a robust scheme based on the MLESAC algorithm, which is used in a similar way to the EM algorithm. This approach ensures robust and accurate estimations, since we use the MLESAC in combination with a novel error function, based on the angular error between the vanishing point and the image features. To increase the speed of the MLESAC algorithm, the selection of the minimal sample sets is substituted by a random sampling step that takes into account temporal information to provide better initializations. Besides, for the sake of flexibility, the proposed error function has been designed to work using as image features indiscriminately gradient-pixels or line segments. Hence, we increase the range of applications in which our approach can be used, according to the type of information that is available. The results show a real-time system that delivers real-time accurate estimations of multiple vanishing points for online processing, tested in moving camera video sequences of structured scenarios, both indoors and outdoors, such as rooms, corridors, facades, roads, etc

Non-linear optimization for robust estimation of vanishing points

A new method for robust estimation of vanishing points is introduced in this paper. It is based on the MSAC (M-estimator Sample and Consensus) algorithm and on the definition of a new distance function between a vanishing point and a given orientation. Apart from the robustness, our method represents a flexible and efficient solution, since it allows to work with different type of image data, and its iterative nature makes better use of the available information to obtain more accurate estimates. The key issue of the work is the proposed distance function, that makes the error to be independent from the position of an hypothesized vanishing point, which allows to work with points at the infinity. Besides, the estimation process is guided by a non-linear optimization process that enhances the accuracy of the system. The robustness of our proposal, compared with other methods in the literature is shown with a set of tests carried out for both synthetic data and real images. The results show that our approach obtain excellent levels of accuracy and that is definitely robust against the presence of large amounts of outliers, outperforming other state of the art approaches

Automatic video mosaicing for surveillance using vanishing points

Video surveillance typically relies on various methods of 3D scene reconstruction, such as people counting, and object detection and tracking. Video mosaicing is a related technology that enables an expanded view of a scene by pasting together frames from the video stream (superimages) of a moving single camera in real time. This technique provides better resolution than can be achieved with a conventional video camera. Well-known related applications in other fields include composition of panoramic pictures, alignment of satellite images, and generation of synthetic 360°-angle views. Given two frames of a scene, a general approach consists of finding a number of points in both images that correspond to the same objects in the scene. This makes it possible to compute a ‘transform’ that describes the geometrical relation between the views. Known as a planar homography, this transform can be obtained using robust estimation techniques on the point correspondences

Plane rectification through robust vanishing point tracking using the expectation-maximization algorithm

This paper introduces a new strategy for plane rectification in sequences of images, based on the Expectation-Maximization (EM) algorithm. Our approach is able to compute simultaneously the parameters of the dominant vanishing point in the image plane and the most significant lines passing through it. It is based on a novel definition of the likelihood distribution of the gradient image considering both the position and the orientation of the gradient pixels. Besides, the mixture model in which the EM algorithm operates is extended, compared to other works, to consider an additional component to control the presence of outliers. Some synthetic data tests are described to show the robustness and efficiency of the proposed method. The plane rectification results show that the method is able to remove the perspective and affine distortion of real traffic sequences without the need to compute two vanishing point

Robust Multiple Lane Road Modeling Based on Perspective Analysis

Road modeling is the first step towards environment perception within driver assistance video-based systems. Typically, lane modeling allows applications such as lane departure warning or lane invasion by other vehicles. In this paper, a new monocular image processing strategy that achieves a robust multiple lane model is proposed. The identification of multiple lanes is done by firstly detecting the own lane and estimating its geometry under perspective distortion. The perspective analysis and curve fitting allows to hypothesize adjacent lanes assuming some a priori knowledge about the road. The verification of these hypotheses is carried out by a confidence level analysis. Several types of sequences have been tested, with different illumination conditions, presence of shadows and significant curvature, all performing in realtime. Results show the robustness of the system, delivering accurate multiple lane road models in most situations

Robust Road Modeling based on a Hierarchical Bipartite Graph

Driver assistance systems based on video processing deliver a number of warnings to the driver, such as lane departure, lane invasion by other vehicles, collision prediction, etc. This have been a field of intense research for many years, providing solutions based on road models where vehicles are afterwards detected and tracked. Robustness is essential in this field of road safety where outliers represent one of the major problems for road modeling. The motivation of this work is to provide a robust and, at the same time, flexible road model which identifies a variable number of lanes, their widths, the curvature of the road and the position of the vehicle in its lane. The major advantage of this model is that the system gives confidence measures for each lane, determining which lanes are actually present and which not. The model is structured as a hierarchical bipartite graph which simplifies information management, reduces sub-module dependencies and classifies elements of the road in different levels. At each level different strategies are applied, following four overall steps: measurement, estimation, evaluation and extrapolation, which lead to enhanced road model accuracy, reliability and flexibility. Several experimental results are provided, showing the robustness of the system, its stability and accurate results for large test paths

On-board Robust Multiple Vehicle Detection and Tracking Using Adaptive Quality Evaluation

This paper presents a robust method for real-time vehicle detection and tracking in dynamic traffic environments. The proposed strategy aims to find a trade-off between the robustness shown by time-uncorrelated detection techniques and the speed-up obtained with tracking algorithms. It combines both advantages by continuously evaluating the quality of the tracking results along time and triggering new detections to restart the tracking process when quality falls behind a certain quality requirement. Robustness is also ensured within the tracking algorithm with an outlier rejection stage and the use of stochastic filtering. Several sequences from real traffic situations have been tested, obtaining highly accurate multiple vehicle detections