Integrated Stereovision for an Autonomous Ground Vehicle Competing in the Darpa Grand Challenge

The DARPA Grand Challenge (DGC) 2005 was a competition, in form of a desert race for autonomous ground vehicles, arranged by the U.S. Defense Advanced Research Project Agency (DARPA). The purpose was to encourage research and development of related technology. The objective of the race was to track a distance of 131.6 miles in less than 10 hours without any human interaction. Only public GPS signals and terrain sensors were allowed for navigation and obstacle detection. One of the teams competing in the DGC was Team Caltech from California Institute of Technology, consisting primarily of undergraduate students. The vehicle representing Team Caltech was a 2005 Ford E-350 van, named Alice. Alice had been modified for off-road driving and equipped with multiple sensors, computers and actuators. One type of terrain sensors used on Alice was stereovision. Two camera pairs were used for short and long range obstacle detection. This master thesis concerns development, testing and integration of stereovision sensors during the final four months leading to the race. To begin with, the stereovision system on Alice was not ready to use and had not undergone any testing. The work described in this thesis enabled operation of stereovision. It further improved its capability such that it increased the overall performance of Alice. Reliability was demonstrated through multiple desert field tests. Obstacle avoidance and navigation using only stereovision was successfully demonstrated. The completed work includes design and implementation of algorithms to improve camera focus and exposure control, increase processing speed and remove noise. Also hardware and software parameters were configured to achieve best possible operation. Alice managed to qualify to the race as one of the top ten vehicles. However she was only able to complete about 8 miles before running over a concrete barrier and out of the course, as a result of hardware failures and state estimation errors

Pose self-calibration of stereo vision systems for autonomous vehicle applications

Nowadays, intelligent systems applied to vehicles have grown very rapidly; their goal is not only the improvement of safety, but also making autonomous driving possible. Many of these intelligent systems are based on making use of computer vision in order to know the environment and act accordingly. It is of great importance to be able to estimate the pose of the vision system because the measurement matching between the perception system (pixels) and the vehicle environment (meters) depends on the relative position between the perception system and the environment. A new method of camera pose estimation for stereo systems is presented in this paper, whose main contribution regarding the state of the art on the subject is the estimation of the pitch angle without being affected by the roll angle. The validation of the self-calibration method is accomplished by comparing it with relevant methods of camera pose estimation, where a synthetic sequence is used in order to measure the continuous error with a ground truth. This validation is enriched by the experimental results of the method in real traffic environments.This work was supported by the Spanish Government through the CICYTprojects (TRA2013-48314-C3-1-R and TRA2015-63708-R) and Comunidad de Madrid through SEGVAUTO-TRIES (S2013/MIT-2713).Publicad

Depth Recovery with Rectification using Single-Lens Prism based Stereovision System

Ph.DDOCTOR OF PHILOSOPH

Segment-based stereo matching algorithm with rectification for single-lens bi-prism stereovision system

Ph.DDOCTOR OF PHILOSOPH

EXTRACTING DEPTH INFORMATION FROM STEREO VISION SYSTEM, USING A CORRELATION AND A FEATURE BASED METHODS

This thesis presents a new method to extract depth information from stereo-vision acquisitions using a feature and a correlation based approaches. The main implementation of the proposed method is in the area of Autonomous Pick & Place, using a robotic manipulator. Current vision-guided robotics are still based on a priori training and teaching steps, and still suffer from long response time. The study uses a stereo triangulation setup where two Charged Coupled Devices CCDs are arranged to acquire the scene from two different perspectives. The study discusses the details of two methods to calculate the depth; firstly a correlation matching routine is programmed using a Square Sum Difference SSD algorithm to search for the corresponding points from the left and the right images. The SSD is further modified using an adjustable Region Of Interest ROI along with a center of gravity based calculations. Furthermore, the two perspective images are rectified to reduce the required processing time. Secondly, a feature based approach is proposed to match the objects from the two perspectives. The proposed method implements a search kernel based on the 8-connected neighbor principle. The reported error in depth using the feature method is found to be around 1.2 m

Depth recovery and parameter analysis using single-lens prism based stereovision system

Ph.DDOCTOR OF PHILOSOPH

Stereo matching algorithm by propagation of correspondences and stereo vision instrumentation

A new image processing method is described for measuring the 3-D coordinates of a complex, biological surface. One of the problems in stereo vision is known as the accuracy-precision tradeoff problem. This thesis proposes a new method that promises to solve this problem. To do so, two issues are addressed. First, stereo vision instrumentation methods are described. This instrumentation includes a camera system as well as camera calibration, rectification, matching and triangulation. Second, the approach employs an array of cameras that allow accurate computation of the depth map of a surface by propagation of correspondences through pair-wise camera views. The new method proposed in this thesis employs an array of cameras, and preserves the small baseline advantage by finding accurate correspondences in pairs of adjacent cameras. These correspondences are then propagated along the consecutive pairs of cameras in the array until a large baseline is accomplished. The resulting large baseline disparities are then used for triangulation to achieve advantage of precision in depth measurement. The matching is done by an area-based intensity correlation function called Sum of Squared Differences (SSD). In this thesis, the feasibility of using these data for further processing to achieve surface or volume measurements in the future is discussed

Stereo Correspondence and Depth Recovery of Single-lens Bi-prism Based Stereovision System

Ph.DDOCTOR OF PHILOSOPH

Advances in Stereo Vision

Stereopsis is a vision process whose geometrical foundation has been known for a long time, ever since the experiments by Wheatstone, in the 19th century. Nevertheless, its inner workings in biological organisms, as well as its emulation by computer systems, have proven elusive, and stereo vision remains a very active and challenging area of research nowadays. In this volume we have attempted to present a limited but relevant sample of the work being carried out in stereo vision, covering significant aspects both from the applied and from the theoretical standpoints