An evaluation framework for stereo-based driver assistance

This is the post-print version of the Article - Copyright @ 2012 Springer VerlagThe accuracy of stereo algorithms or optical flow methods is commonly assessed by comparing the results against the Middlebury database. However, equivalent data for automotive or robotics applications rarely exist as they are difficult to obtain. As our main contribution, we introduce an evaluation framework tailored for stereo-based driver assistance able to deliver excellent performance measures while circumventing manual label effort. Within this framework one can combine several ways of ground-truthing, different comparison metrics, and use large image databases. Using our framework we show examples on several types of ground truthing techniques: implicit ground truthing (e.g. sequence recorded without a crash occurred), robotic vehicles with high precision sensors, and to a small extent, manual labeling. To show the effectiveness of our evaluation framework we compare three different stereo algorithms on pixel and object level. In more detail we evaluate an intermediate representation called the Stixel World. Besides evaluating the accuracy of the Stixels, we investigate the completeness (equivalent to the detection rate) of the StixelWorld vs. the number of phantom Stixels. Among many findings, using this framework enables us to reduce the number of phantom Stixels by a factor of three compared to the base parametrization. This base parametrization has already been optimized by test driving vehicles for distances exceeding 10000 km

Probabilistic Lane Association

Lane association is the problem of determining in which lane a vehicle is currently driving, which is of interest for automated driving where the vehicle must understand its surroundings. Limited to highway scenarios, a method combining data from different sensors to extract information about the currently associated lane is presented. The suggested method splits the problem in two main parts, lane change identification and road edge detection. The lane change identification mainly uses information from the camera to model the lateral movement on the road and identifies the lane changes as a relative position on the road. This part is implemented with a particle filter. The road edge detection enters radar detections to an iterated Kalman filter and estimates the distances to the road edges. Finally, a combination of the filter outputs makes it possible to compute an absolute position on the road. Comparing the relative and absolute positioning then leads to the desired lane association estimate. The results produced are reliable and encourages to continue approaching this problem in a similar manner, but the current implementation is computationally heavy

Lane and Road Marking Detection with a High Resolution Automotive Radar for Automated Driving

Die Automobilindustrie erlebt gerade einen beispiellosen Wandel, und die Fahrerassistenz und das automatisierte Fahren spielen dabei eine entscheidende Rolle. Automatisiertes Fahren System umfasst haupts\"achlich drei Schritte: Wahrnehmung und Modellierung der Umgebung, Fahrtrichtungsplanung, und Fahrzeugsteuerung. Mit einer guten Wahrnehmung und Modellierung der Umgebung kann ein Fahrzeug Funktionen wie intelligenter Tempomat, Notbremsassistent, Spurwechselassistent, usw. erfolgreich durchf\"uhren. F\"ur Fahrfunktionen, die die Fahrpuren erkennen m\"ussen, werden gegenw\"artig ausnahmslos Kamerasensoren eingesetzt. Bei wechselnden Lichtverh\"altnissen, unzureichender Beleuchtung oder bei Sichtbehinderungen z.B. durch Nebel k\"onnen Videokameras aber empfindlich gest\"ort werden. Um diese Nachteile auszugleichen, wird in dieser Doktorarbeit eine \glqq Radar\textendash taugliche\grqq{} Fahrbahnmakierungerkennung entwickelt, mit der das Fahrzeug die Fahrspuren bei allen Lichtverh\"altnissen erkennen kann. Dazu k\"onnen bereits im Fahrzeug verbaute Radare eingesetzt werden. Die heutigen Fahrbahnmarkierungen k\"onnen mit Kamerasensoren sehr gut erfasst werden. Wegen unzureichender R\"uckstreueigenschaften der existierenden Fahrbahnmarkierungen f\"ur Radarwellen werden diese vom Radar nicht erkannt. Um dies zu bewerkstelligen, werden in dieser Arbeit die R\"uckstreueigenschaften von verschiedenen Reflektortypen, sowohl durch Simulationen als auch mit praktischen Messungen, untersucht und ein Reflektortyp vorgeschlagen, der zur Verarbeitung in heutige Fahrbahnmakierungen oder sogar f\"ur direkten Verbau in der Fahrbahn geeignet ist. Ein weiterer Schwerpunkt dieser Doktorarbeit ist der Einsatz von K\"unstliche Intelligenz (KI), um die Fahrspuren auch mit Radar zu detektieren und zu klassifizieren. Die aufgenommenen Radardaten werden mittels semantischer Segmentierung analysiert und Fahrspurverl\"aufe sowie Freifl\"achenerkennung detektiert. Gleichzeitig wird das Potential von KI\textendash tauglichen Umgebungverstehen mit bildgebenden Radardaten aufgezeigt

Integrating Millimeter Wave Radar with a Monocular Vision Sensor for On-Road Obstacle Detection Applications

This paper presents a systematic scheme for fusing millimeter wave (MMW) radar and a monocular vision sensor for on-road obstacle detection. As a whole, a three-level fusion strategy based on visual attention mechanism and driver’s visual consciousness is provided for MMW radar and monocular vision fusion so as to obtain better comprehensive performance. Then an experimental method for radar-vision point alignment for easy operation with no reflection intensity of radar and special tool requirements is put forward. Furthermore, a region searching approach for potential target detection is derived in order to decrease the image processing time. An adaptive thresholding algorithm based on a new understanding of shadows in the image is adopted for obstacle detection, and edge detection is used to assist in determining the boundary of obstacles. The proposed fusion approach is verified through real experimental examples of on-road vehicle/pedestrian detection. In the end, the experimental results show that the proposed method is simple and feasible

Adaptive Multi-sensor Perception for Driving Automation in Outdoor Contexts

In this research, adaptive perception for driving automation is discussed so as to enable a vehicle to automatically detect driveable areas and obstacles in the scene. It is especially designed for outdoor contexts where conventional perception systems that rely on a priori knowledge of the terrain's geometric properties, appearance properties, or both, is prone to fail, due to the variability in the terrain properties and environmental conditions. In contrast, the proposed framework uses a self-learning approach to build a model of the ground class that is continuously adjusted online to reflect the latest ground appearance. The system also features high flexibility, as it can work using a single sensor modality or a multi-sensor combination. In the context of this research, different embodiments have been demonstrated using range data coming from either a radar or a stereo camera, and adopting self-supervised strategies where monocular vision is automatically trained by radar or stereo vision. A comprehensive set of experimental results, obtained with different ground vehicles operating in the field, are presented to validate and assess the performance of the system

An Investigation into Segmenting Traffic Images Using Various Types of Graph Cuts

In computer vision, graph cuts are a way of segmenting an image into multiple areas. Graphs are built using one node for each pixel in the image combined with two extra nodes, known as the source and the sink. Each node is connected to several other nodes using edges, and each edge has a specific weight. Using different weighting schemes, different segmentations can be performed based on the properties used to create the weights. The cuts themselves are performed using an implementation of a solution to the maximum flow problem, which is then changed into a minimum cut according to the max-flow/min-cut theorem. In this thesis, several types of graph cuts are investigated with the intent to use one of them to segment traffic images. Each of these variations of graph cut is explained in detail and compared to the others. Then, one is chosen to be used to detect traffic. Several weighting schemes based on grayscale value differences, pixel variances, and mean pixel values from the test footage are presented to allow for the segmentation of video footage into vehicles and backgrounds using graph cuts. Our method of segmenting traffic images via graph cuts is then tested on several videos of traffic in various lighting conditions and locations. Finally, we compare our proposed method to a similarly performing method: background subtraction

The Continuing Quest for Missile Defense: When Lofty Goals Confront Reality

For almost three quarters of a century, the United States has spent billions of dollars and countless person-hours in the pursuit of a national missile defense system that would protect the country from intercontinental ballistic missiles (ICBM) carrying nuclear warheads. The system currently in place consists of 44 long-range antiballistic missiles stationed in Alaska and California to protect the United States from a possible nuclear weapon carrying ICBM attack from North Korea. After all this effort, this system is still imperfect, being successful only 10 out of 18 tests. This book will provide an historical description of past efforts in national missile defenses to understand the technical difficulties involved. It will also explain how national security concerns, the evolving international environment, and the complexities of US politics have all affected the story. The book will also describe the current systems in place to protect allies and troops in the field from the threat of shorter range missiles. Finally, the book will describe the current US vision for the future of missile defenses and provide some suggestions for alternative paths.https://cupola.gettysburg.edu/books/1142/thumbnail.jp