An evaluation framework for stereo-based driver assistance

This is the post-print version of the Article - Copyright @ 2012 Springer VerlagThe accuracy of stereo algorithms or optical flow methods is commonly assessed by comparing the results against the Middlebury database. However, equivalent data for automotive or robotics applications rarely exist as they are difficult to obtain. As our main contribution, we introduce an evaluation framework tailored for stereo-based driver assistance able to deliver excellent performance measures while circumventing manual label effort. Within this framework one can combine several ways of ground-truthing, different comparison metrics, and use large image databases. Using our framework we show examples on several types of ground truthing techniques: implicit ground truthing (e.g. sequence recorded without a crash occurred), robotic vehicles with high precision sensors, and to a small extent, manual labeling. To show the effectiveness of our evaluation framework we compare three different stereo algorithms on pixel and object level. In more detail we evaluate an intermediate representation called the Stixel World. Besides evaluating the accuracy of the Stixels, we investigate the completeness (equivalent to the detection rate) of the StixelWorld vs. the number of phantom Stixels. Among many findings, using this framework enables us to reduce the number of phantom Stixels by a factor of three compared to the base parametrization. This base parametrization has already been optimized by test driving vehicles for distances exceeding 10000 km

Ground Truthing: Reimagining the Indigenous Rainforests of BC’s North Coast by Derrick Stacey Denholm

Review of Derrick Stacey Denholm\u27s Ground Truthing: Reimagining the Indigenous Rainforests of BC’s North Coast

Sensor-assisted Video Mapping of the Seafloor

In recent years video surveys have become an increasingly important ground-truthing of acousticseafloor characterization and benthic habitat mapping studies. However, the ground-truthing and detailed characterization provided by video are still typically done using sparse sample imagery supplemented by physical samples. Combining single video frames in a seamless mosaic can provide a tool by which imagery has significant areal coverage, while at the same time showing small fauna and biological features at mm resolution. The generation of such a mosaic is a challenging task due to height variations of the imaged terrain and decimeter scale knowledge of camera position. This paper discusses the current role of underwater video survey, and the potential for generating consistent, quantitative image maps using video data, accompanied by data that can be measured by auxiliary sensors with sufficient accuracy, such as camera tilt and heading, and their use in automated mosaicking techniques. The camera attitude data also provide the necessary information to support the development of a video collage. The collage provides a quick look at the large spatial scale features in a scene and can be used to pinpoint regions that are likely to yield useful information when rendered into high-resolution mosaics. It is proposed that high quality mosaics can be produced using consumer-grade cameras and low-cost sensors, thereby allowing for the economical scientific video surveys. A case study is presented with the results from benthic habitat mapping and the ground-truthing ofseafloor acoustic data using both real underwater imagery and simulations. A computer modeling of the process of video data acquisition (in particular on a non-flat terrain) allows for a better understanding of the main sources of error in mosaic generation and for the choice of near-optimal processing strategies. Various spatial patterns of video survey coverage are compared and it is shown that some patterns have certain advantages in the sense of accumulated error and overall mosaic accuracy

Blending single beam RoxAnn and multi-beam swathe QTC hydro-acoustic discrimination techniques for the Stonehaven area, Scotland, UK

Surface properties of the seabed in a 180 km2 area of coastal waters (14-57 m depth) off northeast Scotland were mapped by hydro-acoustic discrimination using single and multi-beam echosounders linked to signal processing systems (RoxAnn for the single beam, and Questor Tangent Corporation (QTC) Multiview for the multibeam). Subsequently, two ground truthing surveys were carried out, using grab and TV sampling. The RoxAnn and QTC-Multiview outputs showed strong similarity in their classifications of seabed types. Classifications generated by QTC-Multiview were used to supervise those based on seabed roughness and hardness indices produced by the RoxAnn system and thereby develop a ‘blended’ map based on both systems. The resulting hydro-acoustic classes agreed well with a cluster analysis of data on sediment grain sizes from the grab sampling, and indicated that the area could be described by distinct regions of surface texture and surficial sediments ranging from muddy sand to boulders and rock

Small-Scale Irrigation Mapping (SSIM) as a tool for improving and validating irrigated area maps: contextual approach and lessons learnt in Burkina Faso

Recent rapid expansion of private small-scale irrigation provides an opportunity to improve livelihoods and food security, but requires knowledge of where it is happening, in order to sustainably manage water use. Concerns are rising regarding the negative impacts of unchecked expansion of irrigation on downstream water quality and availability, particularly when using sub-optimal practices (de Fraiture et al. 2014; Domenech and Ringler 2013; Shah 2007). Therefore, for informed planning of potential sustainable irrigation expansion, policy makers and resource managers at the national level are interested in maps of the current extent of small-scale irrigation. Although several maps of irrigated areas have been produced for Burkina Faso, these maps, often of 250 meter (m), 300 m or 1 kilometer (km) resolution, are of too low resolution to account for scattered irrigation on areas smaller than 1 hae. Small-scale irrigation in Burkina Faso is typically carried out on individual plots of less than a quarter of hectare, with a small proportion on groups of fields no larger than one hectare, implying that existing maps are not reliably capturing the true extent and distribution of small-scale irrigation in the country

Towards Improved Paper-Based Election Technology

Resources are presented for fostering paper-based election technology. They comprise a diverse collection of real and simulated ballot and survey images, and software tools for ballot synthesis, registration, segmentation, and ground-truthing. The grids underlying the designated location of voter marks are extracted from 13,315 degraded ballot images. The actual skew angles of sample ballots, recorded as part of complete ballot descriptions compiled with the interactive ground-truthing tool, are compared with their automatically extracted parameters. The average error is 0.1 degrees. These results provide a baseline for the application of digital image analysis to the scrutiny of electoral ballots

An Analysis of Binarization Ground Truthing

The accuracy of a binarization algorithm is often calculated relative to a ground truth image. Except for synthetically generated images, no ground truth image exists. Evaluating binarization on real images is preferred. The ground truthing between and among different operators is compared. Four direct metrics were used. The variability of the results of five different automatic binarization algorithms were compared to that of manual ground truth results. Significant variability in the ground truth results was found

The Robust Reading Competition Annotation and Evaluation Platform

The ICDAR Robust Reading Competition (RRC), initiated in 2003 and re-established in 2011, has become a de-facto evaluation standard for robust reading systems and algorithms. Concurrent with its second incarnation in 2011, a continuous effort started to develop an on-line framework to facilitate the hosting and management of competitions. This paper outlines the Robust Reading Competition Annotation and Evaluation Platform, the backbone of the competitions. The RRC Annotation and Evaluation Platform is a modular framework, fully accessible through on-line interfaces. It comprises a collection of tools and services for managing all processes involved with defining and evaluating a research task, from dataset definition to annotation management, evaluation specification and results analysis. Although the framework has been designed with robust reading research in mind, many of the provided tools are generic by design. All aspects of the RRC Annotation and Evaluation Framework are available for research use.Comment: 6 pages, accepted to DAS 201