Methods for reducing visual discomfort in stereoscopic 3D: A review

This work was supported by the EPSRC Grant EP/M01469X/1, “Geometric Evaluation of Stereoscopic Video”

Acquisition, Modeling, and Augmentation of Reflectance for Synthetic Optical Flow Reference Data

This thesis is concerned with the acquisition, modeling, and augmentation of material reflectance to simulate high-fidelity synthetic data for computer vision tasks. The topic is covered in three chapters: I commence with exploring the upper limits of reflectance acquisition. I analyze state-of-the-art BTF reflectance field renderings and show that they can be applied to optical flow performance analysis with closely matching performance to real-world images. Next, I present two methods for fitting efficient BRDF reflectance models to measured BTF data. Both methods combined retain all relevant reflectance information as well as the surface normal details on a pixel level. I further show that the resulting synthesized images are suited for optical flow performance analysis, with a virtually identical performance for all material types. Finally, I present a novel method for augmenting real-world datasets with physically plausible precipitation effects, including ground surface wetting, water droplets on the windshield, and water spray and mists. This is achieved by projecting the realworld image data onto a reconstructed virtual scene, manipulating the scene and the surface reflectance, and performing unbiased light transport simulation of the precipitation effects

Método de selección automática de algoritmos de correspondencia estéreo en ausencia de ground truth

La correspondencia estéreo es un campo ampliamente estudiado que ha recibido una atención notable en las últimas tres décadas. Es posible encontrar en la literatura un número considerable de propuestas para resolver el problema de correspondencia estéreo. En contraste, las propuestas para evaluar cuantitativamente la calidad de los mapas de disparidad obtenidos a partir de los algoritmos de correspondencia estéreo son relativamente escasas. La selección de un algoritmo de correspondencia estéreo y sus respectivos parámetros para un caso de aplicación particular es un problema no trivial dada la dependencia entre la calidad de la estimación de un mapa de disparidad y el contenido de la escena de interés. Este trabajo de investigación propone una estrategia de selección de algoritmos de correspondencia estéreo a partir de los mapas de disparidad estimados, por medio de un proceso de evaluación en ausencia de ground truth. El método propuesto permitiría a un sistema de visión estéreo adaptarse a posibles cambios en las escenas al ser aplicados a problemas en el mundo real. Esta investigación es de interés para investigadores o ingenieros aplicando visión estéreo en campos de aplicación como la industria.Abstract: The stereo correspondence problem has received significant attention in literature during approximately three decades. A plethora of stereo correspondence algorithms can be found in literature. In contrast, the amount of methods to objectively and quantitatively evaluate the accuracy of disparity maps estimated from stereo correspondence algorithms is relatively low. The application of stereo correspondence algorithms on real world applications is not a trivial problem, mainly due to the existing dependence between the estimated disparity map quality, the algorithms parameter definition and the contents on the assessed scene. In this research a stereo correspondence algorithms selection method is proposed by assessing the quality of estimated disparity maps in absence of ground truth. The proposed method could be used in a stereo vision to increase the system robustness by adapting it to possible changes in real world applications. The contribution of this work is relevant to researchers and engineers applying stereo vision in fields such as industryMaestrí

Performance Metrics and Test Data Generation for Depth Estimation Algorithms

This thesis investigates performance metrics and test datasets used for the evaluation of depth estimation algorithms. Stereo and light field algorithms take structured camera images as input to reconstruct a depth map of the depicted scene. Such depth estimation algorithms are employed in a multitude of practical applications such as industrial inspection and the movie industry. Recently, they have also been used for safety-relevant applications such as driver assistance and computer assisted surgery. Despite this increasing practical relevance, depth estimation algorithms are still evaluated with simple error measures and on small academic datasets. To develop and select suitable and safe algorithms, it is essential to gain a thorough understanding of their respective strengths and weaknesses. In this thesis, I demonstrate that computing average pixel errors of depth estimation algorithms is not sufficient for a thorough and reliable performance analysis. The analysis must also take into account the specific requirements of the given applications as well as the characteristics of the available test data. I propose metrics to explicitly quantify depth estimation results at continuous surfaces, depth discontinuities, and fine structures. These geometric entities are particularly relevant for many applications and challenging for algorithms. In contrast to prevalent metrics, the proposed metrics take into account that pixels are neither spatially independent within an image nor uniformly challenging nor equally relevant. Apart from performance metrics, test datasets play an important role for evaluation. Their availability is typically limited in quantity, quality, and diversity. I show how test data deficiencies can be overcome by using specific metrics, additional annotations, and stratified test data. Using systematic test cases, a user study, and a comprehensive case study, I demonstrate that the proposed metrics, test datasets, and visualizations allow for a meaningful quantitative analysis of the strengths and weaknesses of different algorithms. In contrast to existing evaluation methodologies, application-specific priorities can be taken into account to identify the most suitable algorithms

Analysis and Modeling of Passive Stereo and Time-of-Flight Imaging

This thesis is concerned with the analysis and modeling of effects which cause errors in passive stereo and Time-of-Flight imaging systems. The main topics are covered in four chapters: I commence with a treatment of a system combining Time-of-Flight imaging with passive stereo and show how commonly used fusion models relate to the measurements of the individual modalities. In addition, I present novel fusion techniques capable of improving the depth reconstruction over those obtained separately by either modality. Next, I present a pipeline and uncertainty analysis for the generation of large amounts of reference data for quantitative stereo evaluation. The resulting datasets not only contain reference geometry, but also per pixel measures of reference data uncertainty. The next two parts deal with individual effects observed: Time-of-Flight cameras suffer from range ambiguity if the scene extends beyond a certain distance. I show that it is possible to extend the valid range by changing design parameters of the underlying measurement system. Finally, I present methods that make it possible to amend model violation errors in stereo due to reflections. This is done by means of modeling a limited level of light transport and material properties in the scene

Crowdsourced Interactive Computer Vision

In this thesis we address supervised algorithms and semi-manual working steps which are used for scenarios where automatic computer vision approaches cannot achieve desired results. In the first part we present a semi-automatic method to acquire depth maps for 2D-3D film conversions. Companies that deal with film conversions often rely on fully-manual working steps to ensure maximum control. As an alternative we discuss an approach which uses computer vision methods to reduce processing time but still provides opportunities to interactively control the outcome. As result we receive detailed, smooth and dense depth maps with sharp edges at discontinuities. Part II, which presents the major contribution of this work, deals with human annotations used to assist ground truth acquisition for computer vision applications. To optimize this labour-intensive method, we analyse whether annotations created by different online crowds are an adequate alternative to running such projects with experts. For this purpose we propose different methods for improving acquired annotations. We show that appropriate annotation protocols run with laymen can achieve results comparable to those of experts. Since online crowds have much more users than typical expert groups used to run according projects, the presented approach is a viable alternative for large data acquisition projects