Refractive shape from light field distortion

Acquiring transparent, refractive objects is challenging as these kinds of objects can only be observed by analyzing the distortion of reference background patterns. We present a new, single image approach to reconstructing thin transparent surfaces, such as thin solids or surfaces of fluids. Our method is based on observing the distortion of light field background illumination. Light field probes have the potential to encode up to four dimensions in varying colors and intensities: spatial and angular variation on the probe surface; commonly employed reference patterns are only two-dimensional by coding either position or angle on the probe. We show that the additional information can be used to reconstruct refractive surface normals and a sparse set of control points from a single photograph

Material Recognition Meets 3D Reconstruction : Novel Tools for Efficient, Automatic Acquisition Systems

For decades, the accurate acquisition of geometry and reflectance properties has represented one of the major objectives in computer vision and computer graphics with many applications in industry, entertainment and cultural heritage. Reproducing even the finest details of surface geometry and surface reflectance has become a ubiquitous prerequisite in visual prototyping, advertisement or digital preservation of objects. However, today's acquisition methods are typically designed for only a rather small range of material types. Furthermore, there is still a lack of accurate reconstruction methods for objects with a more complex surface reflectance behavior beyond diffuse reflectance. In addition to accurate acquisition techniques, the demand for creating large quantities of digital contents also pushes the focus towards fully automatic and highly efficient solutions that allow for masses of objects to be acquired as fast as possible. This thesis is dedicated to the investigation of basic components that allow an efficient, automatic acquisition process. We argue that such an efficient, automatic acquisition can be realized when material recognition "meets" 3D reconstruction and we will demonstrate that reliably recognizing the materials of the considered object allows a more efficient geometry acquisition. Therefore, the main objectives of this thesis are given by the development of novel, robust geometry acquisition techniques for surface materials beyond diffuse surface reflectance, and the development of novel, robust techniques for material recognition. In the context of 3D geometry acquisition, we introduce an improvement of structured light systems, which are capable of robustly acquiring objects ranging from diffuse surface reflectance to even specular surface reflectance with a sufficient diffuse component. We demonstrate that the resolution of the reconstruction can be increased significantly for multi-camera, multi-projector structured light systems by using overlappings of patterns that have been projected under different projector poses. As the reconstructions obtained by applying such triangulation-based techniques still contain high-frequency noise due to inaccurately localized correspondences established for images acquired under different viewpoints, we furthermore introduce a novel geometry acquisition technique that complements the structured light system with additional photometric normals and results in significantly more accurate reconstructions. In addition, we also present a novel method to acquire the 3D shape of mirroring objects with complex surface geometry. The aforementioned investigations on 3D reconstruction are accompanied by the development of novel tools for reliable material recognition which can be used in an initial step to recognize the present surface materials and, hence, to efficiently select the subsequently applied appropriate acquisition techniques based on these classified materials. In the scope of this thesis, we therefore focus on material recognition for scenarios with controlled illumination as given in lab environments as well as scenarios with natural illumination that are given in photographs of typical daily life scenes. Finally, based on the techniques developed in this thesis, we provide novel concepts towards efficient, automatic acquisition systems

Computational Schlieren Photography with Light Field Probes

We introduce a new approach to capturing refraction in transparent media, which we call light field background oriented Schlieren photography. By optically coding the locations and directions of light rays emerging from a light field probe, we can capture changes of the refractive index field between the probe and a camera or an observer. Our prototype capture setup consists of inexpensive off-the-shelf hardware, including inkjet-printed transparencies, lenslet arrays, and a conventional camera. By carefully encoding the color and intensity variations of 4D light field probes, we show how to code both spatial and angular information of refractive phenomena. Such coding schemes are demonstrated to allow for a new, single image approach to reconstructing transparent surfaces, such as thin solids or surfaces of fluids. The captured visual information is used to reconstruct refractive surface normals and a sparse set of control points independently from a single photograph.Natural Sciences and Engineering Research Council of CanadaAlfred P. Sloan FoundationUnited States. Defense Advanced Research Projects Agency. Young Faculty Awar

Depth from Refraction Using a Transparent Medium with Unknown Pose and Refractive Index

published_or_final_versio

Investigation of Sea Ice Using Multiple Synthetic Aperture Radar Acquisitions

The papers of this thesis are not available in Munin. Paper I: Yitayew, T. G., Ferro-Famil, L., Eltoft, T. & Tebaldini, S. (2017). Tomographic imaging of fjord ice using a very high resolution ground-based SAR system. Available in IEEE Transactions on Geoscience and Remote Sensing, 55 (2):698-714. Paper II: Yitayew, T. G., Ferro-Famil, L., Eltoft, T. & Tebaldini, S. (2017). Lake and fjord ice imaging using a multifrequency ground-based tomographic SAR system. Available in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 10(10):4457-4468. Paper III: Yitayew, T. G., Divine, D. V., Dierking, W., Eltoft, T., Ferro-Famil, L., Rosel, A. & Negrel, J. Validation of Sea ice Topographic Heights Derived from TanDEMX Interferometric SAR Data with Results from Laser Profiler and Photogrammetry. (Manuscript).The thesis investigates imaging in the vertical direction of different types of ice in the arctic using synthetic aperture radar (SAR) tomography and SAR interferometry. In the first part, the magnitude and the positions of the dominant scattering contributions within snow covered fjord and lake ice layers are effectively identified by using a very high resolution ground-based tomographic SAR system. Datasets collected at multiple frequencies and polarizations over two test sites in Tromsø area, northern Norway, are used for characterizing the three-dimensional response of snow and ice. The presented experimental results helped to improve our understanding of the interaction between radar waves and snow and ice layers. The reconstructed radar responses are also used for estimating the refractive indices and the vertical positions of the different sub-layers of snow and ice. The second part of the thesis deals with the retrieval of the surface topography of multi-year sea ice using SAR interferometry. Satellite acquisitions from TanDEM-X over the Svalbard area are used for analysis. The retrieved surface height is validated by using overlapping helicopter-based stereo camera and laser profiler measurements, and a very good agreement has been found. The work contributes to an improved understanding regarding the potential of SAR tomography for imaging the vertical scattering distribution of snow and ice layers, and for studying the influence of both sensor parameters such as its frequency and polarization and scene properties such as layer stratification, air bubbles and small-scale roughness of the interfaces on snow and ice backscattered signal. Moreover, the presented results reveal the potential of SAR interferometry for retrieving the surface topography of sea ice

Depth and IMU aided image deblurring based on deep learning

Abstract. With the wide usage and spread of camera phones, it becomes necessary to tackle the problem of the image blur. Embedding a camera in those small devices implies obviously small sensor size compared to sensors in professional cameras such as full-frame Digital Single-Lens Reflex (DSLR) cameras. As a result, this can dramatically affect the collected amount of photons on the image sensor. To overcome this, a long exposure time is needed, but with slight motions that often happen in handheld devices, experiencing image blur is inevitable. Our interest in this thesis is the motion blur that can be caused by the camera motion, scene (objects in the scene) motion, or generally the relative motion between the camera and scene. We use deep neural network (DNN) models in contrary to conventional (non DNN-based) methods which are computationally expensive and time-consuming. The process of deblurring an image is guided by utilizing the scene depth and camera’s inertial measurement unit (IMU) records. One of the challenges of adopting DNN solutions is that a relatively huge amount of data is needed to train the neural network. Moreover, several hyperparameters need to be tuned including the network architecture itself. To train our network, a novel and promising method of synthesizing spatially-variant motion blur is proposed that considers the depth variations in the scene, which showed improvement of results against other methods. In addition to the synthetic dataset generation algorithm, a real blurry and sharp dataset collection setup is designed. This setup can provide thousands of real blurry and sharp images which can be of paramount benefit in DNN training or fine-tuning