Depth Super-Resolution Meets Uncalibrated Photometric Stereo

A novel depth super-resolution approach for RGB-D sensors is presented. It disambiguates depth super-resolution through high-resolution photometric clues and, symmetrically, it disambiguates uncalibrated photometric stereo through low-resolution depth cues. To this end, an RGB-D sequence is acquired from the same viewing angle, while illuminating the scene from various uncalibrated directions. This sequence is handled by a variational framework which fits high-resolution shape and reflectance, as well as lighting, to both the low-resolution depth measurements and the high-resolution RGB ones. The key novelty consists in a new PDE-based photometric stereo regularizer which implicitly ensures surface regularity. This allows to carry out depth super-resolution in a purely data-driven manner, without the need for any ad-hoc prior or material calibration. Real-world experiments are carried out using an out-of-the-box RGB-D sensor and a hand-held LED light source.Comment: International Conference on Computer Vision (ICCV) Workshop, 201

Epälambertilaiset pinnat ja niiden haasteet konenäössä

This thesis regards non-Lambertian surfaces and their challenges, solutions and study in computer vision. The physical theory for understanding the phenomenon is built first, using the Lambertian reflectance model, which defines Lambertian surfaces as ideally diffuse surfaces, whose luminance is isotropic and the luminous intensity obeys Lambert's cosine law. From these two assumptions, non-Lambertian surfaces violate at least the cosine law and are consequently specularly reflecting surfaces, whose perceived brightness is dependent from the viewpoint. Thus non-Lambertian surfaces violate also brightness and colour constancies, which assume that the brightness and colour of same real-world points stays constant across images. These assumptions are used, for example, in tracking and feature matching and thus non-Lambertian surfaces pose complications for object reconstruction and navigation among other tasks in the field of computer vision. After formulating the theoretical foundation of necessary physics and a more general reflectance model called the bi-directional reflectance distribution function, a comprehensive literature review into significant studies regarding non-Lambertian surfaces is conducted. The primary topics of the survey include photometric stereo and navigation systems, while considering other potential fields, such as fusion methods and illumination invariance. The goal of the survey is to formulate a detailed and in-depth answer to what methods can be used to solve the challenges posed by non-Lambertian surfaces, what are these methods' strengths and weaknesses, what are the used datasets and what remains to be answered by further research. After the survey, a dataset is collected and presented, and an outline of another dataset to be published in an upcoming paper is presented. Then a general discussion about the survey and the study is undertaken and conclusions along with proposed future steps are introduced

The Photometric Effect of Macroscopic Surface Roughness on Sediment Surfaces

The focus of this work was on explaining the effect of macroscopic surface roughness on the reflected light from a soil surface. These questions extend from deciding how to best describe roughness mathematically, to figuring out how to quantify its effect on the spectral reflectance from a soil’s surface. In this document, I provide a background of the fundamental literature in the fields of remote sensing and computer vision that have been instrumental in my research. I then outline the software and hardware tools that I have developed to quantify roughness. This includes a detailed outline of a custom LiDAR operating mode for the GRIT-T goniometer system that was developed and characterized over the course of this research, as well as proposed methods for using convergent images acquired by our goniometer system’s camera to derive useful structure from motion point clouds. These tools and concepts are then used in two experiments that aim to explain the relationship between soil surface roughness and spectral BRF phenomena. In the first experiment, clay sediment samples were gradually pulverized into a smooth powderized state and in steps of reduced surface roughness. Results show that variance in the continuum spectra as a function of viewing angle increased with the roughness of the sediment surface. This result suggests that inter-facet multiple scattering caused a variance in absorption band centering and depth due to an increased path length traveled through the medium. In the second experiment, we examine the performance of the Hapke photometric roughness correction for sand sediment surfaces of controlled sample density. We find that the correction factor potentially underpredicts the effect of shadowing in the forward scattering direction. The percentage difference between forward-modeled BRF measurements and empirically measured BRF measurements is constant across wavelength, suggesting that a factor can be empirically derived. Future results should also investigate the scale at which the photometric correction factor should be applied. Finally, I also outline a structure from motion processing chain aimed at deriving meaningful metrics of vegetation structure. Results show that correlations between these metrics and observed directional reflectance phenomena of chordgrass are strong for peak growing state plants. We observe good agreement between destructive LAI metrics and contact-based LAI metrics

Innovative optical non-contact measurement of respiratory function using photometric stereo

Pulmonary functional testing is very common and widely used in today's clinical environment for testing lung function. The contact based nature of a Spirometer can cause breathing awareness that alters the breathing pattern, affects the amount of air inhaled and exhaled and has hygiene implications. Spirometry also requires a high degree of compliance from the patient, as they have to breathe through a hand held mouth piece. To solve these issues a non-contact computer vision based system was developed for Pulmonary Functional Testing. This employs an improved photometric stereo method that was developed to recover local 3D surface orientation to enable calculation of breathing volumes. Although Photometric Stereo offers an attractive technique for acquiring 3D data using low-cost equipment, inherent limitations in the methodology have served to limit its practical application, particularly in measurement or metrology tasks. Traditional Photometric Stereo assumes that lighting directions at every pixel are the same, which is not usually the case in real applications and especially where the size of object being observed is comparable to the working distance. Such imperfections of the illumination may make the subsequent reconstruction procedures used to obtain the 3D shape of the scene, prone to low frequency geometric distortion and systematic error (bias). Also, the 3D reconstruction of the object results in a geometric shape with an unknown scale. To overcome these problems a novel method of estimating the distance of the object from the camera was developed, which employs Photometric Stereo images without using other additional imaging modality. The method firstly identifies the Lambertian Diffused Maxima regions to calculate the object's distance from the camera, from which the corrected per-pixel light vector is derived and the absolute dimensions of the object can be subsequently estimated. We also propose a new calibration process to allow a dynamic (as an object moves in the field of view) calculation of light vectors for each pixel with little additional computational cost. Experiments performed on synthetic as well as real data demonstrate that the proposed approach offers improved performance, achieving a reduction in the estimated surface normal error by up to 45% as well as the mean height error of reconstructed surface of up to 6 mm. In addition, compared with traditional photometric stereo, the proposed method reduces the mean angular and height error so that it is low, constant and independent of the position of the object placement within a normal working range. A high (0.98) correlation between breathing volume calculated from Photometric Stereo and Spirometer data was observed. This breathing volume is then converted to absolute amount of air by using distance information obtained by Lambertian Diffused Maxima Region. The unique and novel feature of this system is that it views the patients from both front and back and creates a 3D structure of the whole torso. By observing the 3D structure of the torso over time, the amount of air inhaled and exhaled can be estimated

Venus/Mercury swingby with Venus capsule. Preliminary science objectives and experiments for use in advanced mission studies

Venus/Mercury swingby with Venus capsule - preliminary science objectives and experiments for use in advanced mission studie

Material Recognition Meets 3D Reconstruction : Novel Tools for Efficient, Automatic Acquisition Systems

For decades, the accurate acquisition of geometry and reflectance properties has represented one of the major objectives in computer vision and computer graphics with many applications in industry, entertainment and cultural heritage. Reproducing even the finest details of surface geometry and surface reflectance has become a ubiquitous prerequisite in visual prototyping, advertisement or digital preservation of objects. However, today's acquisition methods are typically designed for only a rather small range of material types. Furthermore, there is still a lack of accurate reconstruction methods for objects with a more complex surface reflectance behavior beyond diffuse reflectance. In addition to accurate acquisition techniques, the demand for creating large quantities of digital contents also pushes the focus towards fully automatic and highly efficient solutions that allow for masses of objects to be acquired as fast as possible. This thesis is dedicated to the investigation of basic components that allow an efficient, automatic acquisition process. We argue that such an efficient, automatic acquisition can be realized when material recognition "meets" 3D reconstruction and we will demonstrate that reliably recognizing the materials of the considered object allows a more efficient geometry acquisition. Therefore, the main objectives of this thesis are given by the development of novel, robust geometry acquisition techniques for surface materials beyond diffuse surface reflectance, and the development of novel, robust techniques for material recognition. In the context of 3D geometry acquisition, we introduce an improvement of structured light systems, which are capable of robustly acquiring objects ranging from diffuse surface reflectance to even specular surface reflectance with a sufficient diffuse component. We demonstrate that the resolution of the reconstruction can be increased significantly for multi-camera, multi-projector structured light systems by using overlappings of patterns that have been projected under different projector poses. As the reconstructions obtained by applying such triangulation-based techniques still contain high-frequency noise due to inaccurately localized correspondences established for images acquired under different viewpoints, we furthermore introduce a novel geometry acquisition technique that complements the structured light system with additional photometric normals and results in significantly more accurate reconstructions. In addition, we also present a novel method to acquire the 3D shape of mirroring objects with complex surface geometry. The aforementioned investigations on 3D reconstruction are accompanied by the development of novel tools for reliable material recognition which can be used in an initial step to recognize the present surface materials and, hence, to efficiently select the subsequently applied appropriate acquisition techniques based on these classified materials. In the scope of this thesis, we therefore focus on material recognition for scenarios with controlled illumination as given in lab environments as well as scenarios with natural illumination that are given in photographs of typical daily life scenes. Finally, based on the techniques developed in this thesis, we provide novel concepts towards efficient, automatic acquisition systems

On the Structure and Kinematics of Nebulae around LBVs and LBV Candidates in the LMC

We present a detailed analysis of the morphology and kinematics of nebulae around LBVs and LBV candidates in the Large Magellanic Cloud. HST images and high-resolution Echelle Spectra were used to determine the size, shape, brightness, and expansion velocities of the LBV nebulae around R127, R143, and S61. For S Dor, R71, R99, and R84 we discuss the possible presence of nebular emission, and derive upper limits for the size and lower limits on the expansion velocities of possible nebulae. Including earlier results for the LBV candidates S119 and SK-69 279 we find that in general the nebulae around LBVs in the LMC are comparable in size to those found in the Milky Way. The expansion velocities of the LMC nebulae, however, are significantly lower--by about a factor of 3 to 4--than those of galactic nebulae of comparable size. Galactic and LMC nebulae show about the same diversity of morphologies, but only in the LMC do we find nebulae with outflow. Bipolarity--at least to some degree--is found in nebulae in the LMC as well as in the Milky Way, and manifests a much more general feature among LBV nebulae than previously known.Comment: paper accepted by A&A, 25 pages, 24 figures; paper with images in full resolution available at http://www.astro.ruhr-uni-bochum.de/kweis/publications.htm

Observations of Feedback from Radio-Quiet Quasars - II. Kinematics of Ionized Gas Nebulae

The prevalence and energetics of quasar feedback is a major unresolved problem in galaxy formation theory. In this paper, we present Gemini Integral Field Unit observations of ionized gas around eleven luminous, obscured, radio-quiet quasars at z~0.5 out to ~15 kpc from the quasar; specifically, we measure the kinematics and morphology of [O III]5007 emission. The round morphologies of the nebulae and the large line-of-sight velocity widths (with velocities containing 80% of the emission as high as 1000 km/s combined with relatively small velocity difference across them (from 90 to 520 km/s) point toward wide-angle quasi-spherical outflows. We use the observed velocity widths to estimate a median outflow velocity of 760 km/s, similar to or above the escape velocities from the host galaxies. The line-of-sight velocity dispersion declines slightly toward outer parts of the nebulae (by 3% per kpc on average). The majority of nebulae show blueshifted excesses in their line profiles across most of their extents, signifying gas outflows. For the median outflow velocity, we find a kinetic energy flow between 4x10^{44} and 3x10^{45} erg/s and mass outflow rate between 2000 and 20000 Msun/yr. These values are large enough for the observed quasar winds to have a significant impact on their host galaxies. The median rate of converting bolometric luminosity to kinetic energy of ionized gas clouds is ~2%. We report four new candidates for "super-bubbles" -- outflows that may have broken out of the denser regions of the host galaxy.Comment: 23 pages, 10 figures, 2 tables, accepted for publication in MNRA

Characteristics of flight simulator visual systems

The physical parameters of the flight simulator visual system that characterize the system and determine its fidelity are identified and defined. The characteristics of visual simulation systems are discussed in terms of the basic categories of spatial, energy, and temporal properties corresponding to the three fundamental quantities of length, mass, and time. Each of these parameters are further addressed in relation to its effect, its appropriate units or descriptors, methods of measurement, and its use or importance to image quality

Photometric Depth Super-Resolution

This study explores the use of photometric techniques (shape-from-shading and uncalibrated photometric stereo) for upsampling the low-resolution depth map from an RGB-D sensor to the higher resolution of the companion RGB image. A single-shot variational approach is first put forward, which is effective as long as the target's reflectance is piecewise-constant. It is then shown that this dependency upon a specific reflectance model can be relaxed by focusing on a specific class of objects (e.g., faces), and delegate reflectance estimation to a deep neural network. A multi-shot strategy based on randomly varying lighting conditions is eventually discussed. It requires no training or prior on the reflectance, yet this comes at the price of a dedicated acquisition setup. Both quantitative and qualitative evaluations illustrate the effectiveness of the proposed methods on synthetic and real-world scenarios.Comment: IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2019. First three authors contribute equall