Material Recognition Meets 3D Reconstruction : Novel Tools for Efficient, Automatic Acquisition Systems

For decades, the accurate acquisition of geometry and reflectance properties has represented one of the major objectives in computer vision and computer graphics with many applications in industry, entertainment and cultural heritage. Reproducing even the finest details of surface geometry and surface reflectance has become a ubiquitous prerequisite in visual prototyping, advertisement or digital preservation of objects. However, today's acquisition methods are typically designed for only a rather small range of material types. Furthermore, there is still a lack of accurate reconstruction methods for objects with a more complex surface reflectance behavior beyond diffuse reflectance. In addition to accurate acquisition techniques, the demand for creating large quantities of digital contents also pushes the focus towards fully automatic and highly efficient solutions that allow for masses of objects to be acquired as fast as possible. This thesis is dedicated to the investigation of basic components that allow an efficient, automatic acquisition process. We argue that such an efficient, automatic acquisition can be realized when material recognition "meets" 3D reconstruction and we will demonstrate that reliably recognizing the materials of the considered object allows a more efficient geometry acquisition. Therefore, the main objectives of this thesis are given by the development of novel, robust geometry acquisition techniques for surface materials beyond diffuse surface reflectance, and the development of novel, robust techniques for material recognition. In the context of 3D geometry acquisition, we introduce an improvement of structured light systems, which are capable of robustly acquiring objects ranging from diffuse surface reflectance to even specular surface reflectance with a sufficient diffuse component. We demonstrate that the resolution of the reconstruction can be increased significantly for multi-camera, multi-projector structured light systems by using overlappings of patterns that have been projected under different projector poses. As the reconstructions obtained by applying such triangulation-based techniques still contain high-frequency noise due to inaccurately localized correspondences established for images acquired under different viewpoints, we furthermore introduce a novel geometry acquisition technique that complements the structured light system with additional photometric normals and results in significantly more accurate reconstructions. In addition, we also present a novel method to acquire the 3D shape of mirroring objects with complex surface geometry. The aforementioned investigations on 3D reconstruction are accompanied by the development of novel tools for reliable material recognition which can be used in an initial step to recognize the present surface materials and, hence, to efficiently select the subsequently applied appropriate acquisition techniques based on these classified materials. In the scope of this thesis, we therefore focus on material recognition for scenarios with controlled illumination as given in lab environments as well as scenarios with natural illumination that are given in photographs of typical daily life scenes. Finally, based on the techniques developed in this thesis, we provide novel concepts towards efficient, automatic acquisition systems

Methods for Generating High-Fidelity Trace Chemical Residue Reflectance Signatures for Active Spectroscopy Classification Applications

Standoff detection and identification of trace chemicals in hyperspectral infrared images is an enabling capability in a variety of applications relevant to defense, law enforcement, and intelligence communities. Performance of these methods is impacted by the spectral signature variability due to the presence of contaminants, surface roughness, nonlinear effects, etc. Though multiple classes of algorithms exist for the detection and classification of these signatures, they are limited by the availability of relevant reference datasets. In this work, we first address the lack of physics-based models that can accurately predict trace chemical spectra. Most available models assume that the chemical takes the form of spherical particles or uniform thin films. A more realistic chemical presentation that could be encountered is that of a non-uniform chemical film that is deposited after evaporation of the solvent which contained the chemical. This research presents an improved signature model for this type of solid film. The proposed model, called sparse transfer matrix (STM), includes a log-normal distribution of film thicknesses and is found to reduce the root-mean-square error between simulated and measured data by about 25% when compared with either the particle or uniform thin film models. When applied to measured data, the sparse transfer matrix model provides a 0.10-0.28 increase in classification accuracy over traditional models. There remain limitations in the STM model which prevent the predicted spectra from being well-matched to the measured data in some cases. To overcome this, we leverage the field of domain adaptation to translate data from the simulated to the measured data domain. This thesis presents the first one-dimensional (1D) conditional generative adversarial network (GAN) to perform spectrum-to-spectrum translation of reflectance signatures. We apply the 1D conditional GAN to a library of simulated spectra and quantify the improvement with the translated library. The method demonstrates an increase in overall classification accuracy to 0.723 from the accuracy of 0.622 achieved using the STM model when tested on real data. However, the performance improvement is biased towards data included in the GAN training set. The next phase of the research focuses on learning models that are more robust to different parameter combinations for which we do not have measured data. This part of the research leverages elements from the field of theory-guided data science. Specifically, we develop a physics-guided neural network (PGNN) for predicting chemical reflectance for a set of parameterized inputs that is more accurate than the state-of-the-art physics-based signature model for chemical residues. After training the PGNN, we use it to generate a library of predicted spectra for training a classifier. We compare the classification accuracy when using this PGNN library versus a library generated by the physics-based model. Using the PGNN, the average classification accuracy increases to 0.813 on real chemical reflectance data, including data from chemicals not included in the PGNN training set. The products of this thesis work include methods for producing realistic trace chemical residue reflectance signatures as well as demonstrations of improved performance in active spectroscopy classification applications. These methods provide great value to a range of scientific communities. The novel STM signature model enables existing spectroscopy sensors and algorithms to perform well on real-world problems where chemical contaminants are non-uniform. The 1D conditional GAN is the first of its kind and can be applied to many other 1D datasets, such as audio and other time-series data. Finally, the application of theory-guided data science to the trace chemical problem not only enhances the quality of results for known targets and backgrounds, but also increases the robustness to new targets

Image-based terrain modeling with thematic mapper applied to resolving the limit of Holocene Lake expansion in the Great Salt Lake Desert, Utah, part 1

The LANDSAT Thematic Mapper (TM) scanner records reflected solar energy from the earth's surface in six wavelength regions, or bands, and one band that records emitted energy in the thermal region, giving a total of seven bands. Useful research was extracted about terrain morphometry from remote sensing measurements and this information is used in an image-based terrain model for selected coastal geomorphic features in the Great Salt Lake Desert (GSLD). Technical developments include the incorporation of Aerial Profiling of Terrain System (APTS) data in satellite image analysis, and the production and use of 3-D surface plots of TM reflectance data. Also included in the technical developments is the analysis of the ground control point spatial distribution and its affects on geometric correction, and the terrain mapping procedure; using satellite data in a way that eliminates the need to degrade the data by resampling. The most common approach for terrain mapping with multispectral scanner data includes the techniques of pattern recognition and image classification, as opposed to direct measurement of radiance for identification of terrain features. The research approach in this investigation was based on an understanding of the characteristics of reflected light resulting from the variations in moisture and geometry related to terrain as described by the physical laws of radiative transfer. The image-based terrain model provides quantitative information about the terrain morphometry based on the physical relationship between TM data, the physical character of the GSLD, and the APTS measurements

Procedures for the analysis and use of multiple view angle image data

It is recognised that the majority of vegetative cover types have anisotropic reflectance characteristics that are largely a function of their canopy geometry. Several studies have made attempts at formulating methods for the use of data remotely sensed from off-nadir directions. The best of these methods attempt to utilise the "extra" information implicitly contained in off-nadir image datasets. In this study, an attempt is made to extract information concerning agro-physical parameters of a number of vegetative cover types using imagery acquired by an airborne sensor, the Daedalus Airborne Thematic Mapper (ATM). It is also recognised in the literature that the nature of spatial variance in images is related to the size and distribution of the objects in the scene and the sampling characteristics of the sensor. In previous work this relationship has been explored by examining scenes using images of varying spatial resolutions, using a number of measurements of spatial variance. The underlying trend of these measurements is then used to interpret the nature of the objects in the scene. No previous work exists which attempts to utilise the change in variance of images acquired at different off-nadir view angles. In this study, the understanding of this relationship is developed by examining the change in variance of a number of vegetative cover types from multiple view angle image datasets. The geometry of the ATM sensor is derived to allow an understanding of the sampling characteristics of the instrument. Two important geometric factors are established: first, the area of the ground resolution element increases with view angle, which effectively reduces spatial resolution at off-nadir angles; and second, overlap between adjacent ground resolution elements increases with view angle, increasing the spatial auto-correlation between these samples. The effects of illumination, atmosphere and topography can all influence variance in an image. A parametric procedure for normalising multiple view angle (and therefore multitemporal) datasets for these factors is developed, based upon the production of reflectance images using a sky radiance model of the spectral and spatial distributions of irradiance, ground measurements of irradiance, and a digital terrain model of the study site. Finally, it is shown that image variance is likely to decrease at off-nadir view angles, the magnitude of this decrease being related to the sensor geometry and (more importantly) the geometry of the canopy. By a simple statistical analytical procedure it is possible to construct broad classes within which the nature of the canopy can be classified

Visual Prototyping of Cloth

Realistic visualization of cloth has many applications in computer graphics. An ongoing research problem is how to best represent and capture appearance models of cloth, especially when considering computer aided design of cloth. Previous methods can be used to produce highly realistic images, however, possibilities for cloth-editing are either restricted or require the measurement of large material databases to capture all variations of cloth samples. We propose a pipeline for designing the appearance of cloth directly based on those elements that can be changed within the production process. These are optical properties of fibers, geometrical properties of yarns and compositional elements such as weave patterns. We introduce a geometric yarn model, integrating state-of-the-art textile research. We further present an approach to reverse engineer cloth and estimate parameters for a procedural cloth model from single images. This includes the automatic estimation of yarn paths, yarn widths, their variation and a weave pattern. We demonstrate that we are able to match the appearance of original cloth samples in an input photograph for several examples. Parameters of our model are fully editable, enabling intuitive appearance design. Unfortunately, such explicit fiber-based models can only be used to render small cloth samples, due to large storage requirements. Recently, bidirectional texture functions (BTFs) have become popular for efficient photo-realistic rendering of materials. We present a rendering approach combining the strength of a procedural model of micro-geometry with the efficiency of BTFs. We propose a method for the computation of synthetic BTFs using Monte Carlo path tracing of micro-geometry. We observe that BTFs usually consist of many similar apparent bidirectional reflectance distribution functions (ABRDFs). By exploiting structural self-similarity, we can reduce rendering times by one order of magnitude. This is done in a process we call non-local image reconstruction, which has been inspired by non-local means filtering. Our results indicate that synthesizing BTFs is highly practical and may currently only take a few minutes for small BTFs. We finally propose a novel and general approach to physically accurate rendering of large cloth samples. By using a statistical volumetric model, approximating the distribution of yarn fibers, a prohibitively costly, explicit geometric representation is avoided. As a result, accurate rendering of even large pieces of fabrics becomes practical without sacrificing much generality compared to fiber-based techniques

Characterization and Discrimination of Selected Vegetation Canopies from Field Observations of Bidirectional Reflectances

A full evaluation of the bidirectional reflectance properties of different vegetated surfaces was limited in past studies by instrumental inadequacies. With the development of the PARABOLA, it is now possible to sample reflectances from a large number of view angles in a short period of time, maintaining an almost constant solar zenith angle. PARABOLA data collected over five different canopies in Texas are analyzed. The objective of this investigation was to evaluate the intercanopy and intracanopy differences in bidirectional reflectance patterns. Particular attention was given to the separability of canopy types using different view angles for the red and the near infrared (NIR) spectral bands. Comparisons were repeated for different solar zenith angles. Statistical and other quantitative techniques were used to assess these differences. For the canopies investigated, the greatest reflectances were found in the backscatter direction for both bands. Canopy discrimination was found to vary with both view angle and the spectral reflectance band considered, the forward scatter view angles being most suited to observations in the NIR and backscatter view angles giving better results in the red band. Because of different leaf angle distribution characteristics, discrimination was found to be better at small solar zenith angles in both spectral bands

A Dataset of Multi-Illumination Images in the Wild

Collections of images under a single, uncontrolled illumination have enabled the rapid advancement of core computer vision tasks like classification, detection, and segmentation. But even with modern learning techniques, many inverse problems involving lighting and material understanding remain too severely ill-posed to be solved with single-illumination datasets. To fill this gap, we introduce a new multi-illumination dataset of more than 1000 real scenes, each captured under 25 lighting conditions. We demonstrate the richness of this dataset by training state-of-the-art models for three challenging applications: single-image illumination estimation, image relighting, and mixed-illuminant white balance.Comment: ICCV 201

Proceedings of the Second Airborne Imaging Spectrometer Data Analysis Workshop

Topics addressed include: calibration, the atmosphere, data problems and techniques, geological research, and botanical and geobotanical research

a Berlin case study

Durch den Prozess der Urbanisierung verändert die Menschheit die Erdoberfläche in großem Ausmaß und auf unwiederbringliche Weise. Die optische Fernerkundung ist eine Art der Erdbeobachtung, die das Verständnis dieses dynamischen Prozesses und seiner Auswirkungen erweitern kann. Die vorliegende Arbeit untersucht, inwiefern hyperspektrale Daten Informationen über Versiegelung liefern können, die der integrierten Analyse urbaner Mensch-Umwelt-Beziehungen dienen. Hierzu wird die Verarbeitungskette von Vorverarbeitung der Rohdaten bis zur Erstellung referenzierter Karten zu Landbedeckung und Versiegelung am Beispiel von Hyperspectral Mapper Daten von Berlin ganzheitlich untersucht. Die traditionelle Verarbeitungskette wird mehrmals erweitert bzw. abgewandelt. So wird die radiometrische Vorverarbeitung um die Normalisierung von Helligkeitsgradienten erweitert, welche durch die direktionellen Reflexionseigenschaften urbaner Oberflächen entstehen. Die Klassifikation in fünf spektral komplexe Landnutzungsklassen wird mit Support Vector Maschinen ohne zusätzliche Merkmalsextraktion oder Differenzierung von Subklassen durchgeführt...thesi

A photogeologic comparison of Skylab and LANDSAT images of southwestern Nevada and southeastern California

There are no author-identified significant results in this report