On Recognizing Transparent Objects in Domestic Environments Using Fusion of Multiple Sensor Modalities

Current object recognition methods fail on object sets that include both diffuse, reflective and transparent materials, although they are very common in domestic scenarios. We show that a combination of cues from multiple sensor modalities, including specular reflectance and unavailable depth information, allows us to capture a larger subset of household objects by extending a state of the art object recognition method. This leads to a significant increase in robustness of recognition over a larger set of commonly used objects.Comment: 12 page

Gradient metasurfaces: a review of fundamentals and applications

In the wake of intense research on metamaterials the two-dimensional analogue, known as metasurfaces, has attracted progressively increasing attention in recent years due to the ease of fabrication and smaller insertion losses, while enabling an unprecedented control over spatial distributions of transmitted and reflected optical fields. Metasurfaces represent optically thin planar arrays of resonant subwavelength elements that can be arranged in a strictly or quasi periodic fashion, or even in an aperiodic manner, depending on targeted optical wavefronts to be molded with their help. This paper reviews a broad subclass of metasurfaces, viz. gradient metasurfaces, which are devised to exhibit spatially varying optical responses resulting in spatially varying amplitudes, phases and polarizations of scattered fields. Starting with introducing the concept of gradient metasurfaces, we present classification of different metasurfaces from the viewpoint of their responses, differentiating electrical-dipole, geometric, reflective and Huygens' metasurfaces. The fundamental building blocks essential for the realization of metasurfaces are then discussed in order to elucidate the underlying physics of various physical realizations of both plasmonic and purely dielectric metasurfaces. We then overview the main applications of gradient metasurfaces, including waveplates, flat lenses, spiral phase plates, broadband absorbers, color printing, holograms, polarimeters and surface wave couplers. The review is terminated with a short section on recently developed nonlinear metasurfaces, followed by the outlook presenting our view on possible future developments and perspectives for future applications.Comment: Accepted for publication in Reports on Progress in Physic

High-sensitivity analysis of polarization by surface reflection

© 2018, The Author(s). The exploitation of polarization information in the field of computer vision has become progressively more popular during the last few decades. This is primarily due to (1) the fact that polarization is a source of mostly untapped information for machine vision; (2) the relative computational ease by which geometrical information about a scene (e.g.surface normals) may be extracted from polarization data; and (3) the recent introduction of camera hardware able to capture polarization data in real time. The motivation for this paper is that a detailed quantitative study into the precision of polarization measurements with respect to expectation has yet to be performed. The paper therefore presents a detailed analysis and optimization of the key aspects of data capture necessary to acquire the most precise (as opposed to fast) results for the benefit of future research into the field of “polarization vision.” The paper mainly focuses on a rotating polarizer method as this is shown to be the most accurate for high-sensitivity measurements. Commercial polarization cameras by contrast generally sacrifice precision for the benefit of much shorter capture times. That said, the paper reviews the state of the art in polarization camera technology and quantitatively evaluates the performance of one such camera: the Fraunhofer “POLKA.”

New 3D scanning techniques for complex scenes

This thesis presents new 3D scanning methods for complex scenes, such as surfaces with fine-scale geometric details, translucent objects, low-albedo objects, glossy objects, scenes with interreflection, and discontinuous scenes. Starting from the observation that specular reflection is a reliable visual cue for surface mesostructure perception, we propose a progressive acquisition system that captures a dense specularity field as the only information for mesostructure reconstruction. Our method can efficiently recover surfaces with fine-scale geometric details from complex real-world objects. Translucent objects pose a difficult problem for traditional optical-based 3D scanning techniques. We analyze and compare two descattering methods, phaseshifting and polarization, and further present several phase-shifting and polarization based methods for high quality 3D scanning of translucent objects. We introduce the concept of modulation based separation, where a high frequency signal is multiplied on top of another signal. The modulated signal inherits the separation properties of the high frequency signal and allows us to remove artifacts due to global illumination. Thismethod can be used for efficient 3D scanning of scenes with significant subsurface scattering and interreflections.Diese Dissertation präsentiert neuartige Verfahren für die 3D-Digitalisierung komplexer Szenen, wie z.B. Oberflächen mit sehr feinen Strukturen, durchscheinende Objekte, Gegenstände mit geringem Albedo, glänzende Objekte, Szenen mit Lichtinterreflektionen und unzusammenhängende Szenen. Ausgehend von der Beobachtung, daß die spekulare Reflektion ein zuverlässiger, visueller Hinweis für die Mesostruktur einer Oberfläche ist, stellen wir ein progressives Meßsystem vor, um Spekularitätsfelder zu messen. Aus diesen Feldern kann anschließend die Mesostruktur rekonstruiert werden. Mit unserer Methode können Oberflächen mit sehr feinen Strukturen von komplexen, realen Objekten effizient aufgenommen werden. Durchscheinende Objekte stellen ein großes Problem für traditionelle, optischbasierte 3D-Rekonstruktionsmethoden dar. Wir analysieren und vergleichen zwei verschiedene Methoden zum Eliminieren von Lichtstreuung (Descattering): Phasenverschiebung und Polarisation. Weiterhin präsentieren wir mehrere hochqualitative 3D-Rekonstruktionsmethoden für durchscheinende Objekte, die auf Phasenverschiebung und Polarisation basieren. Außerdem führen wir das Konzept der modulationsbasierten Signaltrennung ein. Hierzu wird ein hochfrequentes Signal zu einem anderes Signal multipliziert. Das so modulierte Signal erhält damit die separierenden Eigenschaften des hochfrequenten Signals. Dies erlaubt unsMeßartefakte aufgrund von globalen Beleuchtungseffekten zu vermeiden. Dieses Verfahren kann zum effizienten 3DScannen von Szenen mit durchscheinden Objekten und Interreflektionen benutzt werden

Polarimetric remote sensing system analysis: Digital Imaging and Remote Sensing Image Generation (DIRSIG) model validation and impact of polarization phenomenology on material discriminability

In addition to spectral information acquired by traditional multi/hyperspectral systems, passive electro optical and infrared (EO/IR) polarimetric sensors also measure the polarization response of different materials in the scene. Such an imaging modality can be useful in improving surface characterization; however, the characteristics of polarimetric systems have not been completely explored by the remote sensing community. Therefore, the main objective of this research was to advance our knowledge in polarimetric remote sensing by investigating the impact of polarization phenomenology on material discriminability. The first part of this research focuses on system validation, where the major goal was to assess the fidelity of the polarimetric images simulated using the Digital Imaging and Remote Sensing Image Generation (DIRSIG) model. A theoretical framework, based on polarization vision models used for animal vision studies and industrial defect detection applications, was developed within which the major components of the polarimetric image chain were validated. In the second part of this research, a polarization physics based approach for improved material discriminability was proposed. This approach utilizes the angular variation in the polarization response to infer the physical characteristics of the observed surface by imaging the scene in three different view directions. The usefulness of the proposed approach in improving detection performance in the absence of apriori knowledge about the target geometry was demonstrated. Sensitivity analysis of the proposed system for different scene related parameters was performed to identify the imaging conditions under which the material discriminability is maximized. Furthermore, the detection performance of the proposed polarimetric system was compared to that of the hyperspectral system to identify scenarios where polarization information can be very useful in improving the target contrast

Microparticle manipulation and imaging through a self-calibrated liquid crystal on silicon display

We present in this paper a revision of three different methods we conceived in the framework of liquid crystal on silicon (LCoS) display optimization and application. We preliminarily demonstrate an LCoS self-calibration technique, from which we can perform a complete LCoS characterization. In particular, two important characteristics of LCoS displays are retrieved by using self-addressed digital holograms. On the one hand, we determine its phase-voltage curve by using the interference pattern generated by a digital two-sectorial split-lens configuration. On the other hand, the LCoS surface profile is also determined by using a self-addressed dynamic micro-lens array pattern. Second, the implementation of microparticle manipulation through optical traps created by an LCoS display is demonstrated. Finally, an LCoS display based inline (IL) holographic imaging system is described. By using the LCoS display to implement a double-sideband filter configuration, this inline architecture demonstrates the advantage of obtaining dynamic holographic imaging of microparticles independently of their spatial positions by avoiding the non-desired conjugate images

On-site surface reflectometry

The rapid development of Augmented Reality (AR) and Virtual Reality (VR) applications over the past years has created the need to quickly and accurately scan the real world to populate immersive, realistic virtual environments for the end user to enjoy. While geometry processing has already gone a long way towards that goal, with self-contained solutions commercially available for on-site acquisition of large scale 3D models, capturing the appearance of the materials that compose those models remains an open problem in general uncontrolled environments. The appearance of a material is indeed a complex function of its geometry, intrinsic physical properties and furthermore depends on the illumination conditions in which it is observed, thus traditionally limiting the scope of reflectometry to highly controlled lighting conditions in a laboratory setup. With the rapid development of digital photography, especially on mobile devices, a new trend in the appearance modelling community has emerged, that investigates novel acquisition methods and algorithms to relax the hard constraints imposed by laboratory-like setups, for easy use by digital artists. While arguably not as accurate, we demonstrate the ability of such self-contained methods to enable quick and easy solutions for on-site reflectometry, able to produce compelling, photo-realistic imagery. In particular, this dissertation investigates novel methods for on-site acquisition of surface reflectance based on off-the-shelf, commodity hardware. We successfully demonstrate how a mobile device can be utilised to capture high quality reflectance maps of spatially-varying planar surfaces in general indoor lighting conditions. We further present a novel methodology for the acquisition of highly detailed reflectance maps of permanent on-site, outdoor surfaces by exploiting polarisation from reflection under natural illumination. We demonstrate the versatility of the presented approaches by scanning various surfaces from the real world and show good qualitative and quantitative agreement with existing methods for appearance acquisition employing controlled or semi-controlled illumination setups.Open Acces