Practical acquisition and rendering of diffraction effects in surface reflectance

We propose two novel contributions for measurement based rendering of diffraction effects in surface reflectance of planar homogeneous diffractive materials. As a general solution for commonly manufactured materials, we propose a practical data-driven rendering technique and a measurement approach to efficiently render complex diffraction effects in real-time. Our measurement step simply involves photographing a planar diffractive sam- ple illuminated with an LED flash. Here, we directly record the resultant diffraction pattern on the sample surface due to a narrow band point source illumination. Furthermore, we propose an efficient rendering method that exploits the measurement in conjunction with the Huygens-Fresnel principle to fit relevant diffraction parameters based on a first order approximation. Our proposed data-driven rendering method requires the precomputation of a single diffraction look up table for accurate spectral rendering of com- plex diffraction effects. Secondly, for sharp specular samples, we propose a novel method for practical measurement of the underlying diffraction grating using out-of-focus “bokeh” photography of the specular highlight. We demonstrate how the measured bokeh can be employed as a height field to drive a diffraction shader based on a first order approximation for efficient real-time rendering. Finally, we also drive analytic solutions for a few special cases of diffraction from our measurements and demonstrate realistic rendering results under complex light sources and environments

Dynamic Display of BRDFs

This paper deals with the challenge of physically displaying reflectance, i.e., the appearance of a surface and its variation with the observer position and the illuminating environment. This is commonly described by the bidirectional reflectance distribution function (BRDF). We provide a catalogue of criteria for the display of BRDFs, and sketch a few orthogonal approaches to solving the problem in an optically passive way. Our specific implementation is based on a liquid surface, on which we excite waves in order to achieve a varying degree of anisotropic roughness. The resulting probability density function of the surface normal is shown to follow a Gaussian distribution similar to most established BRDF models

Reconsidering light transport : acquisition and display of real-world reflectance and geometry

In this thesis, we cover three scenarios that violate common simplifying assumptions about the nature of light transport. We begin with the first ingredient to any çD rendering: a geometry model. Most çD scanners require the object-of-interest to show diffuse refectance. The further a material deviates from the Lambertian model, the more likely these setups are to produce corrupted results. By placing a traditional laser scanning setup in a participating (in particular, fuorescent) medium, we have built a light sheet scanner that delivers robust results for a wide range of materials, including glass. Further investigating the phenomenon of fluorescence, we notice that, despite its ubiquity, it has received moderate attention in computer graphics. In particular, to date no datadriven reflectance models of fluorescent materials have been available. To describe the wavelength-shifling reflectance of fluorescent materials, we define the bispectral bidirectional reflectance and reradiation distribution function (BRRDF), for which we introduce an image-based measurement setup as well as an efficient acquisition scheme. Finally, we envision a computer display that showsmaterials instead of colours, and present a prototypical device that can exhibit anisotropic reflectance distributions similar to common models in computer graphics.In der Computergraphik und Computervision ist es unerlässlich, vereinfachende Annahmen über die Ausbreitung von Licht zumachen. In dieser Dissertation stellen wir drei Fälle vor, in denen diese nicht zutreffen. So wird die dreidimensionale Geometrie von Gegenständen oft mit Hilfe von Laserscannern vermessen und dabei davon ausgegangen, dass ihre Oberfläche diffus reflektiert. Dies ist bei den meisten Materialien jedoch nicht gegeben, so dass die Ergebnisse oft fehlerhaft sind. Indem wir das Objekt in einem fluoreszierenden Medium einbetten, kann ein klassischer CD-Scanner-Aufbau so modifiziert werden, dass er verlässliche Geometriedaten für Objekte aus verschiedensten Materialien liefert, einschließlich Glas. Auch die akkurate Nachbildung des Aussehens von Materialien ist wichtig für die photorealistische Bildsynthese. Wieder interessieren wir uns für Fluoreszenz, diesmal allerdings für ihr charakteristisches Erscheinungsbild, das in der Computergraphik bislang kaum Beachtung gefunden hat. Wir stellen einen bildbasierten Aufbau vor, mit dem die winkel- und wellenlängenabhängige Reflektanz fluoreszierender Oberflächen ausgemessen werden kann, und eine Strategie, um solche Messungen effizient abzuwickeln. Schließlich befassen wir uns mit der Idee, nicht nur Farben dynamisch anzuzeigen, sondern auch Materialien und ihr je nach Lichteinfall und Blickwinkel unterschiedliches Aussehen. Einer generellen Beschreibung des Problems folgt die konkrete Umsetzung in Formzweier Prototypen, die verschiedene Reflektanzverteilungen auf einer Oberfläche darstellen können

Roadmap on structured light

Structured light refers to the generation and application of custom light fields. As the tools and technology to create and detect structured light have evolved, steadily the applications have begun to emerge. This roadmap touches on the key fields within structured light from the perspective of experts in those areas, providing insight into the current state and the challenges their respective fields face. Collectively the roadmap outlines the venerable nature of structured light research and the exciting prospects for the future that are yet to be realized.Peer ReviewedPostprint (published version

Metasurface optical characterization using quadriwave lateral shearing interferometry

An optical metasurface consists of a dense and usually non-uniform layer of scattering nanostructures behaving as a continuous and extremely thin optical component, with predefined phase and intensity transmission/reflection profiles. To date, various sorts of metasurfaces (metallic, dielectric, Huygens-like, Pancharatman-Berry, etc.) have been introduced to design ultrathin lenses, beam deflectors, holograms, or polarizing interfaces. Their actual efficiencies depend on the ability to predict their optical properties and to fabricate non-uniform assemblies of billions of nanoscale structures on macroscopic surfaces. To further help improve the design of metasurfaces, precise and versatile post-characterization techniques need to be developed. Today, most of the techniques used to characterize metasurfaces rely on light intensity measurements. Here, we demonstrate how quadriwave lateral shearing interferometry (QLSI), a quantitative phase microscopy technique, can easily achieve full optical characterization of metasurfaces of any kind, as it can probe the local phase imparted by a metasurface with high sensitivity and spatial resolution. As a means to illustrate the versatility of this technique, we present measurements on two types of metasurfaces, namely Pancharatnam-Berry and effective-refractive-index metasurfaces, and present results on uniform metasurfaces, metalenses and deflectors

Efficient image-based rendering

Recent advancements in real-time ray tracing and deep learning have significantly enhanced the realism of computer-generated images. However, conventional 3D computer graphics (CG) can still be time-consuming and resource-intensive, particularly when creating photo-realistic simulations of complex or animated scenes. Image-based rendering (IBR) has emerged as an alternative approach that utilizes pre-captured images from the real world to generate realistic images in real-time, eliminating the need for extensive modeling. Although IBR has its advantages, it faces challenges in providing the same level of control over scene attributes as traditional CG pipelines and accurately reproducing complex scenes and objects with different materials, such as transparent objects. This thesis endeavors to address these issues by harnessing the power of deep learning and incorporating the fundamental principles of graphics and physical-based rendering. It offers an efficient solution that enables interactive manipulation of real-world dynamic scenes captured from sparse views, lighting positions, and times, as well as a physically-based approach that facilitates accurate reproduction of the view dependency effect resulting from the interaction between transparent objects and their surrounding environment. Additionally, this thesis develops a visibility metric that can identify artifacts in the reconstructed IBR images without observing the reference image, thereby contributing to the design of an effective IBR acquisition pipeline. Lastly, a perception-driven rendering technique is developed to provide high-fidelity visual content in virtual reality displays while retaining computational efficiency.Jüngste Fortschritte im Bereich Echtzeit-Raytracing und Deep Learning haben den Realismus computergenerierter Bilder erheblich verbessert. Konventionelle 3DComputergrafik (CG) kann jedoch nach wie vor zeit- und ressourcenintensiv sein, insbesondere bei der Erstellung fotorealistischer Simulationen von komplexen oder animierten Szenen. Das bildbasierte Rendering (IBR) hat sich als alternativer Ansatz herauskristallisiert, bei dem vorab aufgenommene Bilder aus der realen Welt verwendet werden, um realistische Bilder in Echtzeit zu erzeugen, so dass keine umfangreiche Modellierung erforderlich ist. Obwohl IBR seine Vorteile hat, ist es eine Herausforderung, das gleiche Maß an Kontrolle über Szenenattribute zu bieten wie traditionelle CG-Pipelines und komplexe Szenen und Objekte mit unterschiedlichen Materialien, wie z.B. transparente Objekte, akkurat wiederzugeben. In dieser Arbeit wird versucht, diese Probleme zu lösen, indem die Möglichkeiten des Deep Learning genutzt und die grundlegenden Prinzipien der Grafik und des physikalisch basierten Renderings einbezogen werden. Sie bietet eine effiziente Lösung, die eine interaktive Manipulation von dynamischen Szenen aus der realen Welt ermöglicht, die aus spärlichen Ansichten, Beleuchtungspositionen und Zeiten erfasst wurden, sowie einen physikalisch basierten Ansatz, der eine genaue Reproduktion des Effekts der Sichtabhängigkeit ermöglicht, der sich aus der Interaktion zwischen transparenten Objekten und ihrer Umgebung ergibt. Darüber hinaus wird in dieser Arbeit eine Sichtbarkeitsmetrik entwickelt, mit der Artefakte in den rekonstruierten IBR-Bildern identifiziert werden können, ohne das Referenzbild zu betrachten, und die somit zur Entwicklung einer effektiven IBR-Erfassungspipeline beiträgt. Schließlich wird ein wahrnehmungsgesteuertes Rendering-Verfahren entwickelt, um visuelle Inhalte in Virtual-Reality-Displays mit hoherWiedergabetreue zu liefern und gleichzeitig die Rechenleistung zu erhalten

Multiple viewpoint rendering for three-dimensional displays

Thesis (Ph. D.)--Massachusetts Institute of Technology, Program in Media Arts & Sciences, 1997.Includes bibliographical references (leaves 159-164).Michael W. Halle.Ph.D

HOMES: Holographic Method for Exoplanet Spectroscopy

The search for habitable exoplanets in the "neighborhood," that is within 10 parsecs (pc),invites new observational techniques, because telescopes available today have significant limitations. For example, gravitational micro-lensing is a deep field method that has little or no utility within 10 pc. Photometry of transits reduces the availability of exoplanets to a probability of approximately a 1%, and fewer than 1 out of 100 earth twins would ever be seen. For the 1% of exoplanets seen in transit, spectra of albedos cannot be taken.Even more limiting, Doppler shift studies are indirect and cannot characterize exoplanets other than by their orbits and masses through stellar radial velocity (RV). Moreover, RV by Doppler shift cannot detect planets in orbits on a plane perpendicular to our line of sight. Astrometry can be used to measure stellar orbital wobble for those exoplanetary systems in the plane perpendicular to our line of sight, as was contemplated for the cancelled NASA SIM mission 1 and rests now with ESA's GAIA2. In either case, the exoplanets detected by stellar RV or astrometry would not be directly observed, so the assumption that the three phases of water could exist does not answer a key question of habitability. Is there water on the exoplanet? Moreover, the discovery of water on an exoplanet in the habitable zone, exciting though it would be, is insufficient to determine just how habitable the exoplanet might possibly be. Like Goldilocks' porridge - there are many other ingredients beyond water alone in a life-sustaining soup.We propose a new species of optical telescope that has as one of its capabilities the high resolution spectrographic characterization of exoplanets on stars within 10 pc of the observatory. To achieve this performance specification, we propose to break with convention by taking advantage of an optical technology unknown until relatively recent times: the hologram. In this Report we show how holograms used in primary and secondary can extract spectrograms from exoplanets at distances reaching 33 light years from our solar system. We describe a notional space telescope using these novel optics that could make its observations in a space deployment scenario

Fabricating BRDFs at high spatial resolution using wave optics

Recent attempts to fabricate surfaces with custom reflectance functions boast impressive angular resolution, yet their spatial resolution is limited. In this paper we present a method to construct spatially varying reflectance at a high resolution of up to 220dpi, orders of magnitude greater than previous attempts, albeit with a lower angular resolution. The resolution of previous approaches is limited by the machining, but more fundamentally, by the geometric optics model on which they are built. Beyond a certain scale geometric optics models break down and wave effects must be taken into account. We present an analysis of incoherent reflectance based on wave optics and gain important insights into reflectance design. We further suggest and demonstrate a practical method, which takes into account the limitations of existing micro-fabrication techniques such as photolithography to design and fabricate a range of reflection effects, based on wave interference.United States-Israel Binational Science FoundationIntel Corporation (Intel Collaborative Research Institute for Computational Intelligence)National Science Foundation (U.S.) (CGV 1116303

Volumetric rendering for holographic display of medical data

Thesis (M.S.)--Massachusetts Institute of Technology, Dept. of Architecture, 1988.Includes bibliographical references.Work funded by a joint IBM/MIT agreement.by Wendy J. Plesniak.M.S