Learning Wavefront Coding for Extended Depth of Field Imaging

Depth of field is an important factor of imaging systems that highly affects the quality of the acquired spatial information. Extended depth of field (EDoF) imaging is a challenging ill-posed problem and has been extensively addressed in the literature. We propose a computational imaging approach for EDoF, where we employ wavefront coding via a diffractive optical element (DOE) and we achieve deblurring through a convolutional neural network. Thanks to the end-to-end differentiable modeling of optical image formation and computational post-processing, we jointly optimize the optical design, i.e., DOE, and the deblurring through standard gradient descent methods. Based on the properties of the underlying refractive lens and the desired EDoF range, we provide an analytical expression for the search space of the DOE, which is instrumental in the convergence of the end-to-end network. We achieve superior EDoF imaging performance compared to the state of the art, where we demonstrate results with minimal artifacts in various scenarios, including deep 3D scenes and broadband imaging

Snapshot Multispectral Imaging Using a Diffractive Optical Network

Multispectral imaging has been used for numerous applications in e.g., environmental monitoring, aerospace, defense, and biomedicine. Here, we present a diffractive optical network-based multispectral imaging system trained using deep learning to create a virtual spectral filter array at the output image field-of-view. This diffractive multispectral imager performs spatially-coherent imaging over a large spectrum, and at the same time, routes a pre-determined set of spectral channels onto an array of pixels at the output plane, converting a monochrome focal plane array or image sensor into a multispectral imaging device without any spectral filters or image recovery algorithms. Furthermore, the spectral responsivity of this diffractive multispectral imager is not sensitive to input polarization states. Through numerical simulations, we present different diffractive network designs that achieve snapshot multispectral imaging with 4, 9 and 16 unique spectral bands within the visible spectrum, based on passive spatially-structured diffractive surfaces, with a compact design that axially spans ~72 times the mean wavelength of the spectral band of interest. Moreover, we experimentally demonstrate a diffractive multispectral imager based on a 3D-printed diffractive network that creates at its output image plane a spatially-repeating virtual spectral filter array with 2x2=4 unique bands at terahertz spectrum. Due to their compact form factor and computation-free, power-efficient and polarization-insensitive forward operation, diffractive multispectral imagers can be transformative for various imaging and sensing applications and be used at different parts of the electromagnetic spectrum where high-density and wide-area multispectral pixel arrays are not widely available.Comment: 24 Pages, 9 Figure

Image Quality Is Not All You Want: Task-Driven Lens Design for Image Classification

In computer vision, it has long been taken for granted that high-quality images obtained through well-designed camera lenses would lead to superior results. However, we find that this common perception is not a "one-size-fits-all" solution for diverse computer vision tasks. We demonstrate that task-driven and deep-learned simple optics can actually deliver better visual task performance. The Task-Driven lens design approach, which relies solely on a well-trained network model for supervision, is proven to be capable of designing lenses from scratch. Experimental results demonstrate the designed image classification lens (``TaskLens'') exhibits higher accuracy compared to conventional imaging-driven lenses, even with fewer lens elements. Furthermore, we show that our TaskLens is compatible with various network models while maintaining enhanced classification accuracy. We propose that TaskLens holds significant potential, particularly when physical dimensions and cost are severely constrained.Comment: Use an image classification network to supervise the lens design from scratch. The final designs can achieve higher accuracy with fewer optical element

Exploiting Structural Complexity for Robust and Rapid Hyperspectral Imaging

This paper presents several strategies for spectral de-noising of hyperspectral images and hypercube reconstruction from a limited number of tomographic measurements. In particular we show that the non-noisy spectral data, when stacked across the spectral dimension, exhibits low-rank. On the other hand, under the same representation, the spectral noise exhibits a banded structure. Motivated by this we show that the de-noised spectral data and the unknown spectral noise and the respective bands can be simultaneously estimated through the use of a low-rank and simultaneous sparse minimization operation without prior knowledge of the noisy bands. This result is novel for for hyperspectral imaging applications. In addition, we show that imaging for the Computed Tomography Imaging Systems (CTIS) can be improved under limited angle tomography by using low-rank penalization. For both of these cases we exploit the recent results in the theory of low-rank matrix completion using nuclear norm minimization

Doctor of Philosophy

dissertationOptics is an old topic in physical science and engineering. Historically, bulky materials and components were dominantly used to manipulate light. A new hope arrived when Maxwell unveiled the essence of electromagnetic waves in a micro perspective. On the other side, our world recently embraced a revolutionary technology, metasurface, which modifies the properties of matter-interfaces in subwavelength scale. To complete this story, diffractive optic fills right in the gap. It enables ultrathin flat devices without invoking the concept of nanostructured metasurfaces when only scalar diffraction comes into play. This dissertation contributes to developing a new type of digital diffractive optic, called a polychromat. It consists of uniform pixels and multilevel profile in micrometer scale. Essentially, it modulates the phase of a wavefront to generate certain spatial and spectral responses. Firstly, a complete numerical model based on scalar diffraction theory was developed. In order to functionalize the optic, a nonlinear algorithm was then successfully implemented to optimize its topography. The optic can be patterned in transparent dielectric thin film by single-step grayscale lithography and it is replicable for mass production. The microstructures are 3?m wide and no more than 3?m thick, thus do not require slow and expensive nanopatterning techniques, as opposed to metasurfaces. Polychromat is also less demanding in terms of fabrication and scalability. The next theme is focused on demonstrating unprecedented performances of the diffractive optic when applied to address critical issues in modern society. Photovoltaic efficiency can be significantly enhanced using this optic to split and concentrate the solar spectrum. Focusing through a lens is no news, but we transformed our optic into a flat lens that corrects broadband chromatic aberrations. It can also serve as a phase mask for microlithography on oblique and multiplane surfaces. By introducing the powerful tool of computation, we devised two imaging prototypes, replacing the conventional Bayer filter with the diffractive optic. One system increases light sensitivity by 3 times compared to commercial color sensors. The other one renders the monochrome sensor a new function of high-resolution multispectral video-imaging

Evaluation and Quantification of Diffractive Plenoptic Camera Algorithm Performance

A diffractive plenoptic camera is a novel approach to the traditional plenoptic camera which replaces the main optic with a Fresnel zone plate making the camera sensitive to wavelength instead of range. However, algorithms are necessary to reconstruct the image produced by plenoptic cameras. While many algorithms exist for traditional plenoptic cameras, their ability to create spectral images in a diffractive plenoptic camera is unknown. This paper evaluates digital refocusing, super resolution, and 3D deconvolution through a Richardson-Lucy algorithm as well as a new Gaussian smoothing algorithm. All of the algorithms worked well near the Fresnel zone plate design wavelength, but Gaussian smoothing provided better looking images at a cost of high computation time. For wavelengths off the design wavelength, 3D deconvolution produced the best images but also required more computation time. 3D deconvolution also had the best spectral resolution, which increased away from the design wavelength. These results, along with consideration of mission constraints and spectral content in the scene, can guide algorithm selection for future sensor designs

Astigmatism and Pseudoaccommodation in Pseudophakic Eyes

noAdvanced IOLs with circumferential zones of different power provide pseudoaccommodation. We investigated the potential for power variation with meridian, namely astigmatism, to provide pseudo-accommodation. With appropriate power and axis orientations, acceptable pseudo-accommodation can be achieved

Depolarized Holography with Polarization-multiplexing Metasurface

The evolution of computer-generated holography (CGH) algorithms has prompted significant improvements in the performances of holographic displays. Nonetheless, they start to encounter a limited degree of freedom in CGH optimization and physical constraints stemming from the coherent nature of holograms. To surpass the physical limitations, we consider polarization as a new degree of freedom by utilizing a novel optical platform called metasurface. Polarization-multiplexing metasurfaces enable incoherent-like behavior in holographic displays due to the mutual incoherence of orthogonal polarization states. We leverage this unique characteristic of a metasurface by integrating it into a holographic display and exploiting polarization diversity to bring an additional degree of freedom for CGH algorithms. To minimize the speckle noise while maximizing the image quality, we devise a fully differentiable optimization pipeline by taking into account the metasurface proxy model, thereby jointly optimizing spatial light modulator phase patterns and geometric parameters of metasurface nanostructures. We evaluate the metasurface-enabled depolarized holography through simulations and experiments, demonstrating its ability to reduce speckle noise and enhance image quality.Comment: 15 pages, 13 figures, to be published in SIGGRAPH Asia 202