Compressive Photon-Sieve Spectral Imaging

We develop a new compressive spectral imaging modality that utilizes a coded aperture and a photon-sieve for dispersion. The 3D spectral data cube is successfully reconstructed with as little as two shots using sparse recover

Compresión adicional de los datos del plano focal en arquitecturas de sensado comprimido en imágenes espectrales

Context: It is possible to capture High-resolution 3D hyper-spectral images in a single 2D image through techniques based on compressed sensing. A variety of architectures have proposed Compressive Spectral Imaging (CSI) technique during the last years. An optical camera designed to capture spatio-spectral information of the scene prints projections towards a Focal Plane Array (FPA) giving the capability of storing or transmitting them. Afterwards, the original image can be reconstructed via an -norm-based optimization algorithm. The size in bytes of the FPA measurement is less than the original image; for that reason, this FPA is considered a 2D compressed version of the original 3D image.Objective: To perform a further compression of the FPA measurement for four CSI architectures, in order to increase transfer rates or to decrease storing sizes.Method: In this work, the design of the further compression using arithmetic coding is presented for four CSI architectures, and an inverse transformation is proposed. This transformation is applied to the FPA based on the structure of the optical filters and the coded apertures of the cameras used in the CSI, allowing an increasing in the compression rate. Results: Results show that the compression rate rises between 1 and 2 points in three of the architectures.Conclusions: Despite data loss in the process of transformation-quantification-compression-decompression of the FPA, the quality of the reconstructed data cube (expressed in terms of the PSNR between the reconstructed image and the original one) remains close to the original version with no further compression.Contexto: Las imágenes híper-espectrales 3D de alta resolución pueden ser capturadas en una imagen 2D mediante técnicas basadas en “sensado compresivo” (compressive sensing, en inglés). Entre estas técnicas, hay una denominada Compressive Spectral Imaging (CSI), de la cual se han propuesto diversas arquitecturas en los últimos ocho años. Una cámara óptica especialmente diseñada captura la información espacio-espectral de la escena e imprime proyecciones en un plano focal 2D (Focal Plane Array, FPA). Estas muestras se pueden transmitir o almacenar; luego la imagen original puede ser reconstruida usando comúnmente un algoritmo de optimización de la norma . El tamaño en bytes del FPA es menor que la imagen original, y éste por lo tanto puede ser considerado una versión comprimida en 2D de la imagen original en 3D.Objetivo: Realizar una compresión adicional del FPA para cuatro arquitecturas CSI, para incrementar las velocidades de transmisión o disminuir tamaños de almacenamiento.Método: En este trabajo se presentan los resultados de esta compresión adicional usando la codificación aritmética y se propone una transformación inversa que se aplica al FPA con base en la estructura de los filtros ópticos y los códigos de apertura de las cámaras usadas en CSI, lo cual permite aumentar su factor de compresión.Resultados: Los resultados muestran que el factor de compresión aumenta entre 1 y 2 puntos en tres de las arquitecturas.Conclusiones: A pesar de que hay pérdidas de datos en el proceso de transformación-cuantificación-compresión-descompresión del FPA, para cada arquitectura CSI usada, la calidad del cubo de datos posteriormente reconstruido expresada en el PSNR entre la imagen original y la reconstruida, no difiere significativamente de la versión original

Snapshot Multispectral Imaging Using a Diffractive Optical Network

Multispectral imaging has been used for numerous applications in e.g., environmental monitoring, aerospace, defense, and biomedicine. Here, we present a diffractive optical network-based multispectral imaging system trained using deep learning to create a virtual spectral filter array at the output image field-of-view. This diffractive multispectral imager performs spatially-coherent imaging over a large spectrum, and at the same time, routes a pre-determined set of spectral channels onto an array of pixels at the output plane, converting a monochrome focal plane array or image sensor into a multispectral imaging device without any spectral filters or image recovery algorithms. Furthermore, the spectral responsivity of this diffractive multispectral imager is not sensitive to input polarization states. Through numerical simulations, we present different diffractive network designs that achieve snapshot multispectral imaging with 4, 9 and 16 unique spectral bands within the visible spectrum, based on passive spatially-structured diffractive surfaces, with a compact design that axially spans ~72 times the mean wavelength of the spectral band of interest. Moreover, we experimentally demonstrate a diffractive multispectral imager based on a 3D-printed diffractive network that creates at its output image plane a spatially-repeating virtual spectral filter array with 2x2=4 unique bands at terahertz spectrum. Due to their compact form factor and computation-free, power-efficient and polarization-insensitive forward operation, diffractive multispectral imagers can be transformative for various imaging and sensing applications and be used at different parts of the electromagnetic spectrum where high-density and wide-area multispectral pixel arrays are not widely available.Comment: 24 Pages, 9 Figure

Multiresolution-based reconstruction for compressive spectral video sensing using a spectral multiplexing sensor

Spectral multiplexing sensors based on compressive sensing attempt to break the Nyquist barrier to acquire high spectral resolution scenes. Particularly, the colored coded aperture-based compressive spectral imager extended to video, or video C-CASSI, is a spectral multiplexing sensor that allows capturing spectral dynamic scenes by projecting each spectral frame onto a bidimensional detector using a 3D coded aperture. Afterwards, the compressed signal reconstruction is performed iteratively by finding a sparse solution to an undetermined linear system of equations. Even though the acquired signal can be recovered from much fewer observations by an  −  -norm recovery algorithm than using conventional sensors, the reconstruction exhibits diverse challenges originated by the temporal variable or motion. The motion during the reconstruction produces artifacts that damages the entire data. In this work, a multiresolution-based reconstruction method for compressive spectral video sensing is proposed. In this way, it obtains the temporal information from the measurements at a low computational cost. Thereby, the optimization problem to recover the signal is extended by adding temporal information in order to correct the errors originated by the scene motion. Computational experiments performed over four different spectral videos show an improvement up to 4dB in terms of peak-signal to noise ratio (PSNR) in the reconstruction quality using the multiresolution approach applied to the spectral video reconstruction with respect to the traditional inverse problem.  Los sensores de multiplexación espectral basados en muestreo compresivo intentan romper la barrera de Nyquist para adquirir escenas de alta resolución espectral. Particularmente, el sistema de imágenes espectrales de única captura basado en aperturas codificadas de color extendido a vídeo, o video – CCASSI, es un sensor de multiplexación espectral que permite la adquisición de imágenes espectrales dinámicas proyectando cada fotograma espectral sobre un detector bidimensional usando un apertura de codificación 3D. Posteriormente, la reconstrucción de la señal comprimida se realiza iterativamente encontrando una solución escasa a un sistema lineal de ecuaciones indeterminado. Si bien la señal adquirida puede ser recuperada desde un algoritmo basado en la norma l_2 − l_1, con  muchas menos observaciones en comparación a los sistemas convencionales, dicha reconstrucción presenta diversos desafíos originados por la variable temporal o el movimiento. El movimiento durante la reconstrucción produce artefactos que dañan la totalidad de los datos. En este trabajo, se propone un método de reconstrucción basado en múltiples resoluciones para la adquisición compresiva de video espectral. De este modo, el problema de optimización para recuperar la señal se amplía añadiendo la información temporal como restricción, con el objetivo de corregir los errores originados por el movimiento de la escena. Los experimentos computacionales realizados en cuatro videos espectrales diferentes muestran una mejora de hasta 4dB en términos de relación pico-señal a ruido (PSNR) en la calidad de reconstrucción usando el enfoque multirresolución aplicado a la reconstrucción de video espectral con respecto al problema de optimización que no considera el movimiento

Multiresolution-based reconstruction for compressive spectral video sensing using a spectral multiplexing sensor

Spectral multiplexing sensors based on compressive sensing attempt to break the Nyquist barrier to acquire high spectral resolution scenes. Particularly, the colored coded aperture-based compressive spectral imager extended to video, or video C-CASSI, is a spectral multiplexing sensor that allows capturing spectral dynamic scenes by projecting each spectral frame onto a bidimensional detector using a 3D coded aperture. Afterwards, the compressed signal reconstruction is performed iteratively by finding a sparse solution to an undetermined linear system of equations. Even though the acquired signal can be recovered from much fewer observations by an  −  -norm recovery algorithm than using conventional sensors, the reconstruction exhibits diverse challenges originated by the temporal variable or motion. The motion during the reconstruction produces artifacts that damages the entire data. In this work, a multiresolution-based reconstruction method for compressive spectral video sensing is proposed. In this way, it obtains the temporal information from the measurements at a low computational cost. Thereby, the optimization problem to recover the signal is extended by adding temporal information in order to correct the errors originated by the scene motion. Computational experiments performed over four different spectral videos show an improvement up to 4dB in terms of peak-signal to noise ratio (PSNR) in the reconstruction quality using the multiresolution approach applied to the spectral video reconstruction with respect to the traditional inverse problem.  Los sensores de multiplexación espectral basados en muestreo compresivo intentan romper la barrera de Nyquist para adquirir escenas de alta resolución espectral. Particularmente, el sistema de imágenes espectrales de única captura basado en aperturas codificadas de color extendido a vídeo, o video – CCASSI, es un sensor de multiplexación espectral que permite la adquisición de imágenes espectrales dinámicas proyectando cada fotograma espectral sobre un detector bidimensional usando un apertura de codificación 3D. Posteriormente, la reconstrucción de la señal comprimida se realiza iterativamente encontrando una solución escasa a un sistema lineal de ecuaciones indeterminado. Si bien la señal adquirida puede ser recuperada desde un algoritmo basado en la norma l_2 − l_1, con  muchas menos observaciones en comparación a los sistemas convencionales, dicha reconstrucción presenta diversos desafíos originados por la variable temporal o el movimiento. El movimiento durante la reconstrucción produce artefactos que dañan la totalidad de los datos. En este trabajo, se propone un método de reconstrucción basado en múltiples resoluciones para la adquisición compresiva de video espectral. De este modo, el problema de optimización para recuperar la señal se amplía añadiendo la información temporal como restricción, con el objetivo de corregir los errores originados por el movimiento de la escena. Los experimentos computacionales realizados en cuatro videos espectrales diferentes muestran una mejora de hasta 4dB en términos de relación pico-señal a ruido (PSNR) en la calidad de reconstrucción usando el enfoque multirresolución aplicado a la reconstrucción de video espectral con respecto al problema de optimización que no considera el movimiento

High-resolution Multi-spectral Imaging with Diffractive Lenses and Learned Reconstruction

Spectral imaging is a fundamental diagnostic technique with widespread application. Conventional spectral imaging approaches have intrinsic limitations on spatial and spectral resolutions due to the physical components they rely on. To overcome these physical limitations, in this paper, we develop a novel multi-spectral imaging modality that enables higher spatial and spectral resolutions. In the developed computational imaging modality, we exploit a diffractive lens, such as a photon sieve, for both dispersing and focusing the optical field, and achieve measurement diversity by changing the focusing behavior of this lens. Because the focal length of a diffractive lens is wavelength-dependent, each measurement is a superposition of differently blurred spectral components. To reconstruct the individual spectral images from these superimposed and blurred measurements, model-based fast reconstruction algorithms are developed with deep and analytical priors using alternating minimization and unrolling. Finally, the effectiveness and performance of the developed technique is illustrated for an application in astrophysical imaging under various observation scenarios in the extreme ultraviolet (EUV) regime. The results demonstrate that the technique provides not only diffraction-limited high spatial resolution, as enabled by diffractive lenses, but also the capability of resolving close-by spectral sources that would not otherwise be possible with the existing techniques. This work enables high resolution multi-spectral imaging with low cost designs for a variety of applications and spectral regimes.Comment: accepted for publication in IEEE Transactions on Computational Imaging, see DOI belo