InSPECtor: an end-to-end design framework for compressive pixelated hyperspectral instruments

Classic designs of hyperspectral instrumentation densely sample the spatial and spectral information of the scene of interest. Data may be compressed after the acquisition. In this paper we introduce a framework for the design of an optimized, micro-patterned snapshot hyperspectral imager that acquires an optimized subset of the spatial and spectral information in the scene. The data is thereby compressed already at the sensor level, but can be restored to the full hyperspectral data cube by the jointly optimized reconstructor. This framework is implemented with TensorFlow and makes use of its automatic differentiation for the joint optimization of the layout of the micro-patterned filter array as well as the reconstructor. We explore the achievable compression ratio for different numbers of filter passbands, number of scanning frames, and filter layouts using data collected by the Hyperscout instrument. We show resulting instrument designs that take snapshot measurements without losing significant information while reducing the data volume, acquisition time, or detector space by a factor of 40 as compared to classic, dense sampling. The joint optimization of a compressive hyperspectral imager design and the accompanying reconstructor provides an avenue to substantially reduce the data volume from hyperspectral imagers.Comment: 23 pages, 12 figures, published in Applied Optic

Universal Demosaicking of Color Filter Arrays

A large number of color filter arrays (CFAs), periodic or aperiodic, have been proposed. To reconstruct images from all different CFAs and compare their imaging quality, a universal demosaicking method is needed. This paper proposes a new universal demosaicking method based on inter-pixel chrominance capture and optimal demosaicking transformation. It skips the commonly used step to estimate the luminance component at each pixel, and thus, avoids the associated estimation error. Instead, we directly use the acquired CFA color intensity at each pixel as an input component. Two independent chrominance components are estimated at each pixel based on the interpixel chrominance in the window, which is captured with the difference of CFA color values between the pixel of interest and its neighbors. Two mechanisms are employed for the accurate estimation: distance-related and edge-sensing weighting to reflect the confidence levels of the inter-pixel chrominance components, and pseudoinverse-based estimation from the components in a window. Then from the acquired CFA color component and two estimated chrominance components, the three primary colors are reconstructed by a linear color transform, which is optimized for the least transform error. Our experiments show that the proposed method is much better than other published universal demosaicking methods.National Key Basic Research Project of China (973 Program) [2015CB352303, 2011CB302400]; National Natural Science Foundation (NSF) of China [61071156, 61671027]SCI(E)[email protected]; [email protected]; [email protected]; [email protected]

Demosaicing of Color Images by Accurate Estimation of Luminance

Digital cameras acquire color images using a single sensor with Color filter Arrays. A single color component per pixel is acquired using color filter arrays and the remaining two components are obtained using demosaicing techniques. The conventional demosaicing techniques existent induce artifacts in resultant images effecting reconstruction quality. To overcome this drawback a frequency based demosaicing technique is proposed. The luminance and chrominance components extracted from the frequency domain of the image are interpolated to produce intermediate demosaiced images. A novel Neural Network Based Image Reconstruction Algorithm is applied to the intermediate demosaiced image to obtain resultant demosaiced images. The results presented in the paper prove the proposed demosaicing technique exhibits the best performance and is applicable to a wide variety of images