A review of snapshot multidimensional optical imaging: Measuring photon tags in parallel

Multidimensional optical imaging has seen remarkable growth in the past decade. Rather than measuring only the two-dimensional spatial distribution of light, as in conventional photography, multidimensional optical imaging captures light in up to nine dimensions, providing unprecedented information about incident photons’ spatial coordinates, emittance angles, wavelength, time, and polarization. Multidimensional optical imaging can be accomplished either by scanning or parallel acquisition. Compared with scanning-based imagers, parallel acquisition–also dubbed snapshot imaging–has a prominent advantage in maximizing optical throughput, particularly when measuring a datacube of high dimensions. Here, we first categorize snapshot multidimensional imagers based on their acquisition and image reconstruction strategies, then highlight the snapshot advantage in the context of optical throughput, and finally we discuss their state-of-the-art implementations and applications

Low-power CMOS digital-pixel Imagers for high-speed uncooled PbSe IR applications

This PhD dissertation describes the research and development of a new low-cost medium wavelength infrared MWIR monolithic imager technology for high-speed uncooled industrial applications. It takes the baton on the latest technological advances in the field of vapour phase deposition (VPD) PbSe-based medium wavelength IR (MWIR) detection accomplished by the industrial partner NIT S.L., adding fundamental knowledge on the investigation of novel VLSI analog and mixed-signal design techniques at circuit and system levels for the development of the readout integrated device attached to the detector. The work supports on the hypothesis that, by the use of the preceding design techniques, current standard inexpensive CMOS technologies fulfill all operational requirements of the VPD PbSe detector in terms of connectivity, reliability, functionality and scalability to integrate the device. The resulting monolithic PbSe-CMOS camera must consume very low power, operate at kHz frequencies, exhibit good uniformity and fit the CMOS read-out active pixels in the compact pitch of the focal plane, all while addressing the particular characteristics of the MWIR detector: high dark-to-signal ratios, large input parasitic capacitance values and remarkable mismatching in PbSe integration. In order to achieve these demands, this thesis proposes null inter-pixel crosstalk vision sensor architectures based on a digital-only focal plane array (FPA) of configurable pixel sensors. Each digital pixel sensor (DPS) cell is equipped with fast communication modules, self-biasing, offset cancellation, analog-to-digital converter (ADC) and fixed pattern noise (FPN) correction. In-pixel power consumption is minimized by the use of comprehensive MOSFET subthreshold operation. The main aim is to potentiate the integration of PbSe-based infra-red (IR)-image sensing technologies so as to widen its use, not only in distinct scenarios, but also at different stages of PbSe-CMOS integration maturity. For this purpose, we posit to investigate a comprehensive set of functional blocks distributed in two parallel approaches: • Frame-based “Smart” MWIR imaging based on new DPS circuit topologies with gain and offset FPN correction capabilities. This research line exploits the detector pitch to offer fully-digital programmability at pixel level and complete functionality with input parasitic capacitance compensation and internal frame memory. • Frame-free “Compact”-pitch MWIR vision based on a novel DPS lossless analog integrator and configurable temporal difference, combined with asynchronous communication protocols inside the focal plane. This strategy is conceived to allow extensive pitch compaction and readout speed increase by the suppression of in-pixel digital filtering, and the use of dynamic bandwidth allocation in each pixel of the FPA. In order make the electrical validation of first prototypes independent of the expensive PbSe deposition processes at wafer level, investigation is extended as well to the development of affordable sensor emulation strategies and integrated test platforms specifically oriented to image read-out integrated circuits. DPS cells, imagers and test chips have been fabricated and characterized in standard 0.15μm 1P6M, 0.35μm 2P4M and 2.5μm 2P1M CMOS technologies, all as part of research projects with industrial partnership. The research has led to the first high-speed uncooled frame-based IR quantum imager monolithically fabricated in a standard VLSI CMOS technology, and has given rise to the Tachyon series [1], a new line of commercial IR cameras used in real-time industrial, environmental and transportation control systems. The frame-free architectures investigated in this work represent a firm step forward to push further pixel pitch and system bandwidth up to the limits imposed by the evolving PbSe detector in future generations of the device.La present tesi doctoral descriu la recerca i el desenvolupament d'una nova tecnologia monolítica d'imatgeria infraroja de longitud d'ona mitja (MWIR), no refrigerada i de baix cost, per a usos industrials d'alta velocitat. El treball pren el relleu dels últims avenços assolits pel soci industrial NIT S.L. en el camp dels detectors MWIR de PbSe depositats en fase vapor (VPD), afegint-hi coneixement fonamental en la investigació de noves tècniques de disseny de circuits VLSI analògics i mixtes pel desenvolupament del dispositiu integrat de lectura unit al detector pixelat. Es parteix de la hipòtesi que, mitjançant l'ús de les esmentades tècniques de disseny, les tecnologies CMOS estàndard satisfan tots els requeriments operacionals del detector VPD PbSe respecte a connectivitat, fiabilitat, funcionalitat i escalabilitat per integrar de forma econòmica el dispositiu. La càmera PbSe-CMOS resultant ha de consumir molt baixa potència, operar a freqüències de kHz, exhibir bona uniformitat, i encabir els píxels actius CMOS de lectura en el pitch compacte del pla focal de la imatge, tot atenent a les particulars característiques del detector: altes relacions de corrent d'obscuritat a senyal, elevats valors de capacitat paràsita a l'entrada i dispersions importants en el procés de fabricació. Amb la finalitat de complir amb els requisits previs, es proposen arquitectures de sensors de visió de molt baix acoblament interpíxel basades en l'ús d'una matriu de pla focal (FPA) de píxels actius exclusivament digitals. Cada píxel sensor digital (DPS) està equipat amb mòduls de comunicació d'alta velocitat, autopolarització, cancel·lació de l'offset, conversió analògica-digital (ADC) i correcció del soroll de patró fixe (FPN). El consum en cada cel·la es minimitza fent un ús exhaustiu del MOSFET operant en subllindar. L'objectiu últim és potenciar la integració de les tecnologies de sensat d'imatge infraroja (IR) basades en PbSe per expandir-ne el seu ús, no només a diferents escenaris, sinó també en diferents estadis de maduresa de la integració PbSe-CMOS. En aquest sentit, es proposa investigar un conjunt complet de blocs funcionals distribuïts en dos enfocs paral·lels: - Dispositius d'imatgeria MWIR "Smart" basats en frames utilitzant noves topologies de circuit DPS amb correcció de l'FPN en guany i offset. Aquesta línia de recerca exprimeix el pitch del detector per oferir una programabilitat completament digital a nivell de píxel i plena funcionalitat amb compensació de la capacitat paràsita d'entrada i memòria interna de fotograma. - Dispositius de visió MWIR "Compact"-pitch "frame-free" en base a un novedós esquema d'integració analògica en el DPS i diferenciació temporal configurable, combinats amb protocols de comunicació asíncrons dins del pla focal. Aquesta estratègia es concep per permetre una alta compactació del pitch i un increment de la velocitat de lectura, mitjançant la supressió del filtrat digital intern i l'assignació dinàmica de l'ample de banda a cada píxel de l'FPA. Per tal d'independitzar la validació elèctrica dels primers prototips respecte a costosos processos de deposició del PbSe sensor a nivell d'oblia, la recerca s'amplia també al desenvolupament de noves estratègies d'emulació del detector d'IR i plataformes de test integrades especialment orientades a circuits integrats de lectura d'imatge. Cel·les DPS, dispositius d'imatge i xips de test s'han fabricat i caracteritzat, respectivament, en tecnologies CMOS estàndard 0.15 micres 1P6M, 0.35 micres 2P4M i 2.5 micres 2P1M, tots dins el marc de projectes de recerca amb socis industrials. Aquest treball ha conduït a la fabricació del primer dispositiu quàntic d'imatgeria IR d'alta velocitat, no refrigerat, basat en frames, i monolíticament fabricat en tecnologia VLSI CMOS estàndard, i ha donat lloc a Tachyon, una nova línia de càmeres IR comercials emprades en sistemes de control industrial, mediambiental i de transport en temps real.Postprint (published version

Lenslet array tunable snapshot imaging spectrometer (LATIS) for hyperspectral fluorescence microscopy

Snapshot hyperspectral imaging augments pixel dwell time and acquisition speeds over existing scanning systems, making it a powerful tool for fluorescence microscopy. While most snapshot systems contain fixed datacube parameters (x,y,λ), our novel snapshot system, called the lenslet array tunable snapshot imaging spectrometer (LATIS), demonstrates tuning its average spectral resolution from 22.66 nm (80x80x22) to 13.94 nm (88x88x46) over 485 to 660 nm. We also describe a fixed LATIS with a datacube of 200x200x27 for larger field-of-view (FOV) imaging. We report <1 sec exposure times and high resolution fluorescence imaging with minimal artifacts

A comprehensive approach for the efficient acquisition and processing of hyperspectral images and sequence

Programa Oficial de Doctorado en Computación. 5009P01[Abstract] Despite the scientific and technological developments achieved during the last two decades in the hyperspectral field, some methodological, operational and conceptual issues have restricted the progress, promotion and popular dissemination of this technology. These shortcomings include the specialized knowledge required for the acquisition of hyperspectral images, the shortage of publicly accessible hyperspectral image repositories with reliable ground truth images or the lack of methodologies that allow for the adaptation of algorithms to particular user or application processing needs. The work presented here has the objective of contributing to the hyperspectral field with procedures for the automatic acquisition of hyperspectral scenes, including the hardware adaptation of our own imagers and the development of methods for the calibration and correction of the hyperspectral datacubes, the creation of a publicly available hyperspectral repository of well categorized and labeled images and the design and implementation of novel computational intelligence based processing techniques that solve typical issues related to the segmentation and denoising of hyperspectral images as well as sequences of them taking into account their temporal evolution.[Resumen] A pesar de los desarrollos tecnológicos y científicos logrados en el campo hiperespectral durante las dos últimas décadas, alg\mas limitaciones de tipo metodológico, operacional y conceptual han restringido el progreso, difusión y popularización de esta tecnología, entre ellas, el conocimiento especializado requerido en la adquisición de imágenes hiperespectrales, la carencia de repositorios de imágenes hiperespectrales con etiquetados fiables y de acceso público o la falta de metodologías que posibiliten la adaptación de algoritmos a usuarios o necesidades de procesamiento concretas. Este trabajo doctoral tiene el objetivo de contribuir al campo hiperespectral con procedimientos para la adquisición automática de escenas hiperespectrales, incluyendo la adaptación hardware de cámaras hiperespectrales propias y el desarrollo de métodos para la calibración y corrección de cubos de datos hiperespectrales; la creación de un repositorio hiperespectral de acceso público con imágenes categorizadas y con verdades de terreno fiables; y el diseño e implementación de técnicas de procesamiento basadas en inteligencia computacional para la resolución de problemas típicamente relacionados con las tareas de segmentación y eliminación de ruido en imágenes estáticas y secuencias de imágenes hiperespectrales teniendo en consideración su evolución temporal.[Resumo] A pesar dos desenvolvementos tecnolóxicos e científicos logrados no campo hiperespectral durante as dúas últimas décadas, algunhas lirrútacións de tipo metodolóxico¡ operacional e conceptual restrinxiron o progreso) difusión e popularización desta tecnoloxía, entre elas, o coñecemento especializado requirido na adquisición de imaxes hiperespectrales¡ a carencia de repositorios de irnaxes hiperespectrales con etiquetaxes fiables e de acceso público ou a falta de metodoloxías que posibiliten a adaptación de algoritmos a usuarios ou necesidades de procesamento concretas. Este traballo doutoral ten o obxectívo de contribuir ao campo hiperespectral con procedementos para a adquisición automática de eicenas hiperespectrais, incluíndo a adaptación hardware de cámaras hiperespectrales propias e o desenvolvemento de métodos para a calibración e corrección de cubos de datos hiperespectrais; a creación dun repositorio hiperespectral de acceso público con imaxes categorizadas e con verdades de terreo fiables; e o deseño e implementación de técnicas de procesamento baseadas en intelixencia computacional para a resolución de problemas tipicamente relacionado~ coas tarefas de segmentación e eliminación de ruído en imaxes estáticas e secuencias de imaxes hiperespectrai~ tendo en consideración a súa evolución temporal

A Compact, High Resolution Hyperspectral Imager for Remote Sensing of Soil Moisture

Measurement of soil moisture content is a key challenge across a variety of fields, ranging from civil engineering through to defence and agriculture. While dedicated satellite platforms like SMAP and SMOS provide high spatial coverage, their low spatial resolution limits their application to larger regional studies. The advent of compact, high lift capacity UAVs has enabled small scale surveys of specific farmland cites. This thesis presents work on the development of a compact, high spatial and spectral resolution hyperspectral imager, designed for remote measurement of soil moisture content. The optical design of the system incorporates a bespoke freeform blazed diffraction grating, providing higher optical performance at a similar aperture to conventional Offner-Chrisp designs. The key challenges of UAV-borne hyperspectral imaging relate to using only solar illumination, with both intermittent cloud cover and atmospheric water absorption creating challenges in obtaining accurate reflectance measurements. A hardware based calibration channel for mitigating cloud cover effects is introduced, along with a comparison of methods for recovering soil moisture content from reflectance data under varying illumination conditions. The data processing pipeline required to process the raw pushbroom data into georectified images is also discussed. Finally, preliminary work on applying soil moisture techniques to leaf imaging are presented

Instrumentation for solar spectropolarimetry: state of the art and prospects

Given its unchallenged capabilities in terms of sensitivity and spatial resolution, the combination of imaging spectropolarimetry and numeric Stokes inversion represents the dominant technique currently used to remotely sense the physical properties of the solar atmosphere and, in particular, its important driving magnetic field. Solar magnetism manifests itself in a wide range of spatial, temporal, and energetic scales. The ubiquitous but relatively small and weak fields of the so-called quiet Sun are believed today to be crucial for answering many open questions in solar physics, some of which have substantial practical relevance due to the strong Sun?Earth connection. However, such fields are very challenging to detect because they require spectropolarimetric measurements with high spatial (sub-arcsec), spectral (<100  mÅ), and temporal (<10  s) resolution along with high polarimetric sensitivity (<0.1  %   of the intensity). We collect and discuss both well-established and upcoming instrumental solutions developed during the last decades to push solar observations toward the above-mentioned parameter regime. This typically involves design trade-offs due to the high dimensionality of the data and signal-to-noise-ratio considerations, among others. We focus on the main three components that form a spectropolarimeter, namely, wavelength discriminators, the devices employed to encode the incoming polarization state into intensity images (polarization modulators), and the sensor technologies used to register them. We consider the instrumental solutions introduced to perform this kind of measurements at different optical wavelengths and from various observing locations, i.e., ground-based, from the stratosphere or near space.Fil: Iglesias, Francisco Andres. Universidad Tecnológica Nacional. Facultad Regional de Mendoza; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Mendoza; ArgentinaFil: Feller, Alex. Max Planck Institut Fur Sonnensystemforschung; Alemani

A review of snapshot multidimensional optical imaging: Measuring photon tags in parallel

Multidimensional optical imaging has seen remarkable growth in the past decade. Rather than measuring only the two-dimensional spatial distribution of light, as in conventional photography, multidimensional optical imaging captures light in up to nine dimensions, providing unprecedented information about incident photons’ spatial coordinates, emittance angles, wavelength, time, and polarization. Multidimensional optical imaging can be accomplished either by scanning or parallel acquisition. Compared with scanning-based imagers, parallel acquisition–also dubbed snapshot imaging–has a prominent advantage in maximizing optical throughput, particularly when measuring a datacube of high dimensions. Here, we first categorize snapshot multidimensional imagers based on their acquisition and image reconstruction strategies, then highlight the snapshot advantage in the context of optical throughput, and finally we discuss their state-of-the-art implementations and applications

Snapshot Hyperspectral Imaging for Complete Fundus Oximetry

In this work, a snapshot hyperspectral imager capable of tuning its average spectral resolution from 22.7 nm to 13.9 nm in a single integrated form is presented. The principle of this system will enable future snapshot systems to dynamically adapt to a wide range of imaging situations. Additionally, the system overcomes datacube size limitations imposed by detector array size limits. The work done in this thesis also advances oximetry of the retina using data collected by the Image Mapping spectrometer (IMS), a snapshot spectrometer. Hyperspectral images of the retina are acquired, and oximetry of individual vessels in four diseased eyes is presented. Further, oximetry of the entire fundus is performed using a novel algorithm with data collected with the IMS. We present oxyhemoglobin concentration maps of the eye and demonstrate oxygen sensitivity of the maps by comparing normal and diseased eyes. The aim of this work is to advance the general capabilities of snapshot hyperspectral imagers and to advance the integration of retinal oximetry into the standard ophthalmology instrument repertoire

CMOS Approach to Compressed-domain Image Acquisition

A hardware implementation of a real-time compressed-domain image acquisition system is demonstrated. The system performs front-end computational imaging, whereby the inner product between an image and an arbitrarily-specified mask is implemented in silicon. The acquisition system is based on an intelligent readout integrated circuit (iROIC) that is capable of providing independent bias voltages to individual detectors, which enables implementation of spatial multiplication with any prescribed mask through a bias-controlled response-modulation mechanism. The modulated pixels are summed up in the image grabber to generate the compressed samples, namely aperture-coded coefficients, of an image. A rigorous bias-selection algorithm is presented to the readout circuit, which exploits the bias-dependent nature of the imager’s responsivity. Proven functionality of the hardware in transform coding compressed image acquisition, silicon-level compressive sampling, in pixel nonuniformity correction and hardware-level implementation of region-based enhancement is demonstrated