Hardware implementation algorithm and error analysis of high-speed fluorescence lifetime sensing systems using center-of-mass method

A new, simple, high-speed, and hardware-only integrationbased fluorescence-lifetime-sensing algorithm using a center-of-mass method CMM is proposed to implement lifetime calculations, and its signal-to-noise-ratio based on statistics theory is also deduced. Compared to the commonly used iterative least-squares method or the maximum-likelihood-estimation–based, general purpose fluorescence lifetime imaging microscopy FLIM analysis software, the proposed hardware lifetime calculation algorithm with CMM offers direct calculation of fluorescence lifetime based on the collected photon counts and timing information provided by in-pixel circuitry and therefore delivers faster analysis for real-time applications, such as clinical diagnosis. A real-time hardware implementation of this CMM FLIM algorithm suitable for a single-photon avalanche diode array in CMOS imaging technology is now proposed for implementation on field-programmable gate array. The performance of the proposed methods has been tested on Fluorescein, Coumarin 6, and 1,8- anilinonaphthalenesulfonate in water/methanol mixture

Low Noise and High Photodetection Probability SPAD in 180 nm Standard CMOS Technology

A square shaped, low noise and high photo-response single photon avalanche diode suitable for circuit integration, implemented in a standard CMOS 180 nm high voltage technology, is presented. In this work, a p+ to shallow n-well junction was engineered with a very smooth electric field profile guard ring to attain a photo detection probability peak higher than 50% with a median dark count rate lower than 2 Hz/μm2 when operated at an excess bias of 4 V. The reported timing jitter full width at half maximum is below 300 ps for 640 nm laser pulses

High-Speed Wide-Field Time-Correlated Single-Photon Counting Fluorescence Lifetime Imaging Microscopy

Fluorescence microscopy is a powerful imaging technique used in the biological sciences to identify labeled components of a sample with specificity. This is usually accomplished through labeling with fluorescent dyes, isolating these dyes by their spectral signatures with optical filters, and recording the intensity of the fluorescent response. Although these techniques are widely used, fluorescence intensity images can be negatively affected by a variety of factors that impact the fluorescence intensity. Fluorescence lifetime imaging microscopy (FLIM) is an imaging technique that is relatively immune to intensity fluctuations and also provides the unique ability to directly monitor the microenvironment surrounding a fluorophore. Despite the benefits associated with FLIM, the applications to which it is applied are fairly limited due to long image acquisition times and high cost of traditional hardware. Recent advances in complementary metal-oxide-semiconductor (CMOS) single-photon avalanche diodes (SPADs) have enabled the design of low-cost imaging arrays that are capable of recording lifetime images with acquisition times greater than one order of magnitude faster than existing systems. However, these SPAD arrays have yet to realize the full potential of the technology due to limitations in their ability to handle the vast amount of data generated during the commonly used time-correlated single-photon counting (TCSPC) lifetime imaging technique. This thesis presents the design, implementation, characterization, and demonstration of a high speed FLIM imaging system. The components of this design include a CMOS imager chip in a standard 0.13 μm technology containing a custom CMOS SPAD, a 64-by-64 array of these SPADs, pixel control circuitry, independent time-to-digital converters (TDCs), a FLIM specific datapath, and high bandwidth output buffers. In addition to the CMOS imaging array, a complete system was designed and implemented using a printed circuit board (PCB) for capturing data from the imager, creating histograms for the photon arrival data using field-programmable gate arrays, and transferring the data to a computer using a cabled PCIe interface. Finally, software is used to communicate between the imaging system and a computer.The dark count rate of the SPAD was measured to be only 231 Hz at room temperature while maintaining a photon detection probability of up to 30\%. TDCs included on the array have a 62.5 ps resolution and a 64 ns range, which is suitable for measuring the lifetime of most biological fluorophores. Additionally, the on-chip datapath was designed to handle continuous data transfers at rates capable of supporting TCSPC-based lifetime imaging at 100 frames per second. The system level implementation also provides sufficient data throughput for transferring up to 750 frames per second from the imaging system to a computer. The lifetime imaging system was characterized using standard techniques for evaluating SPAD performance and an electrical delay signal for measuring the TDC performance. This thesis concludes with a demonstration of TCSPC-FLIM imaging at 100 frames per second -- the fastest 64-by-64 TCSPC FLIM that has been demonstrated. This system overcomes some of the limitations of existing FLIM systems and has the potential to enable new application domains in dynamic FLIM imaging

Fast fluorescence lifetime imaging and sensing via deep learning

Error on title page – year of award is 2023.Fluorescence lifetime imaging microscopy (FLIM) has become a valuable tool in diverse disciplines. This thesis presents deep learning (DL) approaches to addressing two major challenges in FLIM: slow and complex data analysis and the high photon budget for precisely quantifying the fluorescence lifetimes. DL's ability to extract high-dimensional features from data has revolutionized optical and biomedical imaging analysis. This thesis contributes several novel DL FLIM algorithms that significantly expand FLIM's scope. Firstly, a hardware-friendly pixel-wise DL algorithm is proposed for fast FLIM data analysis. The algorithm has a simple architecture yet can effectively resolve multi-exponential decay models. The calculation speed and accuracy outperform conventional methods significantly. Secondly, a DL algorithm is proposed to improve FLIM image spatial resolution, obtaining high-resolution (HR) fluorescence lifetime images from low-resolution (LR) images. A computational framework is developed to generate large-scale semi-synthetic FLIM datasets to address the challenge of the lack of sufficient high-quality FLIM datasets. This algorithm offers a practical approach to obtaining HR FLIM images quickly for FLIM systems. Thirdly, a DL algorithm is developed to analyze FLIM images with only a few photons per pixel, named Few-Photon Fluorescence Lifetime Imaging (FPFLI) algorithm. FPFLI uses spatial correlation and intensity information to robustly estimate the fluorescence lifetime images, pushing this photon budget to a record-low level of only a few photons per pixel. Finally, a time-resolved flow cytometry (TRFC) system is developed by integrating an advanced CMOS single-photon avalanche diode (SPAD) array and a DL processor. The SPAD array, using a parallel light detection scheme, shows an excellent photon-counting throughput. A quantized convolutional neural network (QCNN) algorithm is designed and implemented on a field-programmable gate array as an embedded processor. The processor resolves fluorescence lifetimes against disturbing noise, showing unparalleled high accuracy, fast analysis speed, and low power consumption.Fluorescence lifetime imaging microscopy (FLIM) has become a valuable tool in diverse disciplines. This thesis presents deep learning (DL) approaches to addressing two major challenges in FLIM: slow and complex data analysis and the high photon budget for precisely quantifying the fluorescence lifetimes. DL's ability to extract high-dimensional features from data has revolutionized optical and biomedical imaging analysis. This thesis contributes several novel DL FLIM algorithms that significantly expand FLIM's scope. Firstly, a hardware-friendly pixel-wise DL algorithm is proposed for fast FLIM data analysis. The algorithm has a simple architecture yet can effectively resolve multi-exponential decay models. The calculation speed and accuracy outperform conventional methods significantly. Secondly, a DL algorithm is proposed to improve FLIM image spatial resolution, obtaining high-resolution (HR) fluorescence lifetime images from low-resolution (LR) images. A computational framework is developed to generate large-scale semi-synthetic FLIM datasets to address the challenge of the lack of sufficient high-quality FLIM datasets. This algorithm offers a practical approach to obtaining HR FLIM images quickly for FLIM systems. Thirdly, a DL algorithm is developed to analyze FLIM images with only a few photons per pixel, named Few-Photon Fluorescence Lifetime Imaging (FPFLI) algorithm. FPFLI uses spatial correlation and intensity information to robustly estimate the fluorescence lifetime images, pushing this photon budget to a record-low level of only a few photons per pixel. Finally, a time-resolved flow cytometry (TRFC) system is developed by integrating an advanced CMOS single-photon avalanche diode (SPAD) array and a DL processor. The SPAD array, using a parallel light detection scheme, shows an excellent photon-counting throughput. A quantized convolutional neural network (QCNN) algorithm is designed and implemented on a field-programmable gate array as an embedded processor. The processor resolves fluorescence lifetimes against disturbing noise, showing unparalleled high accuracy, fast analysis speed, and low power consumption

A Point-of-Care Device for Molecular Diagnosis Based on CMOS SPAD Detectors with Integrated Microfluidics

We describe the integration of techniques and technologies to develop a Point-of-Care for molecular diagnosis PoC-MD, based on a fluorescence lifetime measurement. Our PoC-MD is a low-cost, simple, fast, and easy-to-use general-purpose platform, aimed at carrying out fast diagnostics test through label detection of a variety of biomarkers. It is based on a 1-D array of 10 ultra-sensitive Single-Photon Avalanche Diode (SPAD) detectors made in a 0.18 μm High-Voltage Complementary Metal Oxide Semiconductor (HV-CMOS) technology. A custom microfluidic polydimethylsiloxane cartridge to insert the sample is straightforwardly positioned on top of the SPAD array without any alignment procedure with the SPAD array. Moreover, the proximity between the sample and the gate-operated SPAD sensor makes unnecessary any lens or optical filters to detect the fluorescence for long lifetime fluorescent dyes, such as quantum dots. Additionally, the use of a low-cost laser diode as pulsed excitation source and a Field-Programmable Gate Array (FPGA) to implement the control and processing electronics, makes the device flexible and easy to adapt to the target label molecule by only changing the laser diode. Using this device, reliable and sensitive real-time proof-of-concept fluorescence lifetime measurement of quantum dot QdotTM 605 streptavidin conjugate is demonstrated

A 72 × 60 Angle-Sensitive SPAD Imaging Array for Lens-less FLIM

We present a 72 × 60, angle-sensitive single photon avalanche diode (A-SPAD) array for lens-less 3D fluorescence lifetime imaging. An A-SPAD pixel consists of (1) a SPAD to provide precise photon arrival time where a time-resolved operation is utilized to avoid stimulus-induced saturation, and (2) integrated diffraction gratings on top of the SPAD to extract incident angles of the incoming light. The combination enables mapping of fluorescent sources with different lifetimes in 3D space down to micrometer scale. Futhermore, the chip presented herein integrates pixel-level counters to reduce output data-rate and to enable a precise timing control. The array is implemented in standard 180 nm complementary metal-oxide-semiconductor (CMOS) technology and characterized without any post-processing

A 64x64 SPAD array for portable colorimetric sensing, fluorescence and X-ray imaging

We present the design and application of a 64x64 pixel SPAD array to portable colorimetric sensing, and fluorescence and x-ray imaging. The device was fabricated on an unmodified 180 nm CMOS process and is based on a square p+/n active junction SPAD geometry suitable for detecting green fluorescence emission. The stand-alone SPAD shows a photodetection probability greater than 60% at 5 V excess bias, with a dark count rate of less than 4 cps/µm2 and sub-ns timing jitter performance. It has a global shutter with an in-pixel 8-bit counter; four 5-bit decoders and two 64-to-1 multiplexer blocks allow the data to be read-out. The array of sensors was able to detect fluorescence from a fluorescein isothiocyanate (FITC) solution down to a concentration of 900 pM with a SNR of 9.8 dB. A colorimetric assay was performed on top of the sensor array with a limit of quantification of 3.1 µM. X-rays images, using energies ranging from 10 kVp to 100 kVp, of a lead grating mask were acquired without using a scintillation crystal