GPU acceleration of time-domain fluorescence lifetime imaging

Fluorescence lifetime imaging microscopy (FLIM) plays a significant role in biological sciences, chemistry, and medical research. We propose a Graphic Processing Units (GPUs) based FLIM analysis tool suitable for high-speed and flexible time-domain FLIM applications. With a large number of parallel processors, GPUs can significantly speed up lifetime calculations compared to CPU-OpenMP (parallel computing with multiple CPU cores) based analysis. We demonstrate how to implement and optimize FLIM algorithms on GPUs for both iterative and non-iterative FLIM analysis algorithms. The implemented algorithms have been tested on both synthesized and experimental FLIM data. The results show that at the same precision the GPU analysis can be up to 24-fold faster than its CPU-OpenMP counterpart. This means that even for high precision but time-consuming iterative FLIM algorithms, GPUs enable fast or even real-time analysis

High-resolution remote thermography using luminescent low-dimensional tin-halide perovskites

While metal-halide perovskites have recently revolutionized research in optoelectronics through a unique combination of performance and synthetic simplicity, their low-dimensional counterparts can further expand the field with hitherto unknown and practically useful optical functionalities. In this context, we present the strong temperature dependence of the photoluminescence (PL) lifetime of low-dimensional, perovskite-like tin-halides, and apply this property to thermal imaging with a high precision of 0.05 {\deg}C. The PL lifetimes are governed by the heat-assisted de-trapping of self-trapped excitons, and their values can be varied over several orders of magnitude by adjusting the temperature (up to 20 ns {\deg}C-1). Typically, this sensitive range spans up to one hundred centigrade, and it is both compound-specific and shown to be compositionally and structurally tunable from -100 to 110 {\deg} C going from [C(NH2)3]2SnBr4 to Cs4SnBr6 and (C4N2H14I)4SnI6. Finally, through the innovative implementation of cost-effective hardware for fluorescence lifetime imaging (FLI), based on time-of-flight (ToF) technology, these novel thermoluminophores have been used to record thermographic videos with high spatial and thermal resolution.Comment: 25 pages, 4 figure

Widefield multifrequency fluorescence lifetime imaging using a two-tap complementary metal-oxide semiconductor camera with lateral electric field charge modulators.

Widefield frequency-domain fluorescence lifetime imaging microscopy (FD-FLIM) measures the fluorescence lifetime of entire images in a fast and efficient manner. We report a widefield FD-FLIM system based on a complementary metal-oxide semiconductor camera equipped with two-tap true correlated double sampling lock-in pixels and lateral electric field charge modulators. Owing to the fast intrinsic response and modulation of the camera, our system allows parallel multifrequency FLIM in one measurement via fast Fourier transform. We demonstrate that at a fundamental frequency of 20 MHz, 31-harmonics can be measured with 64 phase images per laser repetition period. As a proof of principle, we analyzed cells transfected with Cerulean and with a construct of Cerulean-Venus that shows Förster Resonance Energy Transfer at different modulation frequencies. We also tracked the temperature change of living cells via the fluorescence lifetime of Rhodamine B at different frequencies. These results indicate that our widefield multifrequency FD-FLIM system is a valuable tool in the biomedical field

Fluorescence lifetime biosensing with DNA microarrays and a CMOS-SPAD imager

Fluorescence lifetime of dye molecules is a sensitive reporter on local microenvironment which is generally independent of fluorophores concentration and can be used as a means of discrimination between molecules with spectrally overlapping emission. It is therefore a potentially powerful multiplexed detection modality in biosensing but requires extremely low light level operation typical of biological analyte concentrations, long data acquisition periods and on-chip processing capability to realize these advantages. We report here fluorescence lifetime data obtained using a CMOS-SPAD imager in conjunction with DNA microarrays and TIRF excitation geometry. This enables acquisition of single photon arrival time histograms for a 320 pixel FLIM map within less than 26 seconds exposure time. From this, we resolve distinct lifetime signatures corresponding to dye-labelled HCV and quantum-dot-labelled HCMV nucleic acid targets at concentrations as low as 10 nM

A 72 × 60 Angle-Sensitive SPAD Imaging Array for Lens-less FLIM

We present a 72 × 60, angle-sensitive single photon avalanche diode (A-SPAD) array for lens-less 3D fluorescence lifetime imaging. An A-SPAD pixel consists of (1) a SPAD to provide precise photon arrival time where a time-resolved operation is utilized to avoid stimulus-induced saturation, and (2) integrated diffraction gratings on top of the SPAD to extract incident angles of the incoming light. The combination enables mapping of fluorescent sources with different lifetimes in 3D space down to micrometer scale. Futhermore, the chip presented herein integrates pixel-level counters to reduce output data-rate and to enable a precise timing control. The array is implemented in standard 180 nm complementary metal-oxide-semiconductor (CMOS) technology and characterized without any post-processing

Fast fluorescence lifetime imaging and sensing via deep learning

Error on title page – year of award is 2023.Fluorescence lifetime imaging microscopy (FLIM) has become a valuable tool in diverse disciplines. This thesis presents deep learning (DL) approaches to addressing two major challenges in FLIM: slow and complex data analysis and the high photon budget for precisely quantifying the fluorescence lifetimes. DL's ability to extract high-dimensional features from data has revolutionized optical and biomedical imaging analysis. This thesis contributes several novel DL FLIM algorithms that significantly expand FLIM's scope. Firstly, a hardware-friendly pixel-wise DL algorithm is proposed for fast FLIM data analysis. The algorithm has a simple architecture yet can effectively resolve multi-exponential decay models. The calculation speed and accuracy outperform conventional methods significantly. Secondly, a DL algorithm is proposed to improve FLIM image spatial resolution, obtaining high-resolution (HR) fluorescence lifetime images from low-resolution (LR) images. A computational framework is developed to generate large-scale semi-synthetic FLIM datasets to address the challenge of the lack of sufficient high-quality FLIM datasets. This algorithm offers a practical approach to obtaining HR FLIM images quickly for FLIM systems. Thirdly, a DL algorithm is developed to analyze FLIM images with only a few photons per pixel, named Few-Photon Fluorescence Lifetime Imaging (FPFLI) algorithm. FPFLI uses spatial correlation and intensity information to robustly estimate the fluorescence lifetime images, pushing this photon budget to a record-low level of only a few photons per pixel. Finally, a time-resolved flow cytometry (TRFC) system is developed by integrating an advanced CMOS single-photon avalanche diode (SPAD) array and a DL processor. The SPAD array, using a parallel light detection scheme, shows an excellent photon-counting throughput. A quantized convolutional neural network (QCNN) algorithm is designed and implemented on a field-programmable gate array as an embedded processor. The processor resolves fluorescence lifetimes against disturbing noise, showing unparalleled high accuracy, fast analysis speed, and low power consumption.Fluorescence lifetime imaging microscopy (FLIM) has become a valuable tool in diverse disciplines. This thesis presents deep learning (DL) approaches to addressing two major challenges in FLIM: slow and complex data analysis and the high photon budget for precisely quantifying the fluorescence lifetimes. DL's ability to extract high-dimensional features from data has revolutionized optical and biomedical imaging analysis. This thesis contributes several novel DL FLIM algorithms that significantly expand FLIM's scope. Firstly, a hardware-friendly pixel-wise DL algorithm is proposed for fast FLIM data analysis. The algorithm has a simple architecture yet can effectively resolve multi-exponential decay models. The calculation speed and accuracy outperform conventional methods significantly. Secondly, a DL algorithm is proposed to improve FLIM image spatial resolution, obtaining high-resolution (HR) fluorescence lifetime images from low-resolution (LR) images. A computational framework is developed to generate large-scale semi-synthetic FLIM datasets to address the challenge of the lack of sufficient high-quality FLIM datasets. This algorithm offers a practical approach to obtaining HR FLIM images quickly for FLIM systems. Thirdly, a DL algorithm is developed to analyze FLIM images with only a few photons per pixel, named Few-Photon Fluorescence Lifetime Imaging (FPFLI) algorithm. FPFLI uses spatial correlation and intensity information to robustly estimate the fluorescence lifetime images, pushing this photon budget to a record-low level of only a few photons per pixel. Finally, a time-resolved flow cytometry (TRFC) system is developed by integrating an advanced CMOS single-photon avalanche diode (SPAD) array and a DL processor. The SPAD array, using a parallel light detection scheme, shows an excellent photon-counting throughput. A quantized convolutional neural network (QCNN) algorithm is designed and implemented on a field-programmable gate array as an embedded processor. The processor resolves fluorescence lifetimes against disturbing noise, showing unparalleled high accuracy, fast analysis speed, and low power consumption

Fast and simple spectral FLIM for biochemical and medical imaging.

Spectrally resolved fluorescence lifetime imaging microscopy (λFLIM) has powerful potential for biochemical and medical imaging applications. However, long acquisition times, low spectral resolution and complexity of λFLIM often narrow its use to specialized laboratories. Therefore, we demonstrate here a simple spectral FLIM based on a solid-state detector array providing in-pixel histrogramming and delivering faster acquisition, larger dynamic range, and higher spectral elements than state-of-the-art λFLIM. We successfully apply this novel microscopy system to biochemical and medical imaging demonstrating that solid-state detectors are a key strategic technology to enable complex assays in biomedical laboratories and the clinic.A.E. thanks the EPSRC for the initial funding of the project (EP/F044011/1) from 2009 to 2011. M.P. and L.D.C. were supported by a Programme Grant to A.R.V. from the UK Medical Research Council (MRC). This project was also supported by the MRC’s grant-in-aid to the Cancer Unit, Cambridge (A.E., A.R.V.). C.F.K acknowledges funding from the MRC (grant MR/K015850/1), the Wellcome Trust (grant 089703/Z/09/Z) and the EPSRC (EP/L015889/1).This is the author accepted manuscript. The final version is available from the Optical Society of America via http://dx.doi.org/10.1364/OE.23.02351