Fast fluorescence lifetime imaging and sensing via deep learning

Error on title page – year of award is 2023.Fluorescence lifetime imaging microscopy (FLIM) has become a valuable tool in diverse disciplines. This thesis presents deep learning (DL) approaches to addressing two major challenges in FLIM: slow and complex data analysis and the high photon budget for precisely quantifying the fluorescence lifetimes. DL's ability to extract high-dimensional features from data has revolutionized optical and biomedical imaging analysis. This thesis contributes several novel DL FLIM algorithms that significantly expand FLIM's scope. Firstly, a hardware-friendly pixel-wise DL algorithm is proposed for fast FLIM data analysis. The algorithm has a simple architecture yet can effectively resolve multi-exponential decay models. The calculation speed and accuracy outperform conventional methods significantly. Secondly, a DL algorithm is proposed to improve FLIM image spatial resolution, obtaining high-resolution (HR) fluorescence lifetime images from low-resolution (LR) images. A computational framework is developed to generate large-scale semi-synthetic FLIM datasets to address the challenge of the lack of sufficient high-quality FLIM datasets. This algorithm offers a practical approach to obtaining HR FLIM images quickly for FLIM systems. Thirdly, a DL algorithm is developed to analyze FLIM images with only a few photons per pixel, named Few-Photon Fluorescence Lifetime Imaging (FPFLI) algorithm. FPFLI uses spatial correlation and intensity information to robustly estimate the fluorescence lifetime images, pushing this photon budget to a record-low level of only a few photons per pixel. Finally, a time-resolved flow cytometry (TRFC) system is developed by integrating an advanced CMOS single-photon avalanche diode (SPAD) array and a DL processor. The SPAD array, using a parallel light detection scheme, shows an excellent photon-counting throughput. A quantized convolutional neural network (QCNN) algorithm is designed and implemented on a field-programmable gate array as an embedded processor. The processor resolves fluorescence lifetimes against disturbing noise, showing unparalleled high accuracy, fast analysis speed, and low power consumption.Fluorescence lifetime imaging microscopy (FLIM) has become a valuable tool in diverse disciplines. This thesis presents deep learning (DL) approaches to addressing two major challenges in FLIM: slow and complex data analysis and the high photon budget for precisely quantifying the fluorescence lifetimes. DL's ability to extract high-dimensional features from data has revolutionized optical and biomedical imaging analysis. This thesis contributes several novel DL FLIM algorithms that significantly expand FLIM's scope. Firstly, a hardware-friendly pixel-wise DL algorithm is proposed for fast FLIM data analysis. The algorithm has a simple architecture yet can effectively resolve multi-exponential decay models. The calculation speed and accuracy outperform conventional methods significantly. Secondly, a DL algorithm is proposed to improve FLIM image spatial resolution, obtaining high-resolution (HR) fluorescence lifetime images from low-resolution (LR) images. A computational framework is developed to generate large-scale semi-synthetic FLIM datasets to address the challenge of the lack of sufficient high-quality FLIM datasets. This algorithm offers a practical approach to obtaining HR FLIM images quickly for FLIM systems. Thirdly, a DL algorithm is developed to analyze FLIM images with only a few photons per pixel, named Few-Photon Fluorescence Lifetime Imaging (FPFLI) algorithm. FPFLI uses spatial correlation and intensity information to robustly estimate the fluorescence lifetime images, pushing this photon budget to a record-low level of only a few photons per pixel. Finally, a time-resolved flow cytometry (TRFC) system is developed by integrating an advanced CMOS single-photon avalanche diode (SPAD) array and a DL processor. The SPAD array, using a parallel light detection scheme, shows an excellent photon-counting throughput. A quantized convolutional neural network (QCNN) algorithm is designed and implemented on a field-programmable gate array as an embedded processor. The processor resolves fluorescence lifetimes against disturbing noise, showing unparalleled high accuracy, fast analysis speed, and low power consumption

Estimation of fluorescence lifetimes via rotational invariance techniques

Estimation of signal parameters via rotational invariance techniques is a classical algorithm widely used in array signal processing for direction-of-arrival estimation of emitters. Inspired by this method, a new signal model and a new fluorescence lifetime estimation via rotational invariance techniques (FLERIT) were developed for multi-exponential fluorescence lifetime imaging (FLIM) experiments. The FLERIT only requires a few time bins of a histogram generated by a time-correlated single photon counting FLIM system, greatly reducing the data throughput from the imager to the signal processing units. As a non-iterative method, the FLERIT does not require initial conditions, prior information nor model selection that are usually required by widely used traditional fitting methods, including nonlinear least square methods or maximum likelihood methods. Moreover, its simplicity means it is suitable for implementations in embedded systems for real-time applications. FLERIT was tested on synthesized and experimental fluorescent cell data showing the potentials to be widely applied in FLIM data analysis

Capillary and microchip gel electrophoresis using multiplexed fluorescence detection with both time-resolved and spectral-discrimination capabilities: applications in DNA sequencing using near-infrared fluorescence

Increasing the information content obtainable from a single assay and system miniaturization has continued to be important research areas in analytical chemistry. The research presented in this dissertation involves the development of a two-color, time-resolved fluorescence microscope for the acquisition of both steady-state and time-resolved data during capillary and microchip electrophoresis. The utility of this hybrid fluorescence detector has been demonstrated by applying it to DNA sequencing applications. Coupling color discrimination with time-resolved fluorescence offers increased multiplexing capabilities because the lifetime data adds another layer of information. An optical fiber-based fluorescence microscope was constructed, which utilized fluorescence in near-IR region, greatly simplifying the hardware and allowing superior system sensitivity. Time-resolved data was processed using electronics configured in a time-correlated single photon counting format. Cross-talk between color channels was successfully eliminated by utilizing the intrinsic time-resolved capability associated with the detector. The two-color, time-resolved microscope was first coupled to a single capillary and carried out two-color, two-lifetime sequencing of an M13 template, achieving a read length of 650 bps at a calling accuracy of 95.1%. The feasibility of using this microscope with microchips (glass-based chips) for sequencing was then demonstrated. Results from capillaries and microchips were compared, with the microchips providing faster analysis and adequate electrophoretic performance. Lifetimes of a set of fluorescent dyes were determined with favorable precision, in spite of the low loading levels associated with the microchips. The sequencing products were required to be purified and concentrated prior to electrophoretic sorting to improve data quality. PMMA-based microchips for DNA sequencing application were evaluated. The microchips were produced from thermo plastics, which allowed rapid and inexpensive production of microstructures with high aspect ratios. It was concluded that surface coating was needed on the polymer chips in order to achieve single-base resolution required for DNA sequencing. The capability of the two-color time-resolved microscope operated in a scanning mode was further explored. The successful construction of the scanner allows scanning of multi-channel microchips for high throughput processing

A versatile fluorescence lifetime imaging system for scanning large areas with high time and spatial resolution

"Published in SPIE Proceedings Vol. 9286"We present a flexible fluorescence lifetime imaging device which can be employed to scan large sample areas with a spatial resolution adjustable from many micrometers down to sub-micrometers and a temporal resolution of 20 picoseconds. Several different applications of the system will be presented including protein microarrays analysis, the scanning of historical samples, evaluation of solar cell surfaces and nanocrystalline organic crystals embedded in electrospun polymeric nanofibers. Energy transfer processes within semiconductor quantum dot superstructures as well as between dye probes and graphene layers were also investigated.This work was financially supported by the European Regional Development Fund (ERDF) through Programa Operacional Factores de Competitividade (COMPETE: FCOMP-01-0124-FEDER-014628) and the Portuguese Fundacao para a Ciencia e Tecnologia (FCT) through the projects "Functional structuring, inter-particle interaction and energy transfer in ensembles of nanocrystal dots" (PTDC/FIS/113199/2009), Ultra-fast spectroscopy on the dynamics and relaxation of Dirac electrons in graphene" (PTDC/FIS/101434/ 2008) and "Low dimensional nanostructures for nonlinear optical applications" PTDC/CTmNAN/114269/2009

Single biomarker screening using opto-electronic nanopores for diagnostics

The capability to selectively identify and quantify small nucleic acids or proteins in low abundance in the clinical sample has become a major driving force for the development of next-generation diagnostic strategies. To this end, nanopores have been a promising analytical tool for sensing biomolecules at the single-molecule level, directly in biofluids, without the need for sample processing. Nanopores, however, generally the lack of selectivity and accuracy to discriminate small molecules that have similar size but different structure and biological function, and this, in turn, has limited their applications in diagnostics. In this thesis, I present a novel opto-electronic strategy combining the electrical sensing modality of a nanopore with fluorescence-based detection to address these challenges. By engineering a molecular beacon (MB) attached to a DNA carrier, I was able to show that target molecules could be selectively and accurately identified through reading a synchronised opto-electrical signal originating from the carrier and bound target that was translocated through the nanopore. By quantifying the fraction of synchronisation, one can predict the target concentration as well as determine the target binding affinity without directly labelling of the target itself. The method is based on aptamer binding and is applicable for a wide panel of targets, including proteins with known aptamer recognition sequences or DNA and RNA with known sequences. It was also possible to discriminate the latter with single-nucleotide resolution. In addition, it was possible to detect more than one target simultaneously and increase the throughput by using DNA carriers of a specific length for each target. In this thesis, it is shown that multiple miRNAs (miR-141, miR-375) that have been implicated in prostate cancer can be quantified simultaneously with only one test, performed directly in the clinical sample. I demonstrate the ability of this strategy to distinguish single nucleotide polymorphism (SNP) within the same miRNA family. Notably, this strategy can simultaneously profile the small aberrations in multiple miRNA expression in patients with different stages of prostate cancer. Overall, the present findings in this thesis have pushed the nanopore sensing a step closer to the practical biomedical diagnosis and provide insights for developing next-generation clinical sample assays.Open Acces

Single-Molecule Detection of Unique Genome Signatures: Applications in Molecular Diagnostics and Homeland Security

Single-molecule detection (SMD) offers an attractive approach for identifying the presence of certain markers that can be used for in vitro molecular diagnostics in a near real-time format. The ability to eliminate sample processing steps afforded by the ultra-high sensitivity associated with SMD yields an increased sampling pipeline. When SMD and microfluidics are used in conjunction with nucleic acid-based assays such as the ligase detection reaction coupled with single-pair fluorescent resonance energy transfer (LDR-spFRET), complete molecular profiling and screening of certain cancers, pathogenic bacteria, and other biomarkers becomes possible at remarkable speeds and sensitivities with high specificity. The merging of these technologies and techniques into two different novel instrument formats has been investigated. (1) The use of a charge-coupled device (CCD) in time-delayed integration (TDI) mode as a means for increasing the throughput of any single molecule measurement by simultaneously tracking and detecting single-molecules in multiple microfluidic channels was demonstrated. The CCD/TDI approach allowed increasing the sample throughput by a factor of 8 compared to a single-assay SMD experiment. A sampling throughput of 276 molecules s-1 per channel and 2208 molecules s-1 for an eight channel microfluidic system was achieved. A cyclic olefin copolymer (COC) waveguide was designed and fabricated in a pre-cast poly(dimethylsiloxane) stencil to increase the SNR by controlling the excitation geometry. The waveguide showed an attenuation of 0.67 dB/cm and the launch angle was optimized to increase the depth of penetration of the evanescent wave. (2) A compact SMD (cSMD) instrument was designed and built for the reporting of molecular signatures associated with bacteria. The optical waveguides were poised within the fluidic chip at orientation of 90° with respect to each other for the interrogation of single-molecule events. Molecular beacons (MB) were designed to probe bacteria for the classification of Gram +. MBs were mixed with bacterial cells and pumped though the cSMD which allowed S. aureus to be classified with 2,000 cells in 1 min. Finally, the integration of the LDR-spFRET assay on the cSMD was explored with the future direction of designing a molecular screening approach for stroke diagnostics

The potential of optical proteomic technologies to individualize prognosis and guide rational treatment for cancer patients

Genomics and proteomics will improve outcome prediction in cancer and have great potential to help in the discovery of unknown mechanisms of metastasis, ripe for therapeutic exploitation. Current methods of prognosis estimation rely on clinical data, anatomical staging and histopathological features. It is hoped that translational genomic and proteomic research will discriminate more accurately than is possible at present between patients with a good prognosis and those who carry a high risk of recurrence. Rational treatments, targeted to the specific molecular pathways of an individual’s high-risk tumor, are at the core of tailored therapy. The aim of targeted oncology is to select the right patient for the right drug at precisely the right point in their cancer journey. Optical proteomics uses advanced optical imaging technologies to quantify the activity states of and associations between signaling proteins by measuring energy transfer between fluorophores attached to specific proteins. Förster resonance energy transfer (FRET) and fluorescence lifetime imaging microscopy (FLIM) assays are suitable for use in cell line models of cancer, fresh human tissues and formalin-fixed paraffin-embedded tissue (FFPE). In animal models, dynamic deep tissue FLIM/FRET imaging of cancer cells in vivo is now also feasible. Analysis of protein expression and post-translational modifications such as phosphorylation and ubiquitination can be performed in cell lines and are remarkably efficiently in cancer tissue samples using tissue microarrays (TMAs). FRET assays can be performed to quantify protein-protein interactions within FFPE tissue, far beyond the spatial resolution conventionally associated with light or confocal laser microscopy. Multivariate optical parameters can be correlated with disease relapse for individual patients. FRET-FLIM assays allow rapid screening of target modifiers using high content drug screens. Specific protein-protein interactions conferring a poor prognosis identified by high content tissue screening will be perturbed with targeted therapeutics. Future targeted drugs will be identified using high content/throughput drug screens that are based on multivariate proteomic assays. Response to therapy at a molecular level can be monitored using these assays while the patient receives treatment: utilizing re-biopsy tumor tissue samples in the neoadjuvant setting or by examining surrogate tissues. These technologies will prove to be both prognostic of risk for individuals when applied to tumor tissue at first diagnosis and predictive of response to specifically selected targeted anticancer drugs. Advanced optical assays have great potential to be translated into real-life benefit for cancer patients