Neuronal imaging with ultrahigh dynamic range multiphoton microscopy

Multiphoton microscopes are hampered by limited dynamic range, preventing weak sample features from being detected in the presence of strong features, or preventing the capture of unpredictable bursts in sample strength. We present a digital electronic add-on technique that vastly improves the dynamic range of a multiphoton microscope while limiting potential photodamage. The add-on provides real-time negative feedback to regulate the laser power delivered to the sample, and a log representation of the sample strength to accommodate ultrahigh dynamic range without loss of information. No microscope hardware modifications are required, making the technique readily compatible with commercial instruments. Benefits are shown in both structural and in-vivo functional mouse brain imaging applications.R21 EY027549 - NEI NIH HH

Set up of a light sheet fluorescence microscope for cellular studies

Light-sheet fluorescence microscopy (LSFM) has been present in cell biology laboratories for quite some time, mainly as custom-made systems, with imaging applications ranging from single cells (in the μm scale) to small organisms (mm). Such microscopes distinguish themselves for having very low phototoxicity levels and high spatial and temporal resolution, properties that render it ideal for 3D characterization of cell motility in migration and traction force studies. Cellular motion has proven to be essential in biological processes such as tumor metastasis and tissue development. Experimental setups make extensive use of microdevices (bioMEMS) that are providing higher degrees of empirical complexity. The following report details the process of setting-up a functional LSFM device for imaging cell motion in microfluidic devices. It begins with a brief summary of fluorescence imaging and current techniques, important to understand why single-plane illumination microscopy (SPIM) was chosen among other light-sheet methods. Then, the whole SPIM set-up process is described, containing explanations about the physical and material properties of the hardware used, the intricacies of the control system, and important procedures. These procedures include: calibration of the microscope, sample preparation in microdevices, and image acquisition from the software provided. Real fluorescence images acquired serve as evidence of the functionality of the instrument. The current limitations are highlighted, and pointers on how to improve or enhance the device are given. The report contains many diagrams, tables and pictures to aid in the understanding of important concepts. In the Annex, a comprehensive table listing the project costs by category is attached. This table includes links to the manufacturers and providers. The aim of this writing is to serve as an exhaustive guideline and be of reproducible use for researchers aiming to build SPIM systems for similar applications.Ingeniería Biomédic

Computational Framework For Neuro-Optics Simulation And Deep Learning Denoising

The application of machine learning techniques in microscopic image restoration has shown superior performance. However, the development of such techniques has been hindered by the demand for large datasets and the lack of ground truth. To address these challenges, this study introduces a computer simulation model that accurately captures the neural anatomic volume, fluorescence light transportation within the tissue volume, and the photon collection process of microscopic imaging sensors. The primary goal of this simulation is to generate realistic image data for training and validating machine learning models. One notable aspect of this study is the incorporation of a machine learning denoiser into the simulation, which accelerates the computational efficiency of the entire process. By reducing noise levels in the generated images, the denoiser significantly enhances the simulation\u27s performance, allowing for faster and more accurate modeling and analysis of microscopy images. This approach addresses the limitations of data availability and ground truth annotation, offering a practical and efficient solution for microscopic image restoration. The integration of a machine learning denoiser within the simulation significantly accelerates the overall simulation process, while improving the quality of the generated images. This advancement opens new possibilities for training and validating machine learning models in microscopic image restoration, overcoming the challenges of large datasets and the lack of ground truth

Phase imaging by spatial wavefront sampling

Phase-imaging techniques extract the optical path length information of a scene, whereas wavefront sensors provide the shape of an optical wavefront. Since these two applications have different technical requirements, they have developed their own specific technologies. Here we show how to perform phase imaging combining wavefront sampling using a reconfigurable spatial light modulator with a beam position detector. The result is a time-multiplexed detection scheme, capable of being shortened considerably by compressive sensing. This robust referenceless method does not require the phase-unwrapping algorithms demanded by conventional interferometry, and its lenslet-free nature removes trade-offs usually found in Shack–Hartmann sensors

Fast fluorescence lifetime imaging and sensing via deep learning

Error on title page – year of award is 2023.Fluorescence lifetime imaging microscopy (FLIM) has become a valuable tool in diverse disciplines. This thesis presents deep learning (DL) approaches to addressing two major challenges in FLIM: slow and complex data analysis and the high photon budget for precisely quantifying the fluorescence lifetimes. DL's ability to extract high-dimensional features from data has revolutionized optical and biomedical imaging analysis. This thesis contributes several novel DL FLIM algorithms that significantly expand FLIM's scope. Firstly, a hardware-friendly pixel-wise DL algorithm is proposed for fast FLIM data analysis. The algorithm has a simple architecture yet can effectively resolve multi-exponential decay models. The calculation speed and accuracy outperform conventional methods significantly. Secondly, a DL algorithm is proposed to improve FLIM image spatial resolution, obtaining high-resolution (HR) fluorescence lifetime images from low-resolution (LR) images. A computational framework is developed to generate large-scale semi-synthetic FLIM datasets to address the challenge of the lack of sufficient high-quality FLIM datasets. This algorithm offers a practical approach to obtaining HR FLIM images quickly for FLIM systems. Thirdly, a DL algorithm is developed to analyze FLIM images with only a few photons per pixel, named Few-Photon Fluorescence Lifetime Imaging (FPFLI) algorithm. FPFLI uses spatial correlation and intensity information to robustly estimate the fluorescence lifetime images, pushing this photon budget to a record-low level of only a few photons per pixel. Finally, a time-resolved flow cytometry (TRFC) system is developed by integrating an advanced CMOS single-photon avalanche diode (SPAD) array and a DL processor. The SPAD array, using a parallel light detection scheme, shows an excellent photon-counting throughput. A quantized convolutional neural network (QCNN) algorithm is designed and implemented on a field-programmable gate array as an embedded processor. The processor resolves fluorescence lifetimes against disturbing noise, showing unparalleled high accuracy, fast analysis speed, and low power consumption.Fluorescence lifetime imaging microscopy (FLIM) has become a valuable tool in diverse disciplines. This thesis presents deep learning (DL) approaches to addressing two major challenges in FLIM: slow and complex data analysis and the high photon budget for precisely quantifying the fluorescence lifetimes. DL's ability to extract high-dimensional features from data has revolutionized optical and biomedical imaging analysis. This thesis contributes several novel DL FLIM algorithms that significantly expand FLIM's scope. Firstly, a hardware-friendly pixel-wise DL algorithm is proposed for fast FLIM data analysis. The algorithm has a simple architecture yet can effectively resolve multi-exponential decay models. The calculation speed and accuracy outperform conventional methods significantly. Secondly, a DL algorithm is proposed to improve FLIM image spatial resolution, obtaining high-resolution (HR) fluorescence lifetime images from low-resolution (LR) images. A computational framework is developed to generate large-scale semi-synthetic FLIM datasets to address the challenge of the lack of sufficient high-quality FLIM datasets. This algorithm offers a practical approach to obtaining HR FLIM images quickly for FLIM systems. Thirdly, a DL algorithm is developed to analyze FLIM images with only a few photons per pixel, named Few-Photon Fluorescence Lifetime Imaging (FPFLI) algorithm. FPFLI uses spatial correlation and intensity information to robustly estimate the fluorescence lifetime images, pushing this photon budget to a record-low level of only a few photons per pixel. Finally, a time-resolved flow cytometry (TRFC) system is developed by integrating an advanced CMOS single-photon avalanche diode (SPAD) array and a DL processor. The SPAD array, using a parallel light detection scheme, shows an excellent photon-counting throughput. A quantized convolutional neural network (QCNN) algorithm is designed and implemented on a field-programmable gate array as an embedded processor. The processor resolves fluorescence lifetimes against disturbing noise, showing unparalleled high accuracy, fast analysis speed, and low power consumption

Laser Scanning Microscopy with SPAD Array Detector: Towards a New Class of Fluorescence Microscopy Techniques

Laser scanning microscopy is one of the most common architectures in fluorescence microscopy. In a nutshell: the objective lens focuses the laser beam(s) and generates an effective excitation spot which is scanned on the sample; for each pixel, the fluorescent image is projected into a single-element detector, which \u2013 typically \u2013 spatially and temporally integrates the fluorescent light along its sensitive area and the pixel dwell-time, thus providing a single-intensity value per pixel. Notably, the integration performed by the single-element detector hinders any additional information potentially encoded in the dynamic and image of the fluorescent spot. To address this limitation, we recently upgraded the detection unit of a laser scanning microscope, replacing the single-element detector with a novel SPAD (single photon avalanche diode) array detector. We have shown at first that the additional spatial information allows to overcome the trade-off between resolution and signal-to-noise ratio proper of confocal microscopy: indeed, this architecture represents the natural implementation of image scanning microscopy (ISM). We then exploited the single-photon-timing ability of the SPAD array detector elements to combine ISM with fluorescence lifetime imaging: the results show higher resolution and better lifetime accuracy with respect to the confocal counterpart. Moreover, we explored the combination of our ISM platform with stimulated emission depletion (STED) microscopy, to mitigate the non-negligible chance of photo-damaging a sample. Lastly, we showed how the SPAD array-based microscope can be used in the context of single-molecule/particle tracking (SMT or SPT) and spectroscopy. Indeed, we implemented a real-time, feedback based SMT architecture which can potentially correlate the dynamics of a bio-molecule with its structural changes and micro-environment, taking advantage of the time-resolved spectroscopy ability of the novel detector. We believe that this novel laser scanning microscopy architecture has everything in its favour to substitute current single-element detector approaches; it will enable for a new class of fluorescence microscopy techniques capable of investigating complex living biological samples with unprecedented spatial and temporal characteristics and augmented information content