Holistic improvement of image acquisition and reconstruction in fluorescence microscopy

Recent developments in microscopic imaging led to a better understanding of intra- and intercellular metabolic processes and, for example, to visualize structural properties of viral pathogens. In this thesis, the imaging process of widefield and confocal scanning microscopy techniques is treated holistically to highlight general strategies and maximise their information content. Poisson or shot noise is assumed to be the fundamental noise process for the given measurements. A stable focus position is a basic condition for e.g. long-term measurements in order to provide reliable information about potential changes inside the Field-of-view. While newer microscopy systems can be equipped with hardware autofocus, this is not yet the widespread standard. For image-based focus analysis, different metrics for ideal, noisy and aberrated, in case of spherical aberration and astigmatism, measurements are presented. A stable focus position is also relevant in the example of 2-photon confocal imaging and at the same time the situation is aggravated in the given example, the measurement of the retina in the living mouse. In addition to the natural drift of the focal position, which can be evaluated by means of previously introduced metrics, rhythmic heartbeat, respiration, unrhythmic muscle twitching and movement of the mouse kept in artificial sleep are added. A dejittering algorithm is presented for the measurement data obtained under these circumstances. Using the additional information about the sample distribution in ISM, a method for reconstructing 3D from 2D image data is presented in the form of thick slice unmixing. This method can further be used for suppression of light generated outside the focal layer of 3D data stacks and is compared to selective layer multi-view deconvolution. To reduce phototoxicity and save valuable measurement time for a 3D stack, the method of zLEAP is presented, by which omitted Z-planes are subsequently calculated and inserted

Hyperspectral benthic mapping from underwater robotic platforms

We live on a planet of vast oceans; 70% of the Earth's surface is covered in water. They are integral to supporting life, providing 99% of the inhabitable space on Earth. Our oceans and the habitats within them are under threat due to a variety of factors. To understand the impacts and possible solutions, the monitoring of marine habitats is critically important. Optical imaging as a method for monitoring can provide a vast array of information however imaging through water is complex. To compensate for the selective attenuation of light in water, this thesis presents a novel light propagation model and illustrates how it can improve optical imaging performance. An in-situ hyperspectral system is designed which comprised of two upward looking spectrometers at different positions in the water column. The downwelling light in the water column is continuously sampled by the system which allows for the generation of a dynamic water model. In addition to the two upward looking spectrometers the in-situ system contains an imaging module which can be used for imaging of the seafloor. It consists of a hyperspectral sensor and a trichromatic stereo camera. New calibration methods are presented for the spatial and spectral co-registration of the two optical sensors. The water model is used to create image data which is invariant to the changing optical properties of the water and changing environmental conditions. In this thesis the in-situ optical system is mounted onboard an Autonomous Underwater Vehicle. Data from the imaging module is also used to classify seafloor materials. The classified seafloor patches are integrated into a high resolution 3D benthic map of the surveyed site. Given the limited imaging resolution of the hyperspectral sensor used in this work, a new method is also presented that uses information from the co-registered colour images to inform a new spectral unmixing method to resolve subpixel materials

Creating a platform for the democratisation of Deep Learning in microscopy

One of the major technological success stories of the last decade has been the advent of deep learning (DL), which has touched almost every aspect of modern life after a breakthrough performance in an image detection challenge in 2012. The bioimaging community quickly recognised the prospect of the automated ability to make sense of image data with near-human performance as potentially ground-breaking. In the decade since, hundreds of publications have used this technology to tackle many problems related to image analysis, such as labelling or counting cells, identifying cells or organelles of interest in large image datasets, or removing noise or improving the resolution of images. However, the adoption of DL tools in large parts of the bioimaging community has been slow, and many tools have remained in the hands of developers. In this project, I have identified key barriers which have prevented many bioimage analysts and microscopists from accessing existing DL technology in their field and have, in collaboration with colleagues, developed the ZeroCostDL4Mic platform, which aims to address these barriers. This project is inspired by the observation that the most significant impact technology can have in science is when it becomes ubiquitous, that is, when its use becomes essential to address the community’s questions. This work represents one of the first attempts to make DL tools accessible in a transparent, code-free, and affordable manner for bioimage analysis to unlock the full potential of DL via its democratisation for the bioimaging community

STED Nanoscopy to Illuminate New Avenues in Cancer Research – From Live Cell Staining and Direct Imaging to Decisive Preclinical Insights for Diagnosis and Therapy

Molecular imaging is established as an indispensable tool in various areas of cancer research, ranging from basic cancer biology and preclinical research to clinical trials and medical practice. In particular, the field of fluorescence imaging has experienced exceptional progress during the last three decades with the development of various in vivo technologies. Within this field, fluorescence microscopy is primarily of experimental use since it is especially qualified for addressing the fundamental questions of molecular oncology. As stimulated emission depletion (STED) nanoscopy combines the highest spatial and temporal resolutions with live specimen compatibility, it is best-suited for real-time investigations of the differences in the molecular machineries of malignant and normal cells to eventually translate the acquired knowledge into increased diagnostic and therapeutic efficacy. This thesis presents the application of STED nanoscopy to two acute topics in cancer research of direct or indirect clinical interest. The first project has investigated the structure of telomeres, the ends of the linear eukaryotic chromosomes, in intact human cells at the nanoscale. To protect genome integrity, a telomere can mask the chromosome end by folding back and sequestering its single-stranded 3’-overhang in an upstream part of the double-stranded DNA repeat region. The formed t-loop structure has so far only been visualized by electron microscopy and fluorescence nanoscopy with cross-linked mammalian telomeric DNA after disruption of cell nuclei and spreading. For the first time, this work demonstrates the existence of t-loops within their endogenous nuclear environment in intact human cells. The identification of further telomere conformations has laid the groundwork for distinguishing cancerous cells that use different telomere maintenance mechanisms based on their individual telomere populations by a combined STED nanoscopy and deep learning approach. The population difference was essentially attributed to the promyelocytic leukemia (PML) protein that significantly perturbs the organization of a subpopulation of telomeres towards an open conformation in cancer cells that employ a telomerase-independent, alternative telomere lengthening mechanism. Elucidating the nanoscale topology of telomeres and associated proteins within the nucleus has provided new insight into telomere structure-function relationships relevant for understanding the deregulation of telomere maintenance in cancer cells. After understanding the molecular foundations, this newly gained knowledge can be exploited to develop novel or refined diagnostic and treatment strategies. The second project has characterized the intracellular distribution of recently developed prostate cancer tracers. These novel prostate-specific membrane antigen (PSMA) inhibitors have revolutionized the treatment regimen of prostate cancer by enabling targeted imaging and therapy approaches. However, the exact internalization mechanism and the subcellular fate of these tracers have remained elusive. By combining STED nanoscopy with a newly developed non-standard live cell staining protocol, this work confirmed cell surface clustering of the targeted membrane antigen upon PSMA inhibitor binding, subsequent clathrin-dependent endocytosis and endosomal trafficking of the antigen-inhibitor complex. PSMA inhibitors accumulate in prostate cancer cells at clinically relevant time points, but strikingly and in contrast to the targeted antigen itself, they eventually distribute homogenously in the cytosol. This project has revealed the subcellular fate of PSMA/PSMA inhibitor complexes for the first time and provides crucial knowledge for the future application of these tracers including the development of new strategies in the field of prostate cancer diagnostics and therapeutics. Relying on the photostability and biocompatibility of the applied fluorophores, the performance of live cell STED nanoscopy in the field of cancer research is boosted by the development of improved fluorophores. The third project in this thesis introduces a biocompatible, small molecule near-infrared dye suitable for live cell STED imaging. By the application of a halogen dance rearrangement, a dihalogenated fluorinatable pyridinyl rhodamine could be synthesized at high yield. The option of subsequent radiolabeling combined with excellent optical properties and a non-toxic profile renders this dye an appropriate candidate for medical and bioimaging applications. Providing an intrinsic and highly specific mitochondrial targeting ability, the radiolabeled analogue is suggested as a vehicle for multimodal (positron emission tomography and optical imaging) medical imaging of mitochondria for cancer diagnosis and therapeutic approaches in patients and biopsy tissue. The absence of cytotoxicity is not only a crucial prerequisite for clinically used fluorophores. To guarantee the generation of meaningful data mirroring biological reality, the absence of cytotoxicity is likewise a decisive property of dyes applied in live cell STED nanoscopy. The fourth project in this thesis proposes a universal approach for cytotoxicity testing based on characterizing the influence of the compound of interest on the proliferation behavior of human cell lines using digital holographic cytometry. By applying this approach to recently developed live cell STED compatible dyes, pronounced cytotoxic effects could be excluded. Looking more closely, some of the tested dyes slightly altered cell proliferation, so this project provides guidance on the right choice of dye for the least invasive live cell STED experiments. Ultimately, live cell STED data should be exploited to extract as much biological information as possible. However, some information might be partially hidden by image degradation due the dynamics of living samples and the deliberate choice of rather conservative imaging parameters in order to preserve sample viability. The fifth project in this thesis presents a novel image restoration method in a Bayesian framework that simultaneously performs deconvolution, denoising as well as super-resolution, to restore images suffering from noise with mixed Poisson-Gaussian statistics. Established deconvolution or denoising methods that consider only one type of noise generally do not perform well on images degraded significantly by mixed noise. The newly introduced method was validated with live cell STED telomere data proving that the method can compete with state-of-the-art approaches. Taken together, this thesis demonstrates the value of an integrated approach for STED nanoscopy imaging studies. A coordinated workflow including sample preparation, image acquisition and data analysis provided a reliable platform for deriving meaningful conclusions for current questions in the field of cancer research. Moreover, this thesis emphasizes the strength of iteratively adapting the individual components in the operational chain and it particularly points towards those components that, if further improved, optimize the significance of the final results rendering live cell STED nanoscopy even more powerful

Algorithms for Fluorescence Lifetime Microscopy and Optical Coherence Tomography Data Analysis: Applications for Diagnosis of Atherosclerosis and Oral Cancer

With significant progress made in the design and instrumentation of optical imaging systems, it is now possible to perform high-resolution tissue imaging in near real-time. The prohibitively large amount of data obtained from such high-speed imaging systems precludes the possibility of manual data analysis by an expert. The paucity of algorithms for automated data analysis has been a major roadblock in both evaluating and harnessing the full potential of optical imaging modalities for diagnostic applications. This consideration forms the central theme of the research presented in this dissertation. Specifically, we investigate the potential of automated analysis of data acquired from a multimodal imaging system that combines fluorescence lifetime imaging (FLIM) with optical coherence tomography (OCT), for the diagnosis of atherosclerosis and oral cancer. FLIM is a fluorescence imaging technique that is capable of providing information about auto fluorescent tissue biomolecules. OCT on the other hand, is a structural imaging modality that exploits the intrinsic reflectivity of tissue samples to provide high resolution 3-D tomographic images. Since FLIM and OCT provide complimentary information about tissue biochemistry and structure, respectively, we hypothesize that the combined information from the multimodal system would increase the sensitivity and specificity for the diagnosis of atherosclerosis and oral cancer. The research presented in this dissertation can be divided into two main parts. The first part concerns the development and applications of algorithms for providing quantitative description of FLIM and OCT images. The quantitative FLIM and OCT features obtained in the first part of the research, are subsequently used to perform automated tissue diagnosis based on statistical classification models. The results of the research presented in this dissertation show the feasibility of using automated algorithms for FLIM and OCT data analysis for performing tissue diagnosis

Quantitative photoacoustic tomography: experimental phantom studies

Photoacoustic tomography (PAT) is a promising non-invasive imaging modality exhibiting high resolution, good contrast and specificity to light-absorbing molecules (chromophores). One of the outstanding challenges the technique faces is that PAT images, though dependent on optical absorption, are not its direct representation because they are coloured by the unknown light fluence. Theoretical studies have succeeded in quantifying optical absorption and chromophore concentration by employing model-based inversions (MBI) that can deal with the non-linearity of the problem and the fluence-related distortion. However, experimental translation has been scarce. The aim was to perform quantitative PAT (qPAT) in a rigorous experimental phantom study to show that highly-resolved 3D estimation of chromophore distributions can be achieved. The first consideration was finding a tissue-relevant and stable matrix material and chromophores. Thermoplastic PVCP was fully assessed. Its stability, intrinsic optical properties, thermoelastic efficiency and low-frequency acoustic properties were suitable. The limitation was the lack of photostability of embedded pigments. Separately, we fully characterised aqueous solutions of sulphate salts and found them to be suitable chromophores for qPAT and potential surrogates for oxy- and deoxyhemoglobin. For a phantom made of sub-mm tubes filled with sulphate solutions in an intralipid-rich background, 3D high resolution estimates of chromophore concentrations were obtained through an efficient diffusion-approximation MBI. Uncertainties in optical inputs of the MBI were tackled by assessing in silico their effect on quantification accuracy and then mitigated in the designed experiment through careful measurements. A faithful representation of the multiwavelength photoacoustic tomography images was sought by employing broadband, near-omnidirectional and high-sensitivity sensors and a detection configuration and reconstruction that overcame the limited-view problem. Estimation of the chromophore ratio, analogous to the much sought-after blood oxygenation, gave a mean absolute error of 3.4 p.p., whilst normalised estimates of the two main chromophore distributions gave errors of 13.2% and 17.2%

Imaging studies of peripheral nerve regeneration induced by porous collagen biomaterials

Thesis: Ph. D., Massachusetts Institute of Technology, Department of Mechanical Engineering, 2013.Cataloged from PDF version of thesis.Includes bibliographical references.There is urgent need to develop treatments for inducing regeneration in injured organs. Porous collagen-based scaffolds have been utilized clinically to induce regeneration in skin and peripheral nerves, however still there is no complete explanation about the underlying mechanism. This thesis utilizes advanced microscopy to study the expression of contractile cell phenotypes during wound healing, a phenotype believed to affect significantly the final outcome. The first part develops an efficient pipeline for processing challenging spectral fluorescence microscopy images. Images are segmented into regions of objects by refining the outcome of a pixel-wide model selection classifier by an efficient Markov Random Field model. The methods of this part are utilized by the following parts. The second part extends the image informatics methodology in studying signal transduction networks in cells interacting with 3D matrices. The methodology is applied in a pilot study of TGFP signal transduction by the SMAD pathway in fibroblasts seeded in porous collagen scaffolds. Preliminary analysis suggests that the differential effect of TGFP1 and TGFP3 to cells could be attributed to the "non-canonical" SMADI and SMAD5. The third part is an ex vivo imaging study of peripheral nerve regeneration, which focuses on the formation of a capsule of contractile cells around transected rat sciatic nerves grafted with collagen scaffolds, 1 or 2 weeks post-injury. It follows a recent study that highlights an inverse relationship between the quality of the newly formed nerve tissue and the size of the contractile cell capsule 9 weeks post-injury. Results suggest that "active" biomaterials result in significantly thinner capsule already 1 week post-injury. The fourth part describes a novel method for quantifying the surface chemistry of 3D matrices. The method is an in situ binding assay that utilizes fluorescently labeled recombinant proteins that emulate the receptor of , and is applied to quantify the density of ligands for integrins a113, a2p1 on the surface of porous collagen scaffolds. Results provide estimates for the density of ligands on "active" and "inactive" scaffolds and demonstrate that chemical crosslinking can affect the surface chemistry of biomaterials, therefore can affect the way cells sense and respond to the material.by Dimitrios S. Tzeranis.Ph. D

Application of sound source separation methods to advanced spatial audio systems

This thesis is related to the field of Sound Source Separation (SSS). It addresses the development and evaluation of these techniques for their application in the resynthesis of high-realism sound scenes by means of Wave Field Synthesis (WFS). Because the vast majority of audio recordings are preserved in twochannel stereo format, special up-converters are required to use advanced spatial audio reproduction formats, such as WFS. This is due to the fact that WFS needs the original source signals to be available, in order to accurately synthesize the acoustic field inside an extended listening area. Thus, an object-based mixing is required. Source separation problems in digital signal processing are those in which several signals have been mixed together and the objective is to find out what the original signals were. Therefore, SSS algorithms can be applied to existing two-channel mixtures to extract the different objects that compose the stereo scene. Unfortunately, most stereo mixtures are underdetermined, i.e., there are more sound sources than audio channels. This condition makes the SSS problem especially difficult and stronger assumptions have to be taken, often related to the sparsity of the sources under some signal transformation. This thesis is focused on the application of SSS techniques to the spatial sound reproduction field. As a result, its contributions can be categorized within these two areas. First, two underdetermined SSS methods are proposed to deal efficiently with the separation of stereo sound mixtures. These techniques are based on a multi-level thresholding segmentation approach, which enables to perform a fast and unsupervised separation of sound sources in the time-frequency domain. Although both techniques rely on the same clustering type, the features considered by each of them are related to different localization cues that enable to perform separation of either instantaneous or real mixtures.Additionally, two post-processing techniques aimed at improving the isolation of the separated sources are proposed. The performance achieved by several SSS methods in the resynthesis of WFS sound scenes is afterwards evaluated by means of listening tests, paying special attention to the change observed in the perceived spatial attributes. Although the estimated sources are distorted versions of the original ones, the masking effects involved in their spatial remixing make artifacts less perceptible, which improves the overall assessed quality. Finally, some novel developments related to the application of time-frequency processing to source localization and enhanced sound reproduction are presented.Cobos Serrano, M. (2009). Application of sound source separation methods to advanced spatial audio systems [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/8969Palanci

Mapping the Dynamic Protein Network of Dividing Cells in Space and Time

Live cell imaging is a powerful tool for studying the distribution and dynamics of proteins. However, due to the difficulties in absolute quantification and standardization of data obtained from individual cells, it has not been used to map large sets of proteins that carry out dynamic cellular functions. Cell division is a good example of this challenge for an essential cellular function, as rapid changes in protein localization and protein interactions result in dramatic changes to subcellular structures and cellular morphology, which in turn influence the behavior of the enclosed proteins. Here, I report an integrated experimental and computational pipeline to map the dynamic protein network of dividing human cells in space and time. Using 3D live confocal microscopy, I imaged human cell lines that stably expressed fluorescently tagged mitotic proteins throughout mitosis. To obtain the absolute quantities of protein abundance with high subcellular resolution over time, the microscopy pipeline was calibrated by fluorescence correlation spectroscopy (FCS). Cell and chromosome volumes were segmented as references of cellular context for temporal and spatial alignment based on fluorescent landmarks. Together with my colleague Julius Hossain, we computationally generated a canonical model of mitotic progression for both kinetics (“mitotic standard time”) and morphology (“mitotic standard space”) by averaging and kinetically and geometrically parametrizing many registered dividing cells. The resulting model enabled us to subdivide the mitotic process into 20 characteristic kinetic steps and integrate our complete proof of concept dataset of 13 mitotic proteins imaged in over 300 dividing cells, represented as the 3D protein localization probability of each protein over time. To measure localization similarities between different proteins and make predictions about their dynamic interactions, the integrated data was then mined using supervised as well as unsupervised machine learning. The power of this approach was demonstrated by our ability to automatically identify the major subcellular localizations of all proteins in the dataset and quantify protein fluxes between subcellular compartments and structures. Due to the quantitative nature of our imaging data, we were able to estimate the abundance of each protein in mitotic structures and complexes such as kinetochores, centrosomes, and the midbody, and determine the order and kinetics of their formation and disassembly. The integrated computational and experimental method I present in my thesis is generic and scalable and makes many dynamic cellular processes amenable to dynamic protein network analysis even for large numbers of components. The pipeline provides a powerful instrument for analyzing large sets of quantitative live imaging data of fluorescently tagged proteins. It allows the systematic mapping and prediction of dynamic protein networks that drive complex cellular processes such as mitosis, thus promoting our understanding of the mechanisms by which many molecules together achieve spatio-temporal regulation