Multi-modal and multi-dimensional biomedical image data analysis using deep learning

There is a growing need for the development of computational methods and tools for automated, objective, and quantitative analysis of biomedical signal and image data to facilitate disease and treatment monitoring, early diagnosis, and scientific discovery. Recent advances in artificial intelligence and machine learning, particularly in deep learning, have revolutionized computer vision and image analysis for many application areas. While processing of non-biomedical signal, image, and video data using deep learning methods has been very successful, high-stakes biomedical applications present unique challenges such as different image modalities, limited training data, need for explainability and interpretability etc. that need to be addressed. In this dissertation, we developed novel, explainable, and attention-based deep learning frameworks for objective, automated, and quantitative analysis of biomedical signal, image, and video data. The proposed solutions involve multi-scale signal analysis for oraldiadochokinesis studies; ensemble of deep learning cascades using global soft attention mechanisms for segmentation of meningeal vascular networks in confocal microscopy; spatial attention and spatio-temporal data fusion for detection of rare and short-term video events in laryngeal endoscopy videos; and a novel discrete Fourier transform driven class activation map for explainable-AI and weakly-supervised object localization and segmentation for detailed vocal fold motion analysis using laryngeal endoscopy videos. Experiments conducted on the proposed methods showed robust and promising results towards automated, objective, and quantitative analysis of biomedical data, that is of great value for potential early diagnosis and effective disease progress or treatment monitoring.Includes bibliographical references

New algorithms for the analysis of live-cell images acquired in phase contrast microscopy

La détection et la caractérisation automatisée des cellules constituent un enjeu important dans de nombreux domaines de recherche tels que la cicatrisation, le développement de l'embryon et des cellules souches, l’immunologie, l’oncologie, l'ingénierie tissulaire et la découverte de nouveaux médicaments. Étudier le comportement cellulaire in vitro par imagerie des cellules vivantes et par le criblage à haut débit implique des milliers d'images et de vastes quantités de données. Des outils d'analyse automatisés reposant sur la vision numérique et les méthodes non-intrusives telles que la microscopie à contraste de phase (PCM) sont nécessaires. Comme les images PCM sont difficiles à analyser en raison du halo lumineux entourant les cellules et de la difficulté à distinguer les cellules individuelles, le but de ce projet était de développer des algorithmes de traitement d'image PCM dans Matlab® afin d’en tirer de l’information reliée à la morphologie cellulaire de manière automatisée. Pour développer ces algorithmes, des séries d’images de myoblastes acquises en PCM ont été générées, en faisant croître les cellules dans un milieu avec sérum bovin (SSM) ou dans un milieu sans sérum (SFM) sur plusieurs passages. La surface recouverte par les cellules a été estimée en utilisant un filtre de plage de valeurs, un seuil et une taille minimale de coupe afin d'examiner la cinétique de croissance cellulaire. Les résultats ont montré que les cellules avaient des taux de croissance similaires pour les deux milieux de culture, mais que celui-ci diminue de façon linéaire avec le nombre de passages. La méthode de transformée par ondelette continue combinée à l’analyse d'image multivariée (UWT-MIA) a été élaborée afin d’estimer la distribution de caractéristiques morphologiques des cellules (axe majeur, axe mineur, orientation et rondeur). Une analyse multivariée réalisée sur l’ensemble de la base de données (environ 1 million d’images PCM) a montré d'une manière quantitative que les myoblastes cultivés dans le milieu SFM étaient plus allongés et plus petits que ceux cultivés dans le milieu SSM. Les algorithmes développés grâce à ce projet pourraient être utilisés sur d'autres phénotypes cellulaires pour des applications de criblage à haut débit et de contrôle de cultures cellulaires.Automated cell detection and characterization is important in many research fields such as wound healing, embryo development, immune system studies, cancer research, parasite spreading, tissue engineering, stem cell research and drug research and testing. Studying in vitro cellular behavior via live-cell imaging and high-throughput screening involves thousands of images and vast amounts of data, and automated analysis tools relying on machine vision methods and non-intrusive methods such as phase contrast microscopy (PCM) are a necessity. However, there are still some challenges to overcome, since PCM images are difficult to analyze because of the bright halo surrounding the cells and blurry cell-cell boundaries when they are touching. The goal of this project was to develop image processing algorithms to analyze PCM images in an automated fashion, capable of processing large datasets of images to extract information related to cellular viability and morphology. To develop these algorithms, a large dataset of myoblasts images acquired in live-cell imaging (in PCM) was created, growing the cells in either a serum-supplemented (SSM) or a serum-free (SFM) medium over several passages. As a result, algorithms capable of computing the cell-covered surface and cellular morphological features were programmed in Matlab®. The cell-covered surface was estimated using a range filter, a threshold and a minimum cut size in order to look at the cellular growth kinetics. Results showed that the cells were growing at similar paces for both media, but their growth rate was decreasing linearly with passage number. The undecimated wavelet transform multivariate image analysis (UWT-MIA) method was developed, and was used to estimate cellular morphological features distributions (major axis, minor axis, orientation and roundness distributions) on a very large PCM image dataset using the Gabor continuous wavelet transform. Multivariate data analysis performed on the whole database (around 1 million PCM images) showed in a quantitative manner that myoblasts grown in SFM were more elongated and smaller than cells grown in SSM. The algorithms developed through this project could be used in the future on other cellular phenotypes for high-throughput screening and cell culture control applications

MURIN: Multimodal Retinal Imaging and Navigated-laser-delivery for dynamic and longitudinal tracking of photodamage in murine models

IntroductionLaser-induced photodamage is a robust method for investigating retinal pathologies in small animals. However, aiming of the photocoagulation laser is often limited by manual alignment and lacks real-time feedback on lesion location and severity. Here, we demonstrate MURIN: MUltimodal Retinal Imaging and Navigated-laser-delivery, a multimodality OCT and SLO ophthalmic imaging system with an image-guided scanning laser lesioning module optimized for the murine retina. The proposed system enables targeting of focal and extended area lesions under OCT guidance to benefit visualization of photodamage response and the precision and repeatability of laser lesion models of retinal injury.MethodsMURIN optics were optimized for simultaneous near-infrared and visible wavelength imaging/laser lesioning. Custom LabView control software was developed to steer the photocoagulation laser and automatically deliver laser pulses to targets-of-interest. In vivo retinal imaging was performed in transgenic Müller glia-tdTomato reporter mice (Rlbp1:CreER; Rosaai14, 5 animals, 10 eyes) and microglia-GFP/Müller glia-tdTomato reporter mice (Cx3cr1GFP; Rlbp1:CreER; Rosaai14, 9 animals, 15 eyes) to visualize cellular changes in the retina after laser lesion delivery.ResultsReal-time MURIN imaging concurrent with laser lesioning allowed us to visualize lesion formation dynamics and any corresponding changes in retinal morphology. We observe increasing fluorescence photoconversion on SLO and scattering contrast on OCT. Significant morphological changes are visible on MURIN after high-severity photodamage. OCT cross-sections show the spatial extent of the lesions contract over time from diffusion areas of increased scattering to granular scatterers and corresponding SLO images show a radial pattern surrounding severe focal lesions, which may be a result of a change in Müller cell shape or orientation in response to injury. The inner plexiform layer is distorted and increased RPE thickness and scattering are observed, all of which are confirmed on corresponding hematoxylin and eosin (H&E) histology and differential interference contrast (DIC) microscopy.DiscussionMURIN as a unique imaging platform that enables combined SLO and OCT imaging with an integrated image-guided laser lesioning module. This technology has clear benefits over existing multimodal imaging and laser lesioning systems by enabling simultaneous multimodal imaging, independent and precise control of Iridex laser pulse parameters and patterns, and real-time OCT and SLO visualization of lesion formation

Investigating the mechanotransduction by two-photon fluorescence microscopy measurement of intracellular free energy of binding

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 2008.Includes bibliographical references (p. 99-108).Force, due either to haemodynamic shear stress or relayed directly to the cell through adhesion complexes, is transmitted and translated into biological signals. This process is known as mechanotransduction. Extensive studies have been carried out on the signaling pathways involved in mechanotransduction. However, the mechanism(s) of mechanotransduction has yet to be fully understood. This thesis focuses on the measurement of the intracellular binding constant between focal adhesion proteins of interest, GFP-Paxillin and FAT-mCherry, using two-photon excitation fluorescence microscopy and the utility of it as a measure of protein conformational change. The hypothesis tested is that force-induced changes in protein conformation alter inter-protein binding affinity. A comprehensive toolkit that utilizes fluorescence microscopy techniques, Forster Resonance Energy Transfer (FRET) and its corollary, Fluorescence Lifetime Imaging (FLIM), as well as Fluorescence Correlation Spectroscopy (FCS), was developed. A procedure by which low photon counts cell data from FLIM could be included in global analysis fits and be corrected for was developed. This results in the recovery of maximum information from cellular data. Successful intracellular FCS measurements were combined with FLIM global analysis data to calculate the free energy of binding between GFP-Paxillin and FAT-mCherry. Results demonstrate that inter-cell heterogeneity exists and likely gives rise to differences in measured AIG. The application of these measurement techniques to cells experiencing 10% step strain shows that inter-protein binding is tighter upon stretch application. The source of this change is not clear, though Tyr phosphorylation has been ruled out by biochemical disruption of kinase activity.by Nur Aida Abdul Rahim.Ph.D

Tools for interfacing, extracting, and analyzing neural signals using wide-field fluorescence imaging and optogenetics in awake behaving mice

Imaging of multiple cells has rapidly multiplied the rate of data acquisition as well as our knowledge of the complex dynamics within the mammalian brain. The process of data acquisition has been dramatically enhanced with highly affordable, sensitive image sensors enable high-throughput detection of neural activity in intact animals. Genetically encoded calcium sensors deliver a substantial boost in signal strength and in combination with equally critical advances in the size, speed, and sensitivity of image sensors available in scientific cameras enables high-throughput detection of neural activity in behaving animals using traditional wide-field fluorescence microscopy. However, the tremendous increase in data flow presents challenges to processing, analysis, and storage of captured video, and prompts a reexamination of traditional routines used to process data in neuroscience and now demand improvements in both our hardware and software applications for processing, analyzing, and storing captured video. This project demonstrates the ease with which a dependable and affordable wide-field fluorescence imaging system can be assembled and integrated with behavior control and monitoring system such as found in a typical neuroscience laboratory. An Open-source MATLAB toolbox is employed to efficiently analyze and visualize large imaging data sets in a manner that is both interactive and fully automated. This software package provides a library of image pre-processing routines optimized for batch-processing of continuous functional fluorescence video, and additionally automates a fast unsupervised ROI detection and signal extraction routine. Further, an extension of this toolbox that uses GPU programming to process streaming video, enabling the identification, segmentation and extraction of neural activity signals on-line is described in which specific algorithms improve signal specificity and image quality at the single cell level in a behaving animal. This project describes the strategic ingredients for transforming a large bulk flow of raw continuous video into proportionally informative images and knowledge

Effect of radiation and co-culture with fibroblasts on growth characteristics and invasiveness of 3D breast cancer models

Invasiveness is a major factor contributing to cancer metastasis. Given the broad variety and plasticity of invasion mechanisms, assessing potential metastasis-promoting effects of irradiation for specific mechanisms is important for further understanding of potential adverse effects of radiotherapy. Previous investigations of radiation effects on invasion were mainly done by 2D methods that cannot differentiate different invasion mechanisms. In fibroblasts-led collective invasion mechanisms, fibroblasts degrade the extracellular matrix and produce tracks for cancer cells with epithelial traits to follow. A major goal of the present work was to establish a model for this invasion mechanism and to study the effect of radiation on it. After verifying their epithelial-like, non-invasive characteristics, breast cancer cells (MCF-7 and BT474) were co-cultured in ultra-low adhesion plates with human normal fibroblasts (BJ1-hTert and HDF). Cocultivation with fibroblasts had little effects on spheroid growth, radiation-induced growth delay and repair of DNA damage. Epithelial-like MCF-7 and BT474 cells gain ability to invade into matrix if cocultured with normal fibroblasts. High-resolution imaging showed features of fibroblast-led collective invasion. This new model was used to investigate radiation effects on invasiveness. Irradiation reduced the number of invading cells in models using BJ1-hTert, but not HDF. However, sensitivity to the radiomimetic drug neocarcinostatin was comparable in both fibroblast strains. Radiation had little effects on invasion distance, showing that with this model effects on number of invading cells and distance can be uncoupled. In conclusion, no invasion-promoting effect of irradiation could be found with this model. In proof-of-concept studies, analysis of NAD(P)H and FAD autofluorescence via TPEF and FLIM was tested as tool for label-free differentiation of cancer cells and fibroblasts in 3D situations, including invasion experiments. In addition, preliminary FLIM data suggest that MCF-7 cells undergo a transient metabolic shift towards oxidative phosphorylation after irradiation, which could be verified by mitochondrial staining experiments

Glioma on Chips Analysis of glioma cell guidance and interaction in microfluidic-controlled microenvironment enabled by machine learning

In biosystems, chemical and physical fields established by gradients guide cell migration, which is a fundamental phenomenon underlying physiological and pathophysiological processes such as development, morphogenesis, wound healing, and cancer metastasis. Cells in the supportive tissue of the brain, glia, are electrically stimulated by the local field potentials from neuronal activities. How the electric field may influence glial cells is yet fully understood. Furthermore, the cancer of glia, glioma, is not only the most common type of brain cancer, but the high-grade form of it (glioblastoma) is particularly aggressive with cells migrating into the surrounding tissues (infiltration) and contribute to poor prognosis. In this thesis, I investigate how electric fields in the microenvironment can affect the migration of glioblastoma cells using a versatile microsystem I have developed. I employ a hybrid microfluidic design to combine poly(methylmethacrylate) (PMMA) and poly(dimethylsiloxane) (PDMS), two of the most common materials for microfluidic fabrication. The advantages of the two materials can be complemented while disadvantages can be mitigated. The hybrid microfluidics have advantages such as versatile 3D layouts in PMMA, high dimensional accuracy in PDMS, and rapid prototype turnaround by facile bonding between PMMA and PDMS using a dual-energy double sided tape. To accurately analyze label-free cell migration, a machine learning software, Usiigaci, is developed to automatically segment, track, and analyze single cell movement and morphological changes under phase contrast microscopy. The hybrid microfluidic chip is then used to study the migration of glioblastoma cell models, T98G and U-251MG, in electric field (electrotaxis). The influence of extracellular matrix and chemical ligands on glioblastoma electrotaxis are investigated. I further test if voltage-gated calcium channels are involved in glioblastoma electrotaxis. The electrotaxes of glioblastoma cells are found to require optimal laminin extracellular matrices and depend on different types of voltage-gated calcium channels, voltage-gated potassium channels, and sodium transporters. A reversiblysealed hybrid microfluidic chip is developed to study how electric field and laminar shear can condition confluent endothelial cells and if the biomimetic conditions affect glioma cell adhesion to them. It is found that glioma/endothelial adhesion is mediated by the Ang1/Tie2 signaling axis and adhesion of glioma is slightly increased to endothelial cells conditioned with shear flow and moderate electric field. In conclusion, robust and versatile hybrid microsystems are employed for studying glioma biology with emphasis on cell migration. The hybrid microfluidic tools can enable us to elucidate fundamental mechanisms in the field of the tumor biology and regenerative medicine.Okinawa Institute of Science and Technology Graduate Universit