Deep convolutional neural networks for segmenting 3D in vivo multiphoton images of vasculature in Alzheimer disease mouse models

The health and function of tissue rely on its vasculature network to provide reliable blood perfusion. Volumetric imaging approaches, such as multiphoton microscopy, are able to generate detailed 3D images of blood vessels that could contribute to our understanding of the role of vascular structure in normal physiology and in disease mechanisms. The segmentation of vessels, a core image analysis problem, is a bottleneck that has prevented the systematic comparison of 3D vascular architecture across experimental populations. We explored the use of convolutional neural networks to segment 3D vessels within volumetric in vivo images acquired by multiphoton microscopy. We evaluated different network architectures and machine learning techniques in the context of this segmentation problem. We show that our optimized convolutional neural network architecture, which we call DeepVess, yielded a segmentation accuracy that was better than both the current state-of-the-art and a trained human annotator, while also being orders of magnitude faster. To explore the effects of aging and Alzheimer's disease on capillaries, we applied DeepVess to 3D images of cortical blood vessels in young and old mouse models of Alzheimer's disease and wild type littermates. We found little difference in the distribution of capillary diameter or tortuosity between these groups, but did note a decrease in the number of longer capillary segments ($>75\mu m$) in aged animals as compared to young, in both wild type and Alzheimer's disease mouse models.Comment: 34 pages, 9 figure

Interpretable classification of Alzheimer's disease pathologies with a convolutional neural network pipeline.

Neuropathologists assess vast brain areas to identify diverse and subtly-differentiated morphologies. Standard semi-quantitative scoring approaches, however, are coarse-grained and lack precise neuroanatomic localization. We report a proof-of-concept deep learning pipeline that identifies specific neuropathologies-amyloid plaques and cerebral amyloid angiopathy-in immunohistochemically-stained archival slides. Using automated segmentation of stained objects and a cloud-based interface, we annotate > 70,000 plaque candidates from 43 whole slide images (WSIs) to train and evaluate convolutional neural networks. Networks achieve strong plaque classification on a 10-WSI hold-out set (0.993 and 0.743 areas under the receiver operating characteristic and precision recall curve, respectively). Prediction confidence maps visualize morphology distributions at high resolution. Resulting network-derived amyloid beta (Aβ)-burden scores correlate well with established semi-quantitative scores on a 30-WSI blinded hold-out. Finally, saliency mapping demonstrates that networks learn patterns agreeing with accepted pathologic features. This scalable means to augment a neuropathologist's ability suggests a route to neuropathologic deep phenotyping

A multi-scale imaging approach to understand osteoarthritis development

X-ray phase-contrast imaging is an innovative and advanced imaging method. Contrary to conventional radiology, where the image contrast is primarily determined by X-ray attenuation, phase-contrast images contain additional information generated by the phase shifts or refraction of the X-rays passing through matter. The refractive effect on tissue samples is orders of magnitude higher than the absorption effect in the X-ray energy range used in biomedical imaging. This technique makes it possible to produce excellent and enhanced image contrast, particularly when examining soft biological tissues or features with similar X-ray attenuation properties. In combination with high spatial resolution detector technology and computer tomography, X-ray phase-contrast imaging has been proved to be a powerful method to examine tissue morphology and the evolution of pathologies three-dimensionally, with great detail and without the need of contrast agents. This Thesis work has focused on developing an accurate, multi-scale X-ray-based methodology for imaging and characterizing the early stages of osteoarthritis. X-ray phase-contrast images acquired at different spatial resolutions provide unprecedented insights into cartilage and the development of its degeneration, i.e., osteoarthritis. Other types of X-ray phase-contrast imaging techniques and setups using spatial resolutions ranging from micrometer down to nanometer were applied. Lower spatial resolutions allow large sample coverage and comprehensive representations, while the nanoscale analysis provides a precise depiction of anatomical details and pathological signs. X-ray phase-contrast results are correlated to data obtained, on the same specimens, by standard laboratory methods, such as histology and transmission electron microscopy. Furthermore, X-ray phase-contrast images of cartilage were acquired using different X-ray sources and results were compared in terms of image quality. It was shown that with the use of synchrotron radiation, more detailed images and much faster data acquisitions could be achieved. A second focus in this Thesis work has been the investigation of the reaction of healthy and degenerated cartilage under different physical pressures, simulating the different levels of stress to which the tissue is subject during daily movements. A specifically designed setup was used to dynamically study cartilage response to varying pressures with X-ray phase-contrast micro-computed tomography, and a fully volumetric and quantitative methodology to accurately describe the tissue morphological variations. This study revealed changes in the behavior of the cartilage cell structure, which differ between normal and osteoarthritic cartilage tissues. The third focus of this Thesis is the realization of an automated evaluation procedure for the discrimination of healthy and cartilage images with osteoarthritis. In recent years, developments in neural networks have shown that they are excellently suited for image classification tasks. The transfer learning method was applied, in which a pre-trained neural network with cartilage images is further trained and then used for classification. This enables a fast, robust and automated grouping of images with pathological findings. A neural network constructed in this way could be used as a supporting instrument in pathology. X-ray phase-contrast imaging computed tomography can provide a powerful tool for a fully 3D, highly accurate and quantitative depiction and characterization of healthy and early stage-osteoarthritic cartilage, supporting the understanding of the development of osteoarthritis.Röntgen-Phasenkontrast-Bildgebung ist eine innovative und weiterführende Bildgebungsmethode. Im Gegensatz zu herkömlichen Absorptions-Röntgenaufnahmen, wie sie in der Radiologie verwendet werden, wird der Kontrast bei dieser Methode aus dem Effekt der Phasenverschiebung oder auch Brechung der Röngtenstrahlen gebildet. Der Brechungseffekt bei Gewebeproben ist um ein Vielfaches höher als der Absorptionseffekt des elektromagnetischen Spektrums der Röntgenstrahlen. Diese Methode ermöglicht die Darstellung von großen Kontraste im Gewebe. Unter Verwendung eines hochauflösenden Detektors und in Kombination mit der Computer-Tomographie, ist Phasenkontrast-Bildgebung eine sehr gute Methode um Knorpelgewebe und Arthrose im Knorpel zu untersuchen. Diese Arbeit beschreibt primär ein Verfahren zur Darstellung arthrotischen Knorpels im Anfangsstadium. Die mit verschiedenen Auflösungen und 3D-Phasen-Kontrast-Methoden produzierten Aufnahmen ermöglichen einen noch nie dagewesenen Einblick in den Knorpel und die Entwicklung von Arthrose im Anfangsstadium. Hierbei kam die propagationsbasierte Phasenkontrastmethode mit einer Auflösung im mikrometer Bereich und die (Nano)-Holotomographie-Methode mit einer Auflösung im Submicrometer Bereich zum Einsatz. Durch Auflösung im mikrometer Bereich kann ein großes Volumen im Knorpel gescannt werden, während die Nano-Holotomographie Methode eine sehr große Detailauflösung aufweißt. Die Phasenkontrast-Aufnahmen werden mit zwei anderen wissenschaftlichen Methoden verglichen: mikroskopische Abbildungen histologisch aufgearbeiteter Knorpelproben und Aufnahmen eines Transmissionselektroskop zeigen sehr große Übereinstimmungen zur Röntgen-Phasenkontrast-Bildgebung. Desweiteren wurden Phasenkontrast-Aufnahmen von Knorpel aus unterschiedlichen Röntgenquellen verglichen. Hierbei zeigte sich, dass mit Hilfe des Teilchenbeschleunigers (Synchrotron) detailreichere und schnellere Aufnahmen erzielt werden können. Bilder aus Flüssig-Metall-Quellen zeigen sich durchaus von guter Qualität, erfordern jedoch sehr lange Aufnahmezeiten. In dieser Arbeit wird zudem das Verhalten von Knorpelgewebe, welches ein Anfangsstadium von Arthrose aufweist, unter physikalischem Druck untersucht. Hierfür wurden 3D-Computertomographie-Aufnahmen von komprimiertem Knorpelgewebe angefertig und mit Aufnahmen ohne Komprimierung verglichen. Ein quantitativer Vergleich machte Veränderungen des Verhaltens der Knorpelzellstruktur (Chondronen) sichtbar. Es konnte gezeigt werden, dass Chondrone bei arthrotischem Knorpel ein verändertes Kompressionsverhalten haben. Der dritte Fokus dieser Arbeit liegt auf der automatisierten Auswertung von Aufnahmen gesunden und arthrotischen Knorpelgewebes. Die Entwicklungen im Bereich der Neuronale Netze zeigten in den letzten Jahren, dass diese sich hervoragend für Bildklassifizierungsaufgaben eignen. Es wurde die Methode des transferierenden Lernens angewandt, bei der ein vortrainiertes Neuronales Netz mit Knorpelbildern weitertrainiert und anschließend zur Klassifizierung eingesetzt wird. Dadurch ist eine schnelle, robuste und automatisierte Gruppierung von Bildern mit pathologischen Befunden möglich. Ein derart konstruiertes Neuronales Netz könnte als unterstützendes Instrument in der Pathologie angewandt werden. Röntgen-Phasenkontrast-CT kann ein leistungsstarkes Werkzeug für eine umfassende, hochpräzise und quantitative 3D-Darstellung und Charakterisierung von gesundem Knorpel und athrotischem Knorpel im Frühstadium bieten, um das Verständnis der Entwicklung von Osteoarthritis zu erweitern

Deep Learning-Based Particle Detection and Instance Segmentation for Microscopy Images

Bildgebende mikroskopische Verfahren ermöglichen Forschern, Einblicke in komplexe, bisher unverstandene Prozesse zu gewinnen. Um den Forschern den Weg zu neuen Erkenntnissen zu erleichtern, sind hoch-automatisierte, vielseitige, genaue, benutzerfreundliche und zuverlässige Methoden zur Partikeldetektion und Instanzsegmentierung erforderlich. Diese Methoden sollten insbesondere für unterschiedliche Bildgebungsbedingungen und Anwendungen geeignet sein, ohne dass Expertenwissen für Anpassungen erforderlich ist. Daher werden in dieser Arbeit eine neue auf Deep Learning basierende Methode zur Partikeldetektion und zwei auf Deep Learning basierende Methoden zur Instanzsegmentierung vorgestellt. Der Partikeldetektionsansatz verwendet einen von der Partikelgröße abhängigen Hochskalierungs-Schritt und ein U-Net Netzwerk für die semantische Segmentierung von Partikelmarkern. Nach der Validierung der Hochskalierung mit synthetisch erzeugten Daten wird die Partikeldetektionssoftware BeadNet vorgestellt. Die Ergebnisse auf einem Datensatz mit fluoreszierenden Latex-Kügelchen zeigen, dass BeadNet Partikel genauer als traditionelle Methoden detektieren kann. Die beiden neuen Instanzsegmentierungsmethoden verwenden ein U-Net Netzwerk mit zwei Decodern und werden für vier Objektarten und drei Mikroskopie-Bildgebungsverfahren evaluiert. Für die Evaluierung werden ein einzelner nicht balancierter Trainingsdatensatz und ein einzelner Satz von Postprocessing-Parametern verwendet. Danach wird die bessere Methode in der Cell Tracking Challenge weiter validiert, wobei mehrere Top-3-Platzierungen und für sechs Datensätze eine mit einem menschlichen Experten vergleichbare Leistung erreicht werden. Außerdem wird die neue Instanzsegmentierungssoftware microbeSEG vorgestellt. microbeSEG verwendet, analog zu BeadNet, OMERO für die Datenverwaltung und bietet Funktionen für die Erstellung von Trainingsdaten, das Trainieren von Modellen, die Modellevaluation und die Modellanwendung. Die qualitativen Anwendungen von BeadNet und microbeSEG zeigen, dass beide Tools eine genaue Auswertung vieler verschiedener Mikroskopie-Bilddaten ermöglichen. Abschließend gibt diese Dissertation einen Ausblick auf den Bedarf an weiteren Richtlinien für Bildanalyse-Wettbewerbe und Methodenvergleiche für eine zielgerichtete zukünftige Methodenentwicklung

The Effectiveness of Transfer Learning Systems on Medical Images

Deep neural networks have revolutionized the performances of many machine learning tasks such as medical image classification and segmentation. Current deep learning (DL) algorithms, specifically convolutional neural networks are increasingly becoming the methodological choice for most medical image analysis. However, training these deep neural networks requires high computational resources and very large amounts of labeled data which is often expensive and laborious. Meanwhile, recent studies have shown the transfer learning (TL) paradigm as an attractive choice in providing promising solutions to challenges of shortage in the availability of labeled medical images. Accordingly, TL enables us to leverage the knowledge learned from related data to solve a new problem. The objective of this dissertation is to examine the effectiveness of TL systems on medical images. First, a comprehensive systematic literature review was performed to provide an up-to-date status of TL systems on medical images. Specifically, we proposed a novel conceptual framework to organize the review. Second, a novel DL network was pretrained on natural images and utilized to evaluate the effectiveness of TL on a very large medical image dataset, specifically Chest X-rays images. Lastly, domain adaptation using an autoencoder was evaluated on the medical image dataset and the results confirmed the effectiveness of TL through fine-tuning strategies. We make several contributions to TL systems on medical image analysis: Firstly, we present a novel survey of TL on medical images and propose a new conceptual framework to organize the findings. Secondly, we propose a novel DL architecture to improve learned representations of medical images while mitigating the problem of vanishing gradients. Additionally, we identified the optimal cut-off layer (OCL) that provided the best model performance. We found that the higher layers in the proposed deep model give a better feature representation of our medical image task. Finally, we analyzed the effect of domain adaptation by fine-tuning an autoencoder on our medical images and provide theoretical contributions on the application of the transductive TL approach. The contributions herein reveal several research gaps to motivate future research and contribute to the body of literature in this active research area of TL systems on medical image analysis

Deep Learning in Cardiology

The medical field is creating large amount of data that physicians are unable to decipher and use efficiently. Moreover, rule-based expert systems are inefficient in solving complicated medical tasks or for creating insights using big data. Deep learning has emerged as a more accurate and effective technology in a wide range of medical problems such as diagnosis, prediction and intervention. Deep learning is a representation learning method that consists of layers that transform the data non-linearly, thus, revealing hierarchical relationships and structures. In this review we survey deep learning application papers that use structured data, signal and imaging modalities from cardiology. We discuss the advantages and limitations of applying deep learning in cardiology that also apply in medicine in general, while proposing certain directions as the most viable for clinical use.Comment: 27 pages, 2 figures, 10 table

System-Characterized Artificial Intelligence Approaches for Cardiac cellular systems and Molecular Signature analysis

The dissertation presents a significant advancement in the field of cardiac cellular systems and molecular signature systems by employing machine learning and generative artificial intelligence techniques. These methodologies are systematically characterized and applied to address critical challenges in these domains. A novel computational model is developed, which combines machine learning tools and multi-physics models. The main objective of this model is to accurately predict complex cellular dynamics, taking into account the intricate interactions within the cardiac cellular system. Furthermore, a comprehensive framework based on generative adversarial networks (GANs) is proposed. This framework is designed to generate synthetic data that faithfully represents an in-vitro cardiac cellular system. The generated data can be used to enhance the understanding and analysis of the system’s behavior. Additionally, a novel AI approach is formulated, which integrates deep learning and GAN techniques for Raman characterization. This approach enables efficient detection of multi-analyte mixtures by leveraging the power of deep learning algorithms and the generation of synthetic data through GANs. Overall, the integration of machine learning, generative artificial intelligence, and multi-physics modeling provides valuable insights and tools for precise prediction and efficient detection in cardiac cellular systems and molecular signature systems

Better prognostic markers for nonmuscle invasive papillary urothelial carcinomas

Bladder cancer is a common type of cancer, especially among men in developed countries. Most cancers in the urinary bladder are papillary urothelial carcinomas. They are characterized by a high recurrence frequency (up to 70 %) after local resection. It is crucial for prognosis to discover these recurrent tumours at an early stage, especially before they become muscle-invasive. Reliable prognostic biomarkers for tumour recurrence and stage progression are lacking. This is why patients diagnosed with a non-muscle invasive bladder cancer follow extensive follow-up regimens with possible serious side effects and with high costs for the healthcare systems. WHO grade and tumour stage are two central biomarkers currently having great impact on both treatment decisions and follow-up regimens. However, there are concerns regarding the reproducibility of WHO grading, and stage classification is challenging in small and fragmented tumour material. In Paper I, we examined the reproducibility and the prognostic value of all the individual microscopic features making up the WHO grading system. Among thirteen extracted features there was considerable variation in both reproducibility and prognostic value. The only feature being both reasonably reproducible and statistically significant prognostic was cell polarity. We concluded that further validation studies are needed on these features, and that future grading systems should be based on well-defined features with true prognostic value. With the implementation of immunotherapy, there is increasing interest in tumour immune response and the tumour microenvironment. In a search for better prognostic biomarkers for tumour recurrence and stage progression, in Paper II, we investigated the prognostic value of tumour infiltrating immune cells (CD4, CD8, CD25 and CD138) and previously investigated cell proliferation markers (Ki-67, PPH3 and MAI). Low Ki 67 and tumour multifocality were associated with increased recurrence risk. Recurrence risk was not affected by the composition of immune cells. For stage progression, the only prognostic immune cell marker was CD25. High values for MAI was also strongly associated with stage progression. However, in a multivariate analysis, the most prognostic feature was a combination of MAI and CD25. BCG-instillations in the bladder are indicated in intermediate and high-risk non-muscle invasive bladder cancer patients. This old-fashion immunotherapy has proved to reduce both recurrence- and progression-risk, although it is frequently followed by unpleasant side-effects. As many as 30-50% of high-risk patients receiving BCG instillations, fail by develop high-grade recurrences. They do not only suffer from unnecessary side-effects, but will also have a delay in further treatment. Together with colleagues at three different Dutch hospitals, in Paper III, we looked at the prognostic and predictive value of T1-substaging. A T1-tumour invades the lamina propria, and we wanted to separate those with micro- from those with extensive invasion. We found that BCG-failure was more common among patients with extensive invasion. Furthermore, T1-substaging was associated with both high-grade recurrence-free and progression-free survival. Finally, in Paper IV, we wanted to investigate the prognostic value of two classical immunohistochemical markers, p53 and CK20, and compare them with previously investigated proliferation markers. p53 is a surrogate marker for mutations in the gene TP53, considered to be a main characteristic for muscle-invasive tumours. CK20 is a surrogate marker for luminal tumours in the molecular classification of bladder cancer, and is frequently used to distinguish reactive urothelial changes from urothelial carcinoma in situ. We found both positivity for p53 and CK20 to be significantly associated with stage progression, although not performing better than WHO grade and stage. The proliferation marker MAI, had the highest prognostic value in our study. Any combination of variables did not perform better in a multivariate analysis than MAI alone

Fluorescence microscopy tensor imaging representations for large-scale dataset analysis

Understanding complex biological systems requires the system-wide characterization of cellular and molecular features. Recent advances in optical imaging technologies and chemical tissue clearing have facilitated the acquisition of whole-organ imaging datasets, but automated tools for their quantitative analysis and visualization are still lacking. We have here developed a visualization technique capable of providing whole-organ tensor imaging representations of local regional descriptors based on fluorescence data acquisition. This method enables rapid, multiscale, analysis and virtualization of large-volume, high-resolution complex biological data while generating 3D tractographic representations. Using the murine heart as a model, our method allowed us to analyze and interrogate the cardiac microvasculature and the tissue resident macrophage distribution and better infer and delineate the underlying structural network in unprecedented detail