Computational Methods for Delineating Multiple Nuclear Phenotypes from Different Imaging Modalities

Characterizing histopathology or organoid models of breast cancer can provide fundamental knowledge that will lead to a better understanding of tumors, response to therapeutic agents, and discovery of new targeted therapies. To this aim, the delineation of nuclei is significantly interesting since it provides rich information about the aberrant microanatomy or colony formation. For example, (i) cancer cells tend to be larger and, if coupled with high chromatin content, may indicate aneuploidy; (ii) cellular density can be the result of rapid proliferation; (iii) nuclear micro-texture can be a surrogate for fluctuation of heterochromatin patterns, where epigenetic aberrations in cancers are sometimes correlated with alterations in heterochromatin distribution; and (iv) normalized colony formation of cancer cells, in 3D culture, can serve as a surrogate metric for tumor suppression. These evidences suggest that nuclear segmentation and profiling is a major step for subsequent bioinformatics analysis. However, there are two barriers which include technical variations during the sample preparation step and biological heterogeneity since no two patients/samples are alike. As a result of these complexities, extension of deep learning methodologies will have a significant impact on the robust characterization and profiling of pathology sections or organoid models. In this presentation, we demonstrate that integration of regional and contextual representations, within the framework of a deep encoder-decoder architecture, contribute to robust delineation of various nuclear phenotypes from both bright field and confocal microscopy. The deep encoder-decoder architecture can infer perceptual boundaries that are necessary to decompose clumps of nuclei. The method has been validated on pathology section and organoid models of human mammary epithelial cells

Hover-Net : simultaneous segmentation and classification of nuclei in multi-tissue histology images

Nuclear segmentation and classification within Haematoxylin & Eosin stained histology images is a fundamental prerequisite in the digital pathology work-flow. The development of automated methods for nuclear segmentation and classification enables the quantitative analysis of tens of thousands of nuclei within a whole-slide pathology image, opening up possibilities of further analysis of large-scale nuclear morphometry. However, automated nuclear segmentation and classification is faced with a major challenge in that there are several different types of nuclei, some of them exhibiting large intra-class variability such as the nuclei of tumour cells. Additionally, some of the nuclei are often clustered together. To address these challenges, we present a novel convolutional neural network for simultaneous nuclear segmentation and classification that leverages the instance-rich information encoded within the vertical and horizontal distances of nuclear pixels to their centres of mass. These distances are then utilised to separate clustered nuclei, resulting in an accurate segmentation, particularly in areas with overlapping instances. Then, for each segmented instance the network predicts the type of nucleus via a devoted up-sampling branch. We demonstrate state-of-the-art performance compared to other methods on multiple independent multi-tissue histology image datasets. As part of this work, we introduce a new dataset of Haematoxylin & Eosin stained colorectal adenocarcinoma image tiles, containing 24,319 exhaustively annotated nuclei with associated class labels

AI-Enabled Contextual Representations for Image-based Integration in Health and Safety

Recent advancements in the area of Artificial Intelligence (AI) have made it the field of choice for automatically processing and summarizing information in big-data domains such as high-resolution images. This approach, however, is not a one-size-fits-all solution, and must be tailored to each application. Furthermore, each application comes with its own unique set of challenges including technical variations, validation of AI solutions, and contextual information. These challenges are addressed in three human-health and safety related applications: (i) an early warning system of slope failures in open-pit mining operations; (ii) the modeling and characterization of 3D cell culture models imaged with confocal microscopy; and (iii) precision medicine of biomarker discovery from patients with glioblastoma multiforme through digital pathology. The methodologies and results in each of these domains show how tailor-made AI solutions can be used for automatically extracting and summarizing pertinent information from big-data applications for enhanced decision making

A Structured Tumor-Immune Microenvironment in Triple Negative Breast Cancer Revealed by Multiplexed Ion Beam Imaging

The immune system is critical in modulating cancer progression, but knowledge of immune composition, phenotype, and interactions with tumor is limited. We used multiplexed ion beam imaging by time-of-flight (MIBI-TOF) to simultaneously quantify in situ expression of 36 proteins covering identity, function, and immune regulation at sub-cellular resolution in 41 triple-negative breast cancer patients. Multi-step processing, including deep-learning-based segmentation, revealed variability in the composition of tumor-immune populations across individuals, reconciled by overall immune infiltration and enriched co-occurrence of immune subpopulations and checkpoint expression. Spatial enrichment analysis showed immune mixed and compartmentalized tumors, coinciding with expression of PD1, PD-L1, and IDO in a cell-type- and location-specific manner. Ordered immune structures along the tumor-immune border were associated with compartmentalization and linked to survival. These data demonstrate organization in the tumor-immune microenvironment that is structured in cellular composition, spatial arrangement, and regulatory-protein expression and provide a framework to apply multiplexed imaging to immune oncology

A Structured Tumor-Immune Microenvironment in Triple Negative Breast Cancer Revealed by Multiplexed Ion Beam Imaging

The immune system is critical in modulating cancer progression, but knowledge of immune composition, phenotype, and interactions with tumor is limited. We used multiplexed ion beam imaging by time-of-flight (MIBI-TOF) to simultaneously quantify in situ expression of 36 proteins covering identity, function, and immune regulation at sub-cellular resolution in 41 triple-negative breast cancer patients. Multi-step processing, including deep-learning-based segmentation, revealed variability in the composition of tumor-immune populations across individuals, reconciled by overall immune infiltration and enriched co-occurrence of immune subpopulations and checkpoint expression. Spatial enrichment analysis showed immune mixed and compartmentalized tumors, coinciding with expression of PD1, PD-L1, and IDO in a cell-type- and location-specific manner. Ordered immune structures along the tumor-immune border were associated with compartmentalization and linked to survival. These data demonstrate organization in the tumor-immune microenvironment that is structured in cellular composition, spatial arrangement, and regulatory-protein expression and provide a framework to apply multiplexed imaging to immune oncology

Endogenous Neurosteroid Hormone Production and Early Oligodendricyte Development

The field of neuroendocrinology grew immensely with the realization that steroid hormone production is not confined to the adrenal and reproductive glands but also occurs in the central nervous system (CNS). Steroids synthesized de novo in the brain and spinal cord are referred to as neurosteroid or neuroactive hormones and encompass estrogens, androgens, glucocorticoids, and mineralocorticoids. Though all substrates and enzymes required for neurosteroid biosynthesis exhibit CNS expression, a thorough comprehension of their functionality is lacking. In addition to mediating stress responses, neurosteroids influence CNS-specific processes known to regulate neural development and pathology. Multiple sclerosis (MS), a debilitating neurodegenerative disorder classified by rampant demyelination, exemplifies the neuroendocrine crosstalk facilitated by CNS-resident steroids. Local steroid production from cholesterol, which also happens to be the primary lipid component of myelin sheaths, is critical to myelin repair. Progesterone in particular is implicated in expediting remyelination following demyelinating insults in animal models via an unknown mechanism. Despite the established effect of progesterone on myelin regeneration, its impact on early myelinogenesis remains unclear. This observation inspired the work presented here, in which I investigated a potential role for progesterone in embryonic oligodendrocyte development. Applying the synthetic progestin Nestorone to mouse cerebellar slice cultures, I found that progesterone stimulates the expression of the mature myelin protein, myelin basic protein. Curious as to whether this phenomenon mirrors progesterone-induced remyelination at the molecular level, I implemented the same experimental system in mice genetically altered to delete expression of the nuclear progesterone receptor. Unexpectedly, removal of this receptor from cerebellar slices led not only to an increase in myelin basic protein expression but more robust oligodendrocyte maturation into myelinating cells actively extending processes to axons and participating in fiber formation. The fact that this surprising effect could not be mediated by the nuclear progesterone receptor prompted me to examine potential caveats to studying individual hormones like progesterone in isolation

Bioengineering and Cell-derived Strategies for Salivary Gland Regeneration

Xerostomia (dry mouth symptoms) is a group of incurable debilitating conditions of salivary glands caused by aging, radiation/chemical exposure, or aberrant inflammation in the salivary glands. During this PhD thesis, we aimed to evaluate whether cell-derived strategies (e.g., extracellular vesicles, EVs) could be a potential new therapy to ameliorate salivary gland injury and restore function after radiotherapy or in autoimmune diseases. In addition, we aimed to develop new imaging techniques for both 2D and 3D analysis of larger samples which allows for quantification of disease and regenerative features. Firstly, we constructed an in vivo murine model of 25 Gy irradiation-induced salivary gland damage to evaluate the potential of human dental pulp stem cell (hDPSCs)-derived EVs. EVs were injected 3x weekly via tail vein, beginning immediately after irradiation. Salivary gland function was evaluated 18 days after irradiation using salivary gland flow rate (SFR), gene expression (by qRT-PCR) and histopathology. Next, we tested different methods to generate PCSS using a vibratome and evaluated the slices in terms of viability (by WST-1), gene expression (by qRT-PCR), secreted α-amylase activity (by α-amylase assay kit) and histological/light sheet fluorescence microscopy (LSFM) three-dimensional imaging. Following irradiation, SFR decreased while senescence-associated β-galactosidase-positive cells (via immunofluorescences) and senescence-related genes and secretory-phenotypes (e.g., p21 and MMP3 in qRT-PCR) increased. SFR was unchanged following EVs treatment, but senescence-associated genes and secretory-phenotypes decreased. We also demonstrated that in an animal model of Sjögren’s syndrome, which exhibit dry mouth symptoms, that hDPSCs-EVs could inhibit the acquisition of the senescent phenotype in salivary gland epithelial cells (SGECs) and alleviate the loss of glandular function. EVs were also found to perform these effects through an underlying immunomodulatory mechanism. For PCSS, we developed protocols to produce viable slices of controled thicknesses which retained the ability to secrete functional α-amylase for at least two days in ex vivo culture. Phenotypic salivary gland cell epithelial markers (e.g., Keratin 5 and Aquaporin 5) increased over time in PCSS (by qRT-PCR), indicating the retention of cells that are necessary for salivary glands’ function. We developed workflows to perform LSFM 3D visualization in whole salivary glands as well as the PCSS model. In conclusion, hDPSCs-EVs reduced senescence of salivary gland epithelial cells in both murine irradiation and Sjögren’s syndrome models and may become a promising future for xerostomia patients. For the murine PCSS, we successfully established an executable operating procedure at the methodological level to reliably generate viable and functional murine PCSS and developed new state-of-the-art analytical methods (such as LFSM 3D imaging and qRT-PCR) to increase the diversity of objective tools to evaluate PCSS. Therefore, this work laid the foundation for the future application of other therapies (such as irradiation therapy or EVs therapy) to the PCSS model. Those future applications could include drug screening or mechanism of injury study. At the same time, we developed a sustainable histology process to reduce xylene utilization in histological processing for salivary gland tissue processing. Therefore, this work has developed a set of in vitro and in vivo experiments with state-of-the-art methods to better understand disease mechanisms and to evaluate new therapies for salivary glands

On the histopathological growth patterns of colorectal liver metastasis:a Study of Histology, Immunology, Genetics, and Prognosis

This thesis aims to validate and establish the histopathological growth patterns of colorectal cancer liver metastasis as a relevant biomarker, and to evaluate immunity and genetics as potential underlying biological mechanisms

Deep Learning in Breast Cancer Imaging: A Decade of Progress and Future Directions

Breast cancer has reached the highest incidence rate worldwide among all malignancies since 2020. Breast imaging plays a significant role in early diagnosis and intervention to improve the outcome of breast cancer patients. In the past decade, deep learning has shown remarkable progress in breast cancer imaging analysis, holding great promise in interpreting the rich information and complex context of breast imaging modalities. Considering the rapid improvement in the deep learning technology and the increasing severity of breast cancer, it is critical to summarize past progress and identify future challenges to be addressed. In this paper, we provide an extensive survey of deep learning-based breast cancer imaging research, covering studies on mammogram, ultrasound, magnetic resonance imaging, and digital pathology images over the past decade. The major deep learning methods, publicly available datasets, and applications on imaging-based screening, diagnosis, treatment response prediction, and prognosis are described in detail. Drawn from the findings of this survey, we present a comprehensive discussion of the challenges and potential avenues for future research in deep learning-based breast cancer imaging.Comment: Survey, 41 page