A review and comparison of breast tumor cell nuclei segmentation performances using deep convolutional neural networks

Abstract: Breast cancer is currently the second most common cause of cancer-related death in women. Presently, the clinical benchmark in cancer diagnosis is tissue biopsy examination. However, the manual process of histopathological analysis is laborious, time-consuming, and limited by the quality of the specimen and the experience of the pathologist. This study's objective was to determine if deep convolutional neural networks can be trained, with transfer learning, on a set of histopathological images independent of breast tissue to segment tumor nuclei of the breast. Various deep convolutional neural networks were evaluated for the study, including U-Net, Mask R-CNN, and a novel network (GB U-Net). The networks were trained on a set of Hematoxylin and Eosin (H&E)-stained images of eight diverse types of tissues. GB U-Net demonstrated superior performance in segmenting sites of invasive diseases (AJI = 0.53, mAP = 0.39 & AJI = 0.54, mAP = 0.38), validated on two hold-out datasets exclusively containing breast tissue images of approximately 7,582 annotated cells. The results of the networks, trained on images independent of breast tissue, demonstrated that tumor nuclei of the breast could be accurately segmented

Improving biomarker assessment in breast pathology

The accuracy of prognostic and therapy-predictive biomarker assessment in breast tumours is crucial for management and therapy decision in patients with breast cancer. In this thesis, biomarkers used in clinical practice with emphasise on Ki67 and HER2 were studied using several methods including immunocytochemistry, in situ hybridisation, gene expression assays and digital image analysis, with the overall aim to improve routine biomarker evaluation and clarify the prognostic potential in early breast cancer. In paper I, we reported discordances in biomarker status from aspiration cytology and paired surgical specimens from breast tumours. The limited prognostic potential of immunocytochemistry-based Ki67 scoring demonstrated that immunohistochemistry on resected specimens is the superior method for Ki67 evaluation. In addition, neither of the methods were sufficient to predict molecular subtype. Following this in paper II, biomarker agreement between core needle biopsies and subsequent specimens was investigated, both in the adjuvant and neoadjuvant setting. Discordances in Ki67 and HER2 status between core biopsies and paired specimens suggested that these biomarkers should be re-tested on all surgical breast cancer specimens. In paper III, digital image analysis using a virtual double staining software was used to compare methods for assessment of proliferative activity, including mitotic counts, Ki67 and the alternative marker PHH3, in different tumour regions (hot spot, invasive edge and whole section). Digital image analysis using virtual double staining of hot spot Ki67 outperformed the alternative markers of proliferation, especially in discriminating luminal B from luminal A tumours. Replacing mitosis in histological grade with hot spot-scored Ki67 added significant prognostic information. Following these findings, the optimal definition of a hot spot for Ki67 scoring using virtual double staining in relation to molecular subtype and outcome was investigated in paper IV. With the growing evidence of global scoring as a superior method to improve reproducibility of Ki67 scoring, a different digital image analysis software (QuPath) was also used for comparison. Altogether, we found that automated global scoring of Ki67 using QuPath had independent prognostic potential compared to even the best virtual double staining hot spot algorithm, and is also a practical method for routine Ki67 scoring in breast pathology. In paper V, the clinical value of HER2 status was investigated in a unique trastuzumab-treated HER2-positive cohort, on the protein, mRNA and DNA levels. The results demonstrated that low levels of ERBB2 mRNA but neither HER2 copy numbers, HER2 ratio nor ER status, was associated with risk of recurrence among anti-HER2 treated breast cancer patients. In conclusion, we have identified important clinical aspects of Ki67 and HER2 evaluation and provided methods to improve the prognostic potential of Ki67 using digital image analysis. In addition to protein expression of routine biomarkers, mRNA levels by targeted gene expression assays may add further prognostic value in early breast cance

The impact of pre- and post-image processing techniques on deep learning frameworks: A comprehensive review for digital pathology image analysis

Recently, deep learning frameworks have rapidly become the main methodology for analyzing medical images. Due to their powerful learning ability and advantages in dealing with complex patterns, deep learning algorithms are ideal for image analysis challenges, particularly in the field of digital pathology. The variety of image analysis tasks in the context of deep learning includes classification (e.g., healthy vs. cancerous tissue), detection (e.g., lymphocytes and mitosis counting), and segmentation (e.g., nuclei and glands segmentation). The majority of recent machine learning methods in digital pathology have a pre- and/or post-processing stage which is integrated with a deep neural network. These stages, based on traditional image processing methods, are employed to make the subsequent classification, detection, or segmentation problem easier to solve. Several studies have shown how the integration of pre- and post-processing methods within a deep learning pipeline can further increase the model's performance when compared to the network by itself. The aim of this review is to provide an overview on the types of methods that are used within deep learning frameworks either to optimally prepare the input (pre-processing) or to improve the results of the network output (post-processing), focusing on digital pathology image analysis. Many of the techniques presented here, especially the post-processing methods, are not limited to digital pathology but can be extended to almost any image analysis field

The impact of pre- and post-image processing techniques on deep learning frameworks: A comprehensive review for digital pathology image analysis.

Recently, deep learning frameworks have rapidly become the main methodology for analyzing medical images. Due to their powerful learning ability and advantages in dealing with complex patterns, deep learning algorithms are ideal for image analysis challenges, particularly in the field of digital pathology. The variety of image analysis tasks in the context of deep learning includes classification (e.g., healthy vs. cancerous tissue), detection (e.g., lymphocytes and mitosis counting), and segmentation (e.g., nuclei and glands segmentation). The majority of recent machine learning methods in digital pathology have a pre- and/or post-processing stage which is integrated with a deep neural network. These stages, based on traditional image processing methods, are employed to make the subsequent classification, detection, or segmentation problem easier to solve. Several studies have shown how the integration of pre- and post-processing methods within a deep learning pipeline can further increase the model's performance when compared to the network by itself. The aim of this review is to provide an overview on the types of methods that are used within deep learning frameworks either to optimally prepare the input (pre-processing) or to improve the results of the network output (post-processing), focusing on digital pathology image analysis. Many of the techniques presented here, especially the post-processing methods, are not limited to digital pathology but can be extended to almost any image analysis field

Machine Learning Approaches to Predict Recurrence of Aggressive Tumors

Cancer recurrence is the major cause of cancer mortality. Despite tremendous research efforts, there is a dearth of biomarkers that reliably predict risk of cancer recurrence. Currently available biomarkers and tools in the clinic have limited usefulness to accurately identify patients with a higher risk of recurrence. Consequently, cancer patients suffer either from under- or over- treatment. Recent advances in machine learning and image analysis have facilitated development of techniques that translate digital images of tumors into rich source of new data. Leveraging these computational advances, my work addresses the unmet need to find risk-predictive biomarkers for Triple Negative Breast Cancer (TNBC), Ductal Carcinoma in-situ (DCIS), and Pancreatic Neuroendocrine Tumors (PanNETs). I have developed unique, clinically facile, models that determine the risk of recurrence, either local, invasive, or metastatic in these tumors. All models employ hematoxylin and eosin (H&E) stained digitized images of patient tumor samples as the primary source of data. The TNBC (n=322) models identified unique signatures from a panel of 133 protein biomarkers, relevant to breast cancer, to predict site of metastasis (brain, lung, liver, or bone) for TNBC patients. Even our least significant model (bone metastasis) offered superior prognostic value than clinopathological variables (Hazard Ratio [HR] of 5.123 vs. 1.397 p\u3c0.05). A second model predicted 10-year recurrence risk, in women with DCIS treated with breast conserving surgery, by identifying prognostically relevant features of tumor architecture from digitized H&E slides (n=344), using a novel two-step classification approach. In the validation cohort, our DCIS model provided a significantly higher HR (6.39) versus any clinopathological marker (p\u3c0.05). The third model is a deep-learning based, multi-label (annotation followed by metastasis association), whole slide image analysis pipeline (n=90) that identified a PanNET high risk group with over an 8x higher risk of metastasis (versus the low risk group p\u3c0.05), regardless of cofounding clinical variables. These machine-learning based models may guide treatment decisions and demonstrate proof-of-principle that computational pathology has tremendous clinical utility

Deep Learning in Medical Image Analysis

The accelerating power of deep learning in diagnosing diseases will empower physicians and speed up decision making in clinical environments. Applications of modern medical instruments and digitalization of medical care have generated enormous amounts of medical images in recent years. In this big data arena, new deep learning methods and computational models for efficient data processing, analysis, and modeling of the generated data are crucially important for clinical applications and understanding the underlying biological process. This book presents and highlights novel algorithms, architectures, techniques, and applications of deep learning for medical image analysis

A Pictorial Exploration of Mammary Paget Disease: Insights and Perspectives

Mammary Paget disease (MPD) is a rare condition primarily affecting adult women, characterized by unilateral skin changes in the nipple–areolar complex (NAC) and frequently associated with underlying breast carcinoma. Histologically, MPD is identified by large intraepidermal epithelial cells (Paget cells) with distinct characteristics. Immunohistochemical profiles aid in distinguishing MPD from other skin conditions. Clinical evaluation and imaging techniques, including magnetic resonance imaging (MRI), are recommended if MPD is suspected, although definitive diagnosis always requires histological examination. This review delves into the historical context, epidemiology, pathogenesis, clinical manifestations, and diagnosis of MPD, emphasizing the need for early detection. The classification of MPD based on pathogenesis is explored, shedding light on its varied presentations. Treatment options, including mastectomy and breast-conserving surgery, are discussed with clear guidelines for different scenarios. Adjuvant therapies are considered, particularly in cases with underlying breast cancer. Prognostic factors are outlined, underlining the importance of early intervention. Looking to the future, emerging techniques, like liquid biopsy, new immunohistochemical and molecular markers, and artificial intelligence-based image analysis, hold the potential to transform MPD diagnosis and treatment. These innovations offer hope for early detection and improved patient care, though validation through large-scale clinical trials is needed

Detection and staging of colonic lesions using computed tomography and magnetic resonance imaging

In Sweden, more than 6000 new patients were diagnosed with colorectal cancer in 2015 of which over 4000 patients had colon cancer and 1800 died from the disease. It is the second most common cancer after breast- and prostate cancer. In the last decade, significant improvement in the treatment of both rectal and colon cancer have been achieved. Diagnostic imaging, using CT, MRI and PET/CT, has become essential in the preoperative work-up. New neoadjuvant treatment strategies are under study in colon cancer. In the selection of patients for these treatments, pre- and post-treatment imaging has also become of interest. The overall aim of this thesis is to evaluate cross sectional imaging modalities for detection of colonic polyps and staging of patients with colon cancer using CT and MRI. The aim of paper 1 was to investigate the impact of radiation dose and spatial resolution in detecting colonic polyps in a phantom study simulating computed tomographic colonography (CTC). By using different scanning protocols with different slice -thickness, pitch and tube current we showed that the dose level could be substantially reduced by lowering the tube current without compromising the detection rate for polyps larger than 5 mm. The aim of paper 2 was to evaluate if high resolution MRI of colon cancer contributed to the standard staging procedure with CT with respect to assessment of local tumour extent, nodal staging and extramural venous invasion (EMVI). An advantage of MRI over CT due to its soft tissue discrimination to identify prognostic factors such as tumour stage and extramural venous invasion was found. The result of nodal staging for both modalities were equally moderate. The aim of paper 3 was to evaluate commonly used imaging CT criteria for lymph node metastases in predicting stage III disease. Of the different imaging criteria, morphological features performed best specifically internal heterogeneity and irregular outer border. None of the size criteria were predictive. The aim of paper 4 was to validate morphological CT criteria from paper 3 in a prospectively collected patient cohort using two observers. By using the criteria internal heterogeneity and a combination including irregular outer border, a moderate sensitivity and high specificity was achieved predicting stage III disease. CT and high resolution MRI can be used to classify colonic tumours into not locally advanced or locally advanced. The prediction of lymph node metastases with the most commonly used image modalities is however unsettled and challenged. In the setting of selecting patients to neoadjuvant chemotherapy, the ongoing trials so far have used inclusion criteria based on tumour T-stage only (T3cd-T4 as locally advanced). Patients with lower tumour T-stage but still have other adverse prognostic feature such as regional metastases will therefore potentially be undertreated and patients with no metastases will potentially be overtreated. Search for other prognostic factors identified on cross sectional imaging has to be performed

Evaluation of PD-L1 expression in various formalin-fixed paraffin embedded tumour tissue samples using SP263, SP142 and QR1 antibody clones

Background & objectives: Cancer cells can avoid immune destruction through the inhibitory ligand PD-L1. PD-1 is a surface cell receptor, part of the immunoglobulin family. Its ligand PD-L1 is expressed by tumour cells and stromal tumour infltrating lymphocytes (TIL). Methods: Forty-four cancer cases were included in this study (24 triple-negative breast cancers (TNBC), 10 non-small cell lung cancer (NSCLC) and 10 malignant melanoma cases). Three clones of monoclonal primary antibodies were compared: QR1 (Quartett), SP 142 and SP263 (Ventana). For visualization, ultraView Universal DAB Detection Kit from Ventana was used on an automated platform for immunohistochemical staining Ventana BenchMark GX. Results: Comparing the sensitivity of two different clones on same tissue samples from TNBC, we found that the QR1 clone gave higher percentage of positive cells than clone SP142, but there was no statistically significant difference. Comparing the sensitivity of two different clones on same tissue samples from malignant melanoma, the SP263 clone gave higher percentage of positive cells than the QR1 clone, but again the difference was not statistically significant. Comparing the sensitivity of two different clones on same tissue samples from NSCLC, we found higher percentage of positive cells using the QR1 clone in comparison with the SP142 clone, but once again, the difference was not statistically significant. Conclusion: The three different antibody clones from two manufacturers Ventana and Quartett, gave comparable results with no statistically significant difference in staining intensity/ percentage of positive tumour and/or immune cells. Therefore, different PD-L1 clones from different manufacturers can potentially be used to evaluate the PD- L1 status in different tumour tissues. Due to the serious implications of the PD-L1 analysis in further treatment decisions for cancer patients, every antibody clone, staining protocol and evaluation process should be carefully and meticulously validated