Evaluation of deep learning training strategies for the classification of bone marrow cell images

BACKGROUND AND OBJECTIVE The classification of bone marrow (BM) cells by light microscopy is an important cornerstone of hematological diagnosis, performed thousands of times a day by highly trained specialists in laboratories worldwide. As the manual evaluation of blood or BM smears is very time-consuming and prone to inter-observer variation, new reliable automated systems are needed. METHODS We aim to improve the automatic classification performance of hematological cell types. Therefore, we evaluate four state-of-the-art Convolutional Neural Network (CNN) architectures on a dataset of 171,374 microscopic cytological single-cell images obtained from BM smears from 945 patients diagnosed with a variety of hematological diseases. We further evaluate the effect of an in-domain vs. out-of-domain pre-training, and assess whether class activation maps provide human-interpretable explanations for the models' predictions. RESULTS The best performing pre-trained model (Regnet_y_32gf) yields a mean precision, recall, and F1 scores of 0.787±0.060, 0.755±0.061, and 0.762±0.050, respectively. This is a 53.5% improvement in precision and 7.3% improvement in recall over previous results with CNNs (ResNeXt-50) that were trained from scratch. The out-of-domain pre-training apparently yields general feature extractors/filters that apply very well to the BM cell classification use case. The class activation maps on cell types with characteristic morphological features were found to be consistent with the explanations of a human domain expert. For example, the Auer rods in the cytoplasm were the predictive cellular feature for correctly classified images of faggot cells. CONCLUSIONS Our study provides data that can help hematology laboratories to choose the optimal training strategy for blood cell classification deep learning models to improve computer-assisted blood and bone marrow cell identification. It also highlights the need for more specific training data, i.e. images of difficult-to-classify classes, including cells labeled with disease information

Interpretable Deep Learning Predicts the Molecular Endometrial Cancer Classification from H&E Images: A Combined Analysis of the Portec Randomized Clinical Trials

Background: Endometrial Cancer (EC) is molecularly classified into the POLE- mutated (POLE mut), mismatch-repair deficient (MMRd), p53 abnormal (p53abn) and no specific molecular profile (NSMP) subgroups. We aimed to develop an interpretable DL pipeline for image-based prediction of the four-class molecular EC classification (im4MEC), to identify morpho-molecular correlates and refine EC prognostication. Methods: Diagnostic hematoxylin and Eosin (H&E)-stained slides from 2028 EC patients of the combined PORTEC-1-2,-3 randomized trials and four clinical cohorts were included. im4MEC combined self-supervised learning and an attention mechanism to achieve optimal performance. Tiles with highest attention scores were reviewed to identify morpho-molecular correlates. Predictions of a nuclear classification DL model served to derive interpretable morphological features and their relative contribution in the profiling of each molecular class. Prognostic refinement was explored through morphological and Kaplan-Meier-based survival analyses. Findings: im4MEC achieved a macro-average area under the receiver-operating-characteristic curve (AUROC) on cross-validation of 0·874±0·189, and 0·876 on the independent test set PORTEC-3 with highest performance of 0·928 among p53abn EC. Overall recurrence by image-based molecular class was significantly different in PORTEC-3 (p=1·e-04). Top-attended tiles indicated a significant association between dense lymphocyte infiltrates and POLE mut and MMRd EC, high nuclear atypia and p53abn EC, and an overlap of morphological representations between POLE mut and MMRd EC. im4MEC highlighted low tumor-stroma ratio as a potential novel characteristic feature of NSMP EC. p53abn cases predicted as imMMRd showed inflammatory morphology and better prognosis; NSMP predicted as imp53abn showed nuclear atypia and worse prognosis; MMRd predicted as im POLE mut had excellent prognosis. Interpretation: We present the first interpretable DL model for H&E-based prediction of the molecular EC classification. im4MEC robustly identified morpho-molecular correlates and enables further prognostic refinement of EC patients. Trial Registration: The PORTEC clinical trials are registered at clinicaltrials.gov PORTEC-3 with identifier: NCT00411138 (this trial is used for as independent testset for our model) PORTEC-2 with identifier: NCT00376844, PORTEC-1 was published in the Lancet in 2000 (attached) and ran 1990-1997. It was not registered in clinicaltrials.gov Funding: The Hanarth Foundation. V.H. Koelzer reports grants from the Swiss Federal Institute of Technology Strategic Focus Area: Personalized Health and Related Technologies PHRT and the Promedica Foundation (F-87701-41-01) during the conduct of the study. Declaration of Interest: All authors declare that they have no conflicts of interest in relation to this paper. Ethical Approval: The protocol was approved by the Protocol Review Committee of the Dutch Cancer Society and by the medical ethics committees of the University Hospital Rotterdam/Daniel den Hoed Cancer Centre (DDHCC) and of the participating centres. Keywords: deep learning, Endometrial Cancer, Molecular classification, Morphological features, Prognostic refinement, POLEmut EC, MMRd EC, NSMP EC, p53abn EC, whole slide images, Histopathology image

The evolving role of morphology in endometrial cancer diagnostics: From histopathology and molecular testing towards integrative data analysis by deep learning

Endometrial cancer (EC) diagnostics is evolving into a system in which molecular aspects are increasingly important. The traditional histological subtype-driven classification has shifted to a molecular-based classification that stratifies EC into DNA polymerase epsilon mutated (POLEmut), mismatch repair deficient (MMRd), and p53 abnormal (p53abn), and the remaining EC as no specific molecular profile (NSMP). The molecular EC classification has been implemented in the World Health Organization 2020 classification and the 2021 European treatment guidelines, as it serves as a better basis for patient management. As a result, the integration of the molecular class with histopathological variables has become a critical focus of recent EC research. Pathologists have observed and described several morphological characteristics in association with specific genomic alterations, but these appear insufficient to accurately classify patients according to molecular subgroups. This requires pathologists to rely on molecular ancillary tests in routine workup. In this new era, it has become increasingly challenging to assign clinically relevant weights to histological and molecular features on an individual patient basis. Deep learning (DL) technology opens new options for the integrative analysis of multi-modal image and molecular datasets with clinical outcomes. Proof-of-concept studies in other cancers showed promising accuracy in predicting molecular alterations from H&E-stained tumor slide images. This suggests that some morphological characteristics that are associated with molecular alterations could be identified in EC, too, expanding the current understanding of the molecular-driven EC classification. Here in this review, we report the morphological characteristics of the molecular EC classification currently identified in the literature. Given the new challenges in EC diagnostics, this review discusses, therefore, the potential supportive role that DL could have, by providing an outlook on all relevant studies using DL on histopathology images in various cancer types with a focus on EC. Finally, we touch upon how DL might shape the management of future EC patients. Keywords: computer vision; deep learning; endometrial carcinoma; histopathology image; molecular classification; phenotype; tumour morphology; whole slide image

Evaluation of deep learning training strategies for the classification of bone marrow cell images

Background and Objective: The classification of bone marrow (BM) cells by light mi- croscopy is an important cornerstone of hematological diagnosis, performed thousands of times a day by highly trained specialists in laboratories worldwide. As the manual evaluation of blood or BM smears is very time-consuming and prone to inter-observer variation, new reliable automated systems are needed. Methods: We aim to improve the automatic classification performance of hematolog- ical cell types. Therefore, we evaluate four state-of-the-art Convolutional Neural Net- work (CNN) architectures on a dataset of 171, 374 microscopic cytological single-cell images obtained from BM smears from 945 patients diagnosed with a variety of hema- tological diseases. We further evaluate the effect of an in-domain vs. out-of-domain pre-training, and assess whether class activation maps provide human-interpretable ex- planations for the models’ predictions. Results: The best performing pre-trained model (Regnet y 32gf) yields a mean pre- cision, recall, and F1 scores of 0.787 ± 0.060, 0.755 ± 0.061, and 0.762 ± 0.050, re- spectively. This is a 53.5% improvement in precision and 7.3% improvement in recall over previous results with CNNs (ResNeXt-50) that were trained from scratch. The out-of-domain pre-training apparently yields general feature extractors/filters that ap- ply very well to the BM cell classification use case. The class activation maps on cell types with characteristic morphological features were found to be consistent with the explanations of a human domain expert. For example, the Auer rods in the cytoplasm were the predictive cellular feature for correctly classified images of faggot cells. Conclusions: Our study provides data that can help hematology laboratories to choose the optimal training strategy for blood cell classification deep learning mod- els to improve computer-assisted blood and bone marrow cell identification. It also highlights the need for more specific training data, i.e. images of difficult-to-classify classes, including cells labeled with disease information

miR-579-3p Controls Hepatocellular Carcinoma Formation by Regulating the Phosphoinositide 3-Kinase-Protein Kinase B Pathway in Chronically Inflamed Liver

Chronic liver inflammation causes continuous liver damage with progressive liver fibrosis and cirrhosis, which may eventually lead to hepatocellular carcinoma (HCC). Whereas the 10-year incidence for HCC in patients with cirrhosis is approximately 20%, many of these patients remain tumor free for their entire lives. Clarifying the mechanisms that define the various outcomes of chronic liver inflammation is a key aspect in HCC research. In addition to a wide variety of contributing factors, microRNAs (miRNAs) have also been shown to be engaged in promoting liver cancer. Therefore, we wanted to characterize miRNAs that are involved in the development of HCC, and we designed a longitudinal study with formalin-fixed and paraffin-embedded liver biopsy samples from several pathology institutes from Switzerland. We examined the miRNA expression by nCounterNanostring technology in matched nontumoral liver tissue from patients developing HCC (n = 23) before and after HCC formation in the same patient. Patients with cirrhosis (n = 26) remaining tumor free within a similar time frame served as a control cohort. Comparison of the two cohorts revealed that liver tissue from patients developing HCC displayed a down-regulation of miR-579-3p as an early step in HCC development, which was further confirmed in a validation cohort. Correlation with messenger RNA expression profiles further revealed that miR-579-3p directly attenuated phosphatidylinositol-4,5-bisphosphate 3-kinase catalytic subunit alpha (PIK3CA) expression and consequently protein kinase B (AKT) and phosphorylated AKT. In vitro experiments and the use of clustered regularly interspaced short palindromic repeats (CRISPR)/Cas9 technology confirmed that miR-579-3p controlled cell proliferation and cell migration of liver cancer cell lines. Conclusion: Liver tissues from patients developing HCC revealed changes in miRNA expression. miR-579-3p was identified as a novel tumor suppressor regulating phosphoinositide 3-kinase-AKT signaling at the early stages of HCC development

Towards a national strategy for digital pathology in Switzerland.

Precision medicine is entering a new era of digital diagnostics; the availability of integrated digital pathology (DP) and structured clinical datasets has the potential to become a key catalyst for biomedical research, education and business development. In Europe, national programs for sharing of this data will be crucial for the development, testing, and validation of machine learning-enabled tools supporting clinical decision-making. Here, the Swiss Digital Pathology Consortium (SDiPath) discusses the creation of a Swiss Digital Pathology Infrastructure (SDPI), which aims to develop a unified national DP network bringing together the Swiss Personalized Health Network (SPHN) with Swiss university hospitals and subsequent inclusion of cantonal and private institutions. This effort builds on existing developments for the national implementation of structured pathology reporting. Opening this national infrastructure and data to international researchers in a sequential rollout phase can enable the large-scale integration of health data and pooling of resources for research purposes and clinical trials. Therefore, the concept of a SDPI directly synergizes with the priorities of the European Commission communication on the digital transformation of healthcare on an international level, and with the aims of the Swiss State Secretariat for Economic Affairs (SECO) for advancing research and innovation in the digitalization domain. SDPI directly addresses the needs of existing national and international research programs in neoplastic and non-neoplastic diseases by providing unprecedented access to well-curated clinicopathological datasets for the development and implementation of novel integrative methods for analysis of clinical outcomes and treatment response. In conclusion, a SDPI would facilitate and strengthen inter-institutional collaboration in technology, clinical development, business and research at a national and international scale, promoting improved patient care via precision medicine

Update on the current opinion, status and future development of digital pathology in Switzerland in light of COVID-19

AIMS The transition from analogue to digital pathology (DP) in Switzerland has coincided with the COVID-19 crisis. The Swiss Digital Pathology Consortium conducted a national survey to assess the experience of pathologists in dealing with the challenges of the pandemic and how this has influenced the outlook and adoption of DP. METHODS A survey containing 20 questions relating to DP, personal experiences and challenges during the pandemic was addressed to Swiss pathologists at different experience stages in private practice, community and university hospitals. RESULTS All 74 respondents were pathologists, with 81.1% reporting more than 5 years of diagnostic service experience. 32.5% reported having read 100 digital slides or more in a diagnostic context. 39.2% reported using whole slide imaging systems at their primary workplace. Key DP use cases before the COVID-19 lockdown were tumour boards (39.2%), education (60.8%) and research (44.6%), with DP used for primary diagnosis in 13.5%. During the COVID-19 crisis, the use of DP for primary diagnostics more than doubled (30% vs 13.5%), with internal consults as important drivers (22.5% vs 16.5%), while research use (25% vs 44.6%) and external consults (17.5% vs 41.9%) strongly decreased. Key challenges identified included a lack of established standard operating procedures and availability of specialised hardware and software. CONCLUSIONS This survey indicates that the crisis acted as a catalyst in promoting DP adoption in centres where basic workflows were already established while posing major technical and organisational challenges in institutions that were at an early stage of DP implementation

Colorectal choriocarcinoma in a patient with probable Lynch syndrome

Background: Personalized therapy of colorectal cancer (CRC) is influenced by morphological, molecular and host-related factors. Here we report the comprehensive clinicopathological and molecular analysis of a pure extra-gestational colorectal choriocarcinoma in a patient with probable Lynch syndrome.Case presentation: A 61 year old female with history of gastric cancer at age 36 presented with a transmurally invasive tumor of the right hemicolon and liver metastasis. A right hemicolectomy was performed. Histopathological analysis showed a mixed trophoblastic and syncytiotrophoblastic differentiation, consistent with choriocarcinoma. Disease progression was rapid under oxaliplatin, capecitabine, irinotecan (XELOXIRI) and bevacizumab. Molecular phenotyping identified loss of the mismatch-repair (MMR) protein PMS2, microsatellite instability, a lack of MLH1 promoter methylation and lack of of BRAF mutation suggestive of Lynch-Syndrome. Targeted next generation sequencing revealed an Ataxia Telangiectasia Mutated (ATM p.P604S) missense mutation. A bleomycin, etoposide and cisplatin (BEP) treatment protocol targeting germ-cell neoplasia lead to disease remission and prolonged survival of 34 months.Conclusions: Comprehensive immunohistochemical and genetic testing is essential to identify uncommon cancers possibly related to Lynch syndrome. For rare tumors, personalized therapeutic approaches should take both molecular and morphological information into account.Key words: Colorectal cancer, choriocarcinoma, histopathology, prognostic factors, Lynch syndrome, microsatellite instability, ataxia telangiectasia mutated, molecular pathology, next generation sequencing, personalized medicin