Rotation Invariance and Extensive Data Augmentation: A Strategy for the MItosis DOmain Generalization (MIDOG) Challenge

Automated detection of mitotic figures in histopathology images is a challenging task: here, we present the different steps that describe the strategy we applied to participate in the MIDOG 2021 competition. The purpose of the competition was to evaluate the generalization of solutions to images acquired with unseen target scanners (hidden for the participants) under the constraint of using training data from a limited set of four independent source scanners. Given this goal and constraints, we joined the challenge by proposing a straight-forward solution based on a combination of state-of-the-art deep learning methods with the aim of yielding robustness to possible scanner-related distributional shifts at inference time. Our solution combines methods that were previously shown to be efficient for mitosis detection: hard negative mining, extensive data augmentation, rotation-invariant convolutional networks. We trained five models with different splits of the provided dataset. The subsequent classifiers produced F1-score with a mean and standard deviation of 0.747±0.032 on the test splits. The resulting ensemble constitutes our candidate algorithm: its automated evaluation on the preliminary test set of the challenge returned a F1-score of 0.6828

Evaluation of deep learning training strategies for the classification of bone marrow cell images

BACKGROUND AND OBJECTIVE The classification of bone marrow (BM) cells by light microscopy is an important cornerstone of hematological diagnosis, performed thousands of times a day by highly trained specialists in laboratories worldwide. As the manual evaluation of blood or BM smears is very time-consuming and prone to inter-observer variation, new reliable automated systems are needed. METHODS We aim to improve the automatic classification performance of hematological cell types. Therefore, we evaluate four state-of-the-art Convolutional Neural Network (CNN) architectures on a dataset of 171,374 microscopic cytological single-cell images obtained from BM smears from 945 patients diagnosed with a variety of hematological diseases. We further evaluate the effect of an in-domain vs. out-of-domain pre-training, and assess whether class activation maps provide human-interpretable explanations for the models' predictions. RESULTS The best performing pre-trained model (Regnet_y_32gf) yields a mean precision, recall, and F1 scores of 0.787±0.060, 0.755±0.061, and 0.762±0.050, respectively. This is a 53.5% improvement in precision and 7.3% improvement in recall over previous results with CNNs (ResNeXt-50) that were trained from scratch. The out-of-domain pre-training apparently yields general feature extractors/filters that apply very well to the BM cell classification use case. The class activation maps on cell types with characteristic morphological features were found to be consistent with the explanations of a human domain expert. For example, the Auer rods in the cytoplasm were the predictive cellular feature for correctly classified images of faggot cells. CONCLUSIONS Our study provides data that can help hematology laboratories to choose the optimal training strategy for blood cell classification deep learning models to improve computer-assisted blood and bone marrow cell identification. It also highlights the need for more specific training data, i.e. images of difficult-to-classify classes, including cells labeled with disease information

Interpretable Deep Learning Predicts the Molecular Endometrial Cancer Classification from H&E Images: A Combined Analysis of the Portec Randomized Clinical Trials

Background: Endometrial Cancer (EC) is molecularly classified into the POLE- mutated (POLE mut), mismatch-repair deficient (MMRd), p53 abnormal (p53abn) and no specific molecular profile (NSMP) subgroups. We aimed to develop an interpretable DL pipeline for image-based prediction of the four-class molecular EC classification (im4MEC), to identify morpho-molecular correlates and refine EC prognostication. Methods: Diagnostic hematoxylin and Eosin (H&E)-stained slides from 2028 EC patients of the combined PORTEC-1-2,-3 randomized trials and four clinical cohorts were included. im4MEC combined self-supervised learning and an attention mechanism to achieve optimal performance. Tiles with highest attention scores were reviewed to identify morpho-molecular correlates. Predictions of a nuclear classification DL model served to derive interpretable morphological features and their relative contribution in the profiling of each molecular class. Prognostic refinement was explored through morphological and Kaplan-Meier-based survival analyses. Findings: im4MEC achieved a macro-average area under the receiver-operating-characteristic curve (AUROC) on cross-validation of 0·874±0·189, and 0·876 on the independent test set PORTEC-3 with highest performance of 0·928 among p53abn EC. Overall recurrence by image-based molecular class was significantly different in PORTEC-3 (p=1·e-04). Top-attended tiles indicated a significant association between dense lymphocyte infiltrates and POLE mut and MMRd EC, high nuclear atypia and p53abn EC, and an overlap of morphological representations between POLE mut and MMRd EC. im4MEC highlighted low tumor-stroma ratio as a potential novel characteristic feature of NSMP EC. p53abn cases predicted as imMMRd showed inflammatory morphology and better prognosis; NSMP predicted as imp53abn showed nuclear atypia and worse prognosis; MMRd predicted as im POLE mut had excellent prognosis. Interpretation: We present the first interpretable DL model for H&E-based prediction of the molecular EC classification. im4MEC robustly identified morpho-molecular correlates and enables further prognostic refinement of EC patients. Trial Registration: The PORTEC clinical trials are registered at clinicaltrials.gov PORTEC-3 with identifier: NCT00411138 (this trial is used for as independent testset for our model) PORTEC-2 with identifier: NCT00376844, PORTEC-1 was published in the Lancet in 2000 (attached) and ran 1990-1997. It was not registered in clinicaltrials.gov Funding: The Hanarth Foundation. V.H. Koelzer reports grants from the Swiss Federal Institute of Technology Strategic Focus Area: Personalized Health and Related Technologies PHRT and the Promedica Foundation (F-87701-41-01) during the conduct of the study. Declaration of Interest: All authors declare that they have no conflicts of interest in relation to this paper. Ethical Approval: The protocol was approved by the Protocol Review Committee of the Dutch Cancer Society and by the medical ethics committees of the University Hospital Rotterdam/Daniel den Hoed Cancer Centre (DDHCC) and of the participating centres. Keywords: deep learning, Endometrial Cancer, Molecular classification, Morphological features, Prognostic refinement, POLEmut EC, MMRd EC, NSMP EC, p53abn EC, whole slide images, Histopathology image

The evolving role of morphology in endometrial cancer diagnostics: From histopathology and molecular testing towards integrative data analysis by deep learning

Endometrial cancer (EC) diagnostics is evolving into a system in which molecular aspects are increasingly important. The traditional histological subtype-driven classification has shifted to a molecular-based classification that stratifies EC into DNA polymerase epsilon mutated (POLEmut), mismatch repair deficient (MMRd), and p53 abnormal (p53abn), and the remaining EC as no specific molecular profile (NSMP). The molecular EC classification has been implemented in the World Health Organization 2020 classification and the 2021 European treatment guidelines, as it serves as a better basis for patient management. As a result, the integration of the molecular class with histopathological variables has become a critical focus of recent EC research. Pathologists have observed and described several morphological characteristics in association with specific genomic alterations, but these appear insufficient to accurately classify patients according to molecular subgroups. This requires pathologists to rely on molecular ancillary tests in routine workup. In this new era, it has become increasingly challenging to assign clinically relevant weights to histological and molecular features on an individual patient basis. Deep learning (DL) technology opens new options for the integrative analysis of multi-modal image and molecular datasets with clinical outcomes. Proof-of-concept studies in other cancers showed promising accuracy in predicting molecular alterations from H&E-stained tumor slide images. This suggests that some morphological characteristics that are associated with molecular alterations could be identified in EC, too, expanding the current understanding of the molecular-driven EC classification. Here in this review, we report the morphological characteristics of the molecular EC classification currently identified in the literature. Given the new challenges in EC diagnostics, this review discusses, therefore, the potential supportive role that DL could have, by providing an outlook on all relevant studies using DL on histopathology images in various cancer types with a focus on EC. Finally, we touch upon how DL might shape the management of future EC patients. Keywords: computer vision; deep learning; endometrial carcinoma; histopathology image; molecular classification; phenotype; tumour morphology; whole slide image

Towards IID representation learning and its application on biomedical data

Due to the heterogeneity of real-world data, the widely accepted independent and identically distributed (IID) assumption has been criticized in recent studies on causality. In this paper, we argue that instead of being a questionable assumption, IID is a fundamental task-relevant property that needs to be learned. Consider k independent random vectors Xi=1,…,k, we elaborate on how a variety of different causal questions can be reformulated to learning a task-relevant function ϕ that induces IID among Zi:=ϕ∘Xi, which we term IID representation learning. For proof of concept, we examine the IID representation learning on Out-of-Distribution (OOD) generalization tasks. Concretely, by utilizing the representation obtained via the learned function that induces IID, we conduct prediction of molecular characteristics (molecular prediction) on two biomedical datasets with real-world distribution shifts introduced by a) preanalytical variation and b) sampling protocol. To enable reproducibility and for comparison to the state-of-the-art (SOTA) methods, this is done by following the OOD benchmarking guidelines recommended from WILDS. Compared to the SOTA baselines supported in WILDS, the results confirm the superior performance of IID representation learning on OOD tasks. The code is publicly accessible via this https URL

Case report: Surgical repair of a large tracheo-esophageal fistula in a patient with post-transplant esophageal lymphoproliferative disorder

Introduction and importance The management of large malignant tracheo-esophageal fistulas (TEF) is not standardized. Herein, we report a case with a malignant TEF associated with esophageal post-transplant lymphoproliferative disorder (PTLD) for whom we successfully performed a surgical repair. This contributes to the knowledge on how to treat large acquired malignant TEFs. Case presentation A 69-year old male presented with a one-week history of fever, productive cough and bilateral coarse crackles. In addition, he described a weight loss of 10 kg during the past three months. The patient's history included a kidney transplantation twenty years ago. Esophagogastroduodenoscopy with a biopsy of the esophagus was performed nine days before. Histopathology showed a PTLD of diffuse large B-cell lymphoma subtype. Subsequent diagnostics revealed a progressive TEF (approx. 2.0 × 1.5 cm) 3.0 cm above the carina. PET-CT scan showed an esophagus with slight tracer uptake in the middle third (approx. 11.5 cm length, SUV max 7.4). After decision against stenting, transthoracic subtotal esophagectomy with closure of the tracheal mouth of the fistula by a pedicled flap was performed. PTLD was treated with prednisone and rituximab. Tumor progression (brain metastasis) led to death 95 days after surgery. Clinical discussion The treatment of a malignant TEF is complex and personalized while both the consequences of the esophago-tracheal connection and those of the underlying responsible diagnosis have to be considered concurrently. In this case, we considered surgery as the best treatment option due to a relatively good prognosis of the underlying diagnosis (PTLD) and a large fistula. Esophageal or dual stenting, the treatment of choice for small malignant TEF, would have been associated with a high risk of failure due to the wide trachea, extensively dilated esophagus, proximal location and large diameter of the fistula. Conclusion Surgery can be considered for patients with a large acquired malignant TEF and positive long-term prognosis of the underlying diagnosis. Due to the complexity of TEF management, immediate pre-operative multidisciplinary discussion is advised

Automated causal inference in application to randomized controlled clinical trials

Randomized controlled trials (RCTs) are considered the gold standard for testing causal hypotheses in the clinical domain; however, the investigation of prognostic variables of patient outcome in a hypothesized cause–effect route is not feasible using standard statistical methods. Here we propose a new automated causal inference method (AutoCI) built on the invariant causal prediction (ICP) framework for the causal reinterpretation of clinical trial data. Compared with existing methods, we show that the proposed AutoCI allows one to clearly determine the causal variables of two real-world RCTs of patients with endometrial cancer with mature outcome and extensive clinicopathological and molecular data. This is achieved via suppressing the causal probability of non-causal variables by a wide margin. In ablation studies, we further demonstrate that the assignment of causal probabilities by AutoCI remains consistent in the presence of confounders. In conclusion, these results confirm the robustness and feasibility of AutoCI for future applications in real-world clinical analysis

a retrospective cohort study

Background Metastasis of colorectal cancer (CRC) is directly linked to patient survival. We previously identified the novel gene Metastasis Associated in Colon Cancer 1 (MACC1) in CRC and demonstrated its importance as metastasis inducer and prognostic biomarker. Here, we investigate the geographic expression pattern of MACC1 in colorectal adenocarcinoma and tumor buds in correlation with clinicopathological and molecular features for improvement of survival prognosis. Methods We performed geographic MACC1 expression analysis in tumor center, invasive front and tumor buds on whole tissue sections of 187 well-characterized CRCs by immunohistochemistry. MACC1 expression in each geographic zone was analyzed with Mismatch repair (MMR)-status, BRAF/KRAS- mutations and CpG-island methylation. Results MACC1 was significantly overexpressed in tumor tissue as compared to normal mucosa (p < 0.001). Within colorectal adenocarcinomas, a significant increase of MACC1 from tumor center to front (p = 0.0012) was detected. MACC1 was highly overexpressed in 55% tumor budding cells. Independent of geographic location, MACC1 predicted advanced pT and pN-stages, high grade tumor budding, venous and lymphatic invasion (p < 0.05). High MACC1 expression at the invasive front was decisive for prediction of metastasis (p = 0.0223) and poor survival (p = 0.0217). The geographic pattern of MACC1 did not correlate with MMR-status, BRAF/KRAS- mutations or CpG-island methylation. Conclusion MACC1 is differentially expressed in CRC. At the invasive front, MACC1 expression predicts best aggressive clinicopathological features, tumor budding, metastasis formation and poor survival outcome