Assessment of breast pathologies using nonlinear microscopy

Rapid intraoperative assessment of breast excision specimens is clinically important because up to 40% of patients undergoing breast-conserving cancer surgery require reexcision for positive or close margins. We demonstrate nonlinear microscopy (NLM) for the assessment of benign and malignant breast pathologies in fresh surgical specimens. A total of 179 specimens from 50 patients was imaged with NLM using rapid extrinsic nuclear staining with acridine orange and intrinsic second harmonic contrast generation from collagen. Imaging was performed on fresh, intact specimens without the need for fixation, embedding, and sectioning required for conventional histopathology. A visualization method to aid pathological interpretation is presented that maps NLM contrast from two-photon fluorescence and second harmonic signals to features closely resembling histopathology using hematoxylin and eosin staining. Mosaicking is used to overcome trade-offs between resolution and field of view, enabling imaging of subcellular features over square-centimeter specimens. After NLM examination, specimens were processed for standard paraffin-embedded histology using a protocol that coregistered histological sections to NLM images for paired assessment. Blinded NLM reading by three pathologists achieved 95.4% sensitivity and 93.3% specificity, compared with paraffin-embedded histology, for identifying invasive cancer and ductal carcinoma in situ versus benign breast tissue. Interobserver agreement was κ = 0.88 for NLM and κ = 0.89 for histology. These results show that NLM achieves high diagnostic accuracy, can be rapidly performed on unfixed specimens, and is a promising method for intraoperative margin assessment.National Institutes of Health (U.S.) (Grant R01-CA178636-01)National Institutes of Health (U.S.) (Grant R01-CA75289-16)United States. Air Force Office of Scientific Research (Grant FA9550-10-1-0551)United States. Air Force Office of Scientific Research (Grant FA9550-12-1-0499)National Institutes of Health (U.S.) (National Research Service Award Postdoctoral Fellowship F32-CA165484

Artificial intelligence for detection of microsatellite instability in colorectal cancer-a multicentric analysis of a pre-screening tool for clinical application.

BACKGROUND Microsatellite instability (MSI)/mismatch repair deficiency (dMMR) is a key genetic feature which should be tested in every patient with colorectal cancer (CRC) according to medical guidelines. Artificial intelligence (AI) methods can detect MSI/dMMR directly in routine pathology slides, but the test performance has not been systematically investigated with predefined test thresholds. METHOD We trained and validated AI-based MSI/dMMR detectors and evaluated predefined performance metrics using nine patient cohorts of 8343 patients across different countries and ethnicities. RESULTS Classifiers achieved clinical-grade performance, yielding an area under the receiver operating curve (AUROC) of up to 0.96 without using any manual annotations. Subsequently, we show that the AI system can be applied as a rule-out test: by using cohort-specific thresholds, on average 52.73% of tumors in each surgical cohort [total number of MSI/dMMR = 1020, microsatellite stable (MSS)/ proficient mismatch repair (pMMR) = 7323 patients] could be identified as MSS/pMMR with a fixed sensitivity at 95%. In an additional cohort of N = 1530 (MSI/dMMR = 211, MSS/pMMR = 1319) endoscopy biopsy samples, the system achieved an AUROC of 0.89, and the cohort-specific threshold ruled out 44.12% of tumors with a fixed sensitivity at 95%. As a more robust alternative to cohort-specific thresholds, we showed that with a fixed threshold of 0.25 for all the cohorts, we can rule-out 25.51% in surgical specimens and 6.10% in biopsies. INTERPRETATION When applied in a clinical setting, this means that the AI system can rule out MSI/dMMR in a quarter (with global thresholds) or half of all CRC patients (with local fine-tuning), thereby reducing cost and turnaround time for molecular profiling

Fully transformer-based biomarker prediction from colorectal cancer histology: a large-scale multicentric study

Background: Deep learning (DL) can extract predictive and prognostic biomarkers from routine pathology slides in colorectal cancer. For example, a DL test for the diagnosis of microsatellite instability (MSI) in CRC has been approved in 2022. Current approaches rely on convolutional neural networks (CNNs). Transformer networks are outperforming CNNs and are replacing them in many applications, but have not been used for biomarker prediction in cancer at a large scale. In addition, most DL approaches have been trained on small patient cohorts, which limits their clinical utility. Methods: In this study, we developed a new fully transformer-based pipeline for end-to-end biomarker prediction from pathology slides. We combine a pre-trained transformer encoder and a transformer network for patch aggregation, capable of yielding single and multi-target prediction at patient level. We train our pipeline on over 9,000 patients from 10 colorectal cancer cohorts. Results: A fully transformer-based approach massively improves the performance, generalizability, data efficiency, and interpretability as compared with current state-of-the-art algorithms. After training on a large multicenter cohort, we achieve a sensitivity of 0.97 with a negative predictive value of 0.99 for MSI prediction on surgical resection specimens. We demonstrate for the first time that resection specimen-only training reaches clinical-grade performance on endoscopic biopsy tissue, solving a long-standing diagnostic problem. Interpretation: A fully transformer-based end-to-end pipeline trained on thousands of pathology slides yields clinical-grade performance for biomarker prediction on surgical resections and biopsies. Our new methods are freely available under an open source license

Transformer-based biomarker prediction from colorectal cancer histology: A large-scale multicentric study.

Deep learning (DL) can accelerate the prediction of prognostic biomarkers from routine pathology slides in colorectal cancer (CRC). However, current approaches rely on convolutional neural networks (CNNs) and have mostly been validated on small patient cohorts. Here, we develop a new transformer-based pipeline for end-to-end biomarker prediction from pathology slides by combining a pre-trained transformer encoder with a transformer network for patch aggregation. Our transformer-based approach substantially improves the performance, generalizability, data efficiency, and interpretability as compared with current state-of-the-art algorithms. After training and evaluating on a large multicenter cohort of over 13,000 patients from 16 colorectal cancer cohorts, we achieve a sensitivity of 0.99 with a negative predictive value of over 0.99 for prediction of microsatellite instability (MSI) on surgical resection specimens. We demonstrate that resection specimen-only training reaches clinical-grade performance on endoscopic biopsy tissue, solving a long-standing diagnostic problem

Formalizing Both Refraction-Based and Sequential Executions of Production Rule Programs

Abstract Production systems are declarative, in that they do not ex-plicitly specify the control flow. Yet, the concept of a production system does not include the definition of a given control strategy. The control be-tween rules in a production rule program is, in practice, defined by each implementation of a production rule engine. Engines have traditionally been implemented using the Rete algorithm. Since the turn of the cen-tury, however, production systems have evolved into industrial products known as Business Rules Management Systems (BRMS). BRMS have in-troduced new compilation and execution schemes, which are often called sequential in contrast with the incremental behavior of Rete. This change in execution scheme came with a change in semantics for rule programs. In this paper, we propose a formal description of the execution of pro-duction rule programs. Existing descriptions either ignore the control strategy, or assume a Rete semantics. Ours isolates the handling of rule eligibility in the control strategy, which allows us to describe the sequen-tial execution semantics of rule programs, as well as the Rete semantics, and others.

Occupancy and fractal dimension analyses of the spatial distribution of cytotoxic (CD8+) T cells infiltrating the tumor microenvironment in triple negative breast cancer

Favorable outcomes have been associated with high densities of tumor infiltrating lymphocytes (TILs) such as cytotoxic (CD8+) T cells. However, the clinical signifi- cance of the spatial distribution of TILs is less well understood. We have developed novel statistical techniques to characterize the spatial distribution of TILs at various length scales. These include a box counting method that we call “occupancy” and novel applications of fractal dimensions. We apply these techniques to the spatial distribution of CD8+ T cells in the tumor microenvironment of tissue resected from 35 triple negative breast cancer patients. We find that there is a distinct difference in the spatial distribution of CD8+ T cells between good clinical outcome (no recurrence within at least 5 years of diagnosis) and poor clinical outcome (recurrence within 3 years of diagnosis). The statistical significance of the difference between good and poor outcome in the occupancy, fractal dimension (FD), and FD difference of CD8+ T cells is comparable to that of the CD8+ T cell density. Even when we randomly exclude some of the cells so that the images have the same cell density, we still find that the fractal dimension at short length scales is correlated with cancer recurrence, implying that the actual spatial distribution of CD8+ cells, and not just the CD8+ cell density, is associated with clinical outcome. The occupancy and FD difference indicate that the CD8+ T cells are more spatially dispersed in good outcome and more aggregated in poor outcome. We discuss possible interpretations

Physics Approaches to the Spatial Distribution of Immune Cells in Tumors

The goal of immunotherapy is to mobilize the immune system to kill cancer cells. Immunotherapy is more effective and, in general, the prognosis is better, when more immune cells infiltrate the tumor. We explore the question of whether the spatial distribution rather than just the density of immune cells in the tumor is important in forecasting whether cancer recurs. After reviewing previous work on this issue, we introduce a novel application of maximum entropy to quantify the spatial distribution of discrete point-like objects. We apply our approach to B and T cells in images of tumor tissue taken from triple negative breast cancer (TNBC) patients. We find that the immune cells are more spatially dispersed in good clinical outcome (no recurrence of cancer within at least 5 years of diagnosis) compared to poor clinical outcome (recurrence within 3 years of diagnosis). Our results highlight the importance of spatial distribution of immune cells within tumors with regard to clinical outcome, and raise new questions on their role in cancer recurrence