Topology-Guided Multi-Class Cell Context Generation for Digital Pathology

In digital pathology, the spatial context of cells is important for cell classification, cancer diagnosis and prognosis. To model such complex cell context, however, is challenging. Cells form different mixtures, lineages, clusters and holes. To model such structural patterns in a learnable fashion, we introduce several mathematical tools from spatial statistics and topological data analysis. We incorporate such structural descriptors into a deep generative model as both conditional inputs and a differentiable loss. This way, we are able to generate high quality multi-class cell layouts for the first time. We show that the topology-rich cell layouts can be used for data augmentation and improve the performance of downstream tasks such as cell classification.Comment: To be published in proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 202

Exascale Deep Learning to Accelerate Cancer Research

Deep learning, through the use of neural networks, has demonstrated remarkable ability to automate many routine tasks when presented with sufficient data for training. The neural network architecture (e.g. number of layers, types of layers, connections between layers, etc.) plays a critical role in determining what, if anything, the neural network is able to learn from the training data. The trend for neural network architectures, especially those trained on ImageNet, has been to grow ever deeper and more complex. The result has been ever increasing accuracy on benchmark datasets with the cost of increased computational demands. In this paper we demonstrate that neural network architectures can be automatically generated, tailored for a specific application, with dual objectives: accuracy of prediction and speed of prediction. Using MENNDL--an HPC-enabled software stack for neural architecture search--we generate a neural network with comparable accuracy to state-of-the-art networks on a cancer pathology dataset that is also $16\times$ faster at inference. The speedup in inference is necessary because of the volume and velocity of cancer pathology data; specifically, the previous state-of-the-art networks are too slow for individual researchers without access to HPC systems to keep pace with the rate of data generation. Our new model enables researchers with modest computational resources to analyze newly generated data faster than it is collected.Comment: Submitted to IEEE Big Dat

Keratin 17 Modulates the Immune Topography of Pancreatic Cancer

BACKGROUND: The immune microenvironment impacts tumor growth, invasion, metastasis, and patient survival and may provide opportunities for therapeutic intervention in pancreatic ductal adenocarcinoma (PDAC). Although never studied as a potential modulator of the immune response in most cancers, Keratin 17 (K17), a biomarker of the most aggressive (basal) molecular subtype of PDAC, is intimately involved in the histogenesis of the immune response in psoriasis, basal cell carcinoma, and cervical squamous cell carcinoma. Thus, we hypothesized that K17 expression could also impact the immune cell response in PDAC, and that uncovering this relationship could provide insight to guide the development of immunotherapeutic opportunities to extend patient survival. METHODS: Multiplex immunohistochemistry (mIHC) and automated image analysis based on novel computational imaging technology were used to decipher the abundance and spatial distribution of T cells, macrophages, and tumor cells, relative to K17 expression in 235 PDACs. RESULTS: K17 expression had profound effects on the exclusion of intratumoral CD8+ T cells and was also associated with decreased numbers of peritumoral CD8+ T cells, CD16+ macrophages, and CD163+ macrophages (p \u3c 0.0001). The differences in the intratumor and peritumoral CD8+ T cell abundance were not impacted by neoadjuvant therapy, tumor stage, grade, lymph node status, histologic subtype, nor KRAS, p53, SMAD4, or CDKN2A mutations. CONCLUSIONS: Thus, K17 expression correlates with major differences in the immune microenvironment that are independent of any tested clinicopathologic or tumor intrinsic variables, suggesting that targeting K17-mediated immune effects on the immune system could restore the innate immunologic response to PDAC and might provide novel opportunities to restore immunotherapeutic approaches for this most deadly form of cancer

Localization in the Crowd with Topological Constraints

We address the problem of crowd localization, i.e., the prediction of dots corresponding to people in a crowded scene. Due to various challenges, a localization method is prone to spatial semantic errors, i.e., predicting multiple dots within a same person or collapsing multiple dots in a cluttered region. We propose a topological approach targeting these semantic errors. We introduce a topological constraint that teaches the model to reason about the spatial arrangement of dots. To enforce this constraint, we define a persistence loss based on the theory of persistent homology. The loss compares the topographic landscape of the likelihood map and the topology of the ground truth. Topological reasoning improves the quality of the localization algorithm especially near cluttered regions. On multiple public benchmarks, our method outperforms previous localization methods. Additionally, we demonstrate the potential of our method in improving the performance in the crowd counting task.Comment: AAAI 202

Evaluating histopathology transfer learning with ChampKit

Histopathology remains the gold standard for diagnosis of various cancers. Recent advances in computer vision, specifically deep learning, have facilitated the analysis of histopathology images for various tasks, including immune cell detection and microsatellite instability classification. The state-of-the-art for each task often employs base architectures that have been pretrained for image classification on ImageNet. The standard approach to develop classifiers in histopathology tends to focus narrowly on optimizing models for a single task, not considering the aspects of modeling innovations that improve generalization across tasks. Here we present ChampKit (Comprehensive Histopathology Assessment of Model Predictions toolKit): an extensible, fully reproducible benchmarking toolkit that consists of a broad collection of patch-level image classification tasks across different cancers. ChampKit enables a way to systematically document the performance impact of proposed improvements in models and methodology. ChampKit source code and data are freely accessible at https://github.com/kaczmarj/champkit .Comment: Submitted to NeurIPS 2022 Track on Datasets and Benchmarks. Source code available at https://github.com/kaczmarj/champki

Keratin 17 modulates the immune topography of pancreatic cancer.

BACKGROUND: The immune microenvironment impacts tumor growth, invasion, metastasis, and patient survival and may provide opportunities for therapeutic intervention in pancreatic ductal adenocarcinoma (PDAC). Although never studied as a potential modulator of the immune response in most cancers, Keratin 17 (K17), a biomarker of the most aggressive (basal) molecular subtype of PDAC, is intimately involved in the histogenesis of the immune response in psoriasis, basal cell carcinoma, and cervical squamous cell carcinoma. Thus, we hypothesized that K17 expression could also impact the immune cell response in PDAC, and that uncovering this relationship could provide insight to guide the development of immunotherapeutic opportunities to extend patient survival. METHODS: Multiplex immunohistochemistry (mIHC) and automated image analysis based on novel computational imaging technology were used to decipher the abundance and spatial distribution of T cells, macrophages, and tumor cells, relative to K17 expression in 235 PDACs. RESULTS: K17 expression had profound effects on the exclusion of intratumoral CD8+ T cells and was also associated with decreased numbers of peritumoral CD8+ T cells, CD16+ macrophages, and CD163+ macrophages (p < 0.0001). The differences in the intratumor and peritumoral CD8+ T cell abundance were not impacted by neoadjuvant therapy, tumor stage, grade, lymph node status, histologic subtype, nor KRAS, p53, SMAD4, or CDKN2A mutations. CONCLUSIONS: Thus, K17 expression correlates with major differences in the immune microenvironment that are independent of any tested clinicopathologic or tumor intrinsic variables, suggesting that targeting K17-mediated immune effects on the immune system could restore the innate immunologic response to PDAC and might provide novel opportunities to restore immunotherapeutic approaches for this most deadly form of cancer

GaNDLF: the generally nuanced deep learning framework for scalable end-to-end clinical workflows

<h2>What's Changed</h2> <ul> <li>Version update for development by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/648</li> <li>Added citation file by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/654</li> <li>Added new optimizers by @AdiSir05 in https://github.com/mlcommons/GaNDLF/pull/646</li> <li>Allow histology patches to be extracted without ground truth labels by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/657</li> <li>Added metric calculation from CLI by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/663</li> <li>Added a few segmentation metrics by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/661</li> <li>Repository badges have been updated by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/667</li> <li>Added instructions on creating new tutorials by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/664</li> <li>Ensure parameters are built into the model dictionary by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/673</li> <li>Calculating penalty after all compute objects are initialized by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/675</li> <li>Add image similarity metrics by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/669</li> <li>Allow the penalty and class weights in the config to be used by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/677</li> <li>Added documentation related to OpenFL by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/683</li> <li>Add MLCube wrapper for metrics API by @hasan7n in https://github.com/mlcommons/GaNDLF/pull/681</li> <li>Adding mechanism to curate each extracted patch by @shubhaminnani in https://github.com/mlcommons/GaNDLF/pull/653</li> <li>Added mask to SSIM function call by @FelixSteinbauer in https://github.com/mlcommons/GaNDLF/pull/685</li> <li>Removed history file by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/690</li> <li>Updated the metrics output by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/687</li> <li>Update docker image name in workflow by @hasan7n in https://github.com/mlcommons/GaNDLF/pull/692</li> <li>Fixed plotting function for final stats by @Geeks-Sid in https://github.com/mlcommons/GaNDLF/pull/691</li> <li>Fixed import for collect stats by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/694</li> <li>HED augmentations for digital pathology image by @Geeks-Sid in https://github.com/mlcommons/GaNDLF/pull/649</li> <li>Added focal loss by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/696</li> <li>Added a temporary fix for protobuf by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/702</li> <li>Use torchmetric PSNR implementation and argument ordering by @FelixSteinbauer in https://github.com/mlcommons/GaNDLF/pull/693</li> <li>Introduced percentile normalization for synthesis challenge metrics by @FelixSteinbauer in https://github.com/mlcommons/GaNDLF/pull/700</li> <li>Upgrade openvino version to latest by @Geeks-Sid in https://github.com/mlcommons/GaNDLF/pull/699</li> <li>Additional PSNR evaluations for the normalized synthesis case by @FelixSteinbauer in https://github.com/mlcommons/GaNDLF/pull/703</li> <li>Improved formatting by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/707</li> <li>Updated checkout version and test names for clarity by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/708</li> <li>Updated default options for sgd by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/709</li> <li>Added matthews correlation coefficient loss by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/706</li> <li>Using tuples for PSNR datarange by @FelixSteinbauer in https://github.com/mlcommons/GaNDLF/pull/712</li> <li>Deploy model entrypoint by @hasan7n in https://github.com/mlcommons/GaNDLF/pull/711</li> <li>Added parameter to toggle NCC computation by @FelixSteinbauer in https://github.com/mlcommons/GaNDLF/pull/717</li> <li>Adding second classification tutorial by @vavali08 in https://github.com/mlcommons/GaNDLF/pull/698</li> <li>Minor code refactoring by @tosemml in https://github.com/mlcommons/GaNDLF/pull/719</li> <li>Combined writing and temp file creation in a single step by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/720</li> <li>Update usage information for anonymizer by @sanashah007 in https://github.com/mlcommons/GaNDLF/pull/716</li> <li>Move unit testing data to the mlcommons storage by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/722</li> <li>Fixed model saving when git repo not found by @scap3yvt in https://github.com/mlcommons/GaNDLF/pull/729</li> <li>Removing dev from version for tagging by @scap3yvt in https://github.com/mlcommons/GaNDLF/pull/731</li> </ul> <h2>New Contributors</h2> <ul> <li>@AdiSir05 made their first contribution in https://github.com/mlcommons/GaNDLF/pull/646</li> <li>@shubhaminnani made their first contribution in https://github.com/mlcommons/GaNDLF/pull/653</li> <li>@FelixSteinbauer made their first contribution in https://github.com/mlcommons/GaNDLF/pull/685</li> <li>@vavali08 made their first contribution in https://github.com/mlcommons/GaNDLF/pull/698</li> <li>@tosemml made their first contribution in https://github.com/mlcommons/GaNDLF/pull/719</li> <li>@sanashah007 made their first contribution in https://github.com/mlcommons/GaNDLF/pull/716</li> <li>@scap3yvt made their first contribution in https://github.com/mlcommons/GaNDLF/pull/729</li> </ul> <p><strong>Full Changelog</strong>: https://github.com/mlcommons/GaNDLF/compare/0.0.16...0.0.17</p>If you use this software, please cite it using this manuscript