[Work in progress] Scalable, out-of-the box segmentation of individual particles from mineral samples acquired with micro CT

Minerals are indispensable for a functioning modern society. Yet, their supply is limited causing a need for optimizing their exploration and extraction both from ores and recyclable materials. Typically, these processes must be meticulously adapted to the precise properties of the processed particles, an extensive characterization of their shapes, appearances as well as the overall material composition. Current approaches perform this analysis based on bulk segmentation and characterization of particles imaged with a micro CT, and rely on rudimentary postprocessing techniques to separate touching particles. However, due to their inability to reliably perform this separation as well as the need to retrain or reconfigure methods for each new image, these approaches leave untapped potential to be leveraged. Here, we propose ParticleSeg3D, an instance segmentation method that is able to extract individual particles from large micro CT images taken from mineral samples embedded in an epoxy matrix. Our approach is based on the powerful nnU-Net framework, introduces a particle size normalization, makes use of a border-core representation to enable instance segmentation and is trained with a large dataset containing particles of numerous different materials and minerals. We demonstrate that ParticleSeg3D can be applied out-of-the box to a large variety of particle types, including materials and appearances that have not been part of the training set. Thus, no further manual annotations and retraining are required when applying the method to new mineral samples, enabling substantially higher scalability of experiments than existing methods. Our code and dataset are made publicly available

GaNDLF: A Generally Nuanced Deep Learning Framework for Scalable End-to-End Clinical Workflows in Medical Imaging

Deep Learning (DL) has greatly highlighted the potential impact of optimized machine learning in both the scientific and clinical communities. The advent of open-source DL libraries from major industrial entities, such as TensorFlow (Google), PyTorch (Facebook), and MXNet (Apache), further contributes to DL promises on the democratization of computational analytics. However, increased technical and specialized background is required to develop DL algorithms, and the variability of implementation details hinders their reproducibility. Towards lowering the barrier and making the mechanism of DL development, training, and inference more stable, reproducible, and scalable, without requiring an extensive technical background, this manuscript proposes the Generally Nuanced Deep Learning Framework (GaNDLF). With built-in support for k-fold cross-validation, data augmentation, multiple modalities and output classes, and multi-GPU training, as well as the ability to work with both radiographic and histologic imaging, GaNDLF aims to provide an end-to-end solution for all DL-related tasks, to tackle problems in medical imaging and provide a robust application framework for deployment in clinical workflows

Fake News Detection by Image Montage Recognition

Fake news have been a problem for multiple years now and in addition to this fake images that accompany them are becoming increasingly a problem too. The aim of such fake images is to back up the fake message itself and make it appear authentic. For this purpose, more and more images such as photo-montages are used, which have been spliced from several images. This can be used to defame people by putting them in unfavorable situations or the other way around as propaganda by making them appear more important. In addition, montages may have been altered with noise and other manipulations to make an automatic recognition more difficult. In order to take action against such montages and still detect them automated, a concept based on feature detection is developed. Furthermore, an indexing of the features is carried out by means of a nearest neighbor algorithm in order to be able to quickly compare a high number of images. Afterwards, images suspected to be a montage are reviewed by a verifier. This concept is implemented and evaluated with two feature detectors. Even montages that have been manipulated with different methods are identified as such in an average of 100 milliseconds with a probability of mostly over 90%

M3d-CAM: a PyTorch Library to Generate 3D Attention Maps for Medical Deep Learning

Deep learning models achieve state-of-the-art results in a wide array of medical imaging problems. Yet the lack of interpretability of deep neural networks is a primary concern for medical practitioners and poses a considerable barrier before the deployment of such models in clinical practice. Several techniques have been developed for visualizing the decision process of DNNs. However, few implementations are openly available for the popular PyTorch library, and existing implementations are often limited to two-dimensional data and classification models. We present M3d-CAM, an easy easy to use library for generating attention maps of CNN-based PyTorch models for both 2D and 3D data, and applicable to both classification and segmentation models. The attention maps can be generated with multiple methods: Guided Backpropagation, Grad-CAM, Guided Grad-CAM and Grad-CAM++. The maps visualize the regions in the input data that most heavily influence the model prediction at a certain layer. Only a single line of code is sufficient for generating attention maps for a model, making M3d-CAM a plug-and-play solution that requires minimal previous knowledge

Detecting When Pre-trained nnU-Net Models Fail Silently for Covid-19 Lung Lesion Segmentation

Automatic segmentation of lung lesions in computer tomography has the potential to ease the burden of clinicians during the Covid-19 pandemic. Yet predictive deep learning models are not trusted in the clinical routine due to failing silently in out-of-distribution (OOD) data. We propose a lightweight OOD detection method that exploits the Mahalanobis distance in the feature space. The proposed approach can be seamlessly integrated into state-of-the-art segmentation pipelines without requiring changes in model architecture or training procedure, and can therefore be used to assess the suitability of pre-trained models to new data. We validate our method with a patch-based nnU-Net architecture trained with a multi-institutional dataset and find that it effectively detects samples that the model segments incorrectly

Distance-based detection of out-of-distribution silent failures for Covid-19 lung lesion segmentation

Automatic segmentation of ground glass opacities and consolidations in chest computer tomography (CT) scanscan potentially ease the burden of radiologists during times of high resource utilisation. However, deep learningmodels are not trusted in the clinical routine due to failing silently on out-of-distribution (OOD) data. Wepropose a lightweight OOD detection method that leverages the Mahalanobis distance in the feature space andseamlessly integrates into state-of-the-art segmentation pipelines. The simple approach can even augment pretrainedmodels with clinically relevant uncertainty quantification. We validate our method across four chest CTdistribution shifts and two magnetic resonance imaging applications, namely segmentation of the hippocampusand the prostate. Our results show that the proposed method effectively detects far- and near-OOD samplesacross all explored scenarios

MIC-DKFZ/nnUNet: nnU-Net v2.2

Reworked inference code to be more flexible Minor bug fixe

GaNDLF: the generally nuanced deep learning framework for scalable end-to-end clinical workflows

<h2>What's Changed</h2> <ul> <li>Version update for development by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/648</li> <li>Added citation file by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/654</li> <li>Added new optimizers by @AdiSir05 in https://github.com/mlcommons/GaNDLF/pull/646</li> <li>Allow histology patches to be extracted without ground truth labels by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/657</li> <li>Added metric calculation from CLI by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/663</li> <li>Added a few segmentation metrics by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/661</li> <li>Repository badges have been updated by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/667</li> <li>Added instructions on creating new tutorials by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/664</li> <li>Ensure parameters are built into the model dictionary by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/673</li> <li>Calculating penalty after all compute objects are initialized by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/675</li> <li>Add image similarity metrics by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/669</li> <li>Allow the penalty and class weights in the config to be used by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/677</li> <li>Added documentation related to OpenFL by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/683</li> <li>Add MLCube wrapper for metrics API by @hasan7n in https://github.com/mlcommons/GaNDLF/pull/681</li> <li>Adding mechanism to curate each extracted patch by @shubhaminnani in https://github.com/mlcommons/GaNDLF/pull/653</li> <li>Added mask to SSIM function call by @FelixSteinbauer in https://github.com/mlcommons/GaNDLF/pull/685</li> <li>Removed history file by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/690</li> <li>Updated the metrics output by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/687</li> <li>Update docker image name in workflow by @hasan7n in https://github.com/mlcommons/GaNDLF/pull/692</li> <li>Fixed plotting function for final stats by @Geeks-Sid in https://github.com/mlcommons/GaNDLF/pull/691</li> <li>Fixed import for collect stats by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/694</li> <li>HED augmentations for digital pathology image by @Geeks-Sid in https://github.com/mlcommons/GaNDLF/pull/649</li> <li>Added focal loss by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/696</li> <li>Added a temporary fix for protobuf by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/702</li> <li>Use torchmetric PSNR implementation and argument ordering by @FelixSteinbauer in https://github.com/mlcommons/GaNDLF/pull/693</li> <li>Introduced percentile normalization for synthesis challenge metrics by @FelixSteinbauer in https://github.com/mlcommons/GaNDLF/pull/700</li> <li>Upgrade openvino version to latest by @Geeks-Sid in https://github.com/mlcommons/GaNDLF/pull/699</li> <li>Additional PSNR evaluations for the normalized synthesis case by @FelixSteinbauer in https://github.com/mlcommons/GaNDLF/pull/703</li> <li>Improved formatting by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/707</li> <li>Updated checkout version and test names for clarity by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/708</li> <li>Updated default options for sgd by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/709</li> <li>Added matthews correlation coefficient loss by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/706</li> <li>Using tuples for PSNR datarange by @FelixSteinbauer in https://github.com/mlcommons/GaNDLF/pull/712</li> <li>Deploy model entrypoint by @hasan7n in https://github.com/mlcommons/GaNDLF/pull/711</li> <li>Added parameter to toggle NCC computation by @FelixSteinbauer in https://github.com/mlcommons/GaNDLF/pull/717</li> <li>Adding second classification tutorial by @vavali08 in https://github.com/mlcommons/GaNDLF/pull/698</li> <li>Minor code refactoring by @tosemml in https://github.com/mlcommons/GaNDLF/pull/719</li> <li>Combined writing and temp file creation in a single step by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/720</li> <li>Update usage information for anonymizer by @sanashah007 in https://github.com/mlcommons/GaNDLF/pull/716</li> <li>Move unit testing data to the mlcommons storage by @sarthakpati in https://github.com/mlcommons/GaNDLF/pull/722</li> <li>Fixed model saving when git repo not found by @scap3yvt in https://github.com/mlcommons/GaNDLF/pull/729</li> <li>Removing dev from version for tagging by @scap3yvt in https://github.com/mlcommons/GaNDLF/pull/731</li> </ul> <h2>New Contributors</h2> <ul> <li>@AdiSir05 made their first contribution in https://github.com/mlcommons/GaNDLF/pull/646</li> <li>@shubhaminnani made their first contribution in https://github.com/mlcommons/GaNDLF/pull/653</li> <li>@FelixSteinbauer made their first contribution in https://github.com/mlcommons/GaNDLF/pull/685</li> <li>@vavali08 made their first contribution in https://github.com/mlcommons/GaNDLF/pull/698</li> <li>@tosemml made their first contribution in https://github.com/mlcommons/GaNDLF/pull/719</li> <li>@sanashah007 made their first contribution in https://github.com/mlcommons/GaNDLF/pull/716</li> <li>@scap3yvt made their first contribution in https://github.com/mlcommons/GaNDLF/pull/729</li> </ul> <p><strong>Full Changelog</strong>: https://github.com/mlcommons/GaNDLF/compare/0.0.16...0.0.17</p>If you use this software, please cite it using this manuscript