Iterative annotation to ease neural network training: Specialized machine learning in medical image analysis

Neural networks promise to bring robust, quantitative analysis to medical fields, but adoption is limited by the technicalities of training these networks. To address this translation gap between medical researchers and neural networks in the field of pathology, we have created an intuitive interface which utilizes the commonly used whole slide image (WSI) viewer, Aperio ImageScope (Leica Biosystems Imaging, Inc.), for the annotation and display of neural network predictions on WSIs. Leveraging this, we propose the use of a human-in-the-loop strategy to reduce the burden of WSI annotation. We track network performance improvements as a function of iteration and quantify the use of this pipeline for the segmentation of renal histologic findings on WSIs. More specifically, we present network performance when applied to segmentation of renal micro compartments, and demonstrate multi-class segmentation in human and mouse renal tissue slides. Finally, to show the adaptability of this technique to other medical imaging fields, we demonstrate its ability to iteratively segment human prostate glands from radiology imaging data.Comment: 15 pages, 7 figures, 2 supplemental figures (on the last page

Iterative annotation to ease neural network training: Specialized machine learning in medical image analysis

Neural networks promise to bring robust, quantitative analysis to medical fields, but adoption is limited by the technicalities of training these networks. To address this translation gap between medical researchers and neural networks in the field of pathology, we have created an intuitive interface which utilizes the commonly used whole slide image (WSI) viewer, Aperio ImageScope (Leica Biosystems Imaging, Inc.), for the annotation and display of neural network predictions on WSIs. Leveraging this, we propose the use of a human-in-the-loop strategy to reduce the burden of WSI annotation. We track network performance improvements as a function of iteration and quantify the use of this pipeline for the segmentation of renal histologic findings on WSIs. More specifically, we present network performance when applied to segmentation of renal micro compartments, and demonstrate multi-class segmentation in human and mouse renal tissue slides. Finally, to show the adaptability of this technique to other medical imaging fields, we demonstrate its ability to iteratively segment human prostate glands from radiology imaging data.Comment: 15 pages, 7 figures, 2 supplemental figures (on the last page

The impact of pre- and post-image processing techniques on deep learning frameworks: A comprehensive review for digital pathology image analysis

Recently, deep learning frameworks have rapidly become the main methodology for analyzing medical images. Due to their powerful learning ability and advantages in dealing with complex patterns, deep learning algorithms are ideal for image analysis challenges, particularly in the field of digital pathology. The variety of image analysis tasks in the context of deep learning includes classification (e.g., healthy vs. cancerous tissue), detection (e.g., lymphocytes and mitosis counting), and segmentation (e.g., nuclei and glands segmentation). The majority of recent machine learning methods in digital pathology have a pre- and/or post-processing stage which is integrated with a deep neural network. These stages, based on traditional image processing methods, are employed to make the subsequent classification, detection, or segmentation problem easier to solve. Several studies have shown how the integration of pre- and post-processing methods within a deep learning pipeline can further increase the model's performance when compared to the network by itself. The aim of this review is to provide an overview on the types of methods that are used within deep learning frameworks either to optimally prepare the input (pre-processing) or to improve the results of the network output (post-processing), focusing on digital pathology image analysis. Many of the techniques presented here, especially the post-processing methods, are not limited to digital pathology but can be extended to almost any image analysis field

The impact of pre- and post-image processing techniques on deep learning frameworks: A comprehensive review for digital pathology image analysis.

Recently, deep learning frameworks have rapidly become the main methodology for analyzing medical images. Due to their powerful learning ability and advantages in dealing with complex patterns, deep learning algorithms are ideal for image analysis challenges, particularly in the field of digital pathology. The variety of image analysis tasks in the context of deep learning includes classification (e.g., healthy vs. cancerous tissue), detection (e.g., lymphocytes and mitosis counting), and segmentation (e.g., nuclei and glands segmentation). The majority of recent machine learning methods in digital pathology have a pre- and/or post-processing stage which is integrated with a deep neural network. These stages, based on traditional image processing methods, are employed to make the subsequent classification, detection, or segmentation problem easier to solve. Several studies have shown how the integration of pre- and post-processing methods within a deep learning pipeline can further increase the model's performance when compared to the network by itself. The aim of this review is to provide an overview on the types of methods that are used within deep learning frameworks either to optimally prepare the input (pre-processing) or to improve the results of the network output (post-processing), focusing on digital pathology image analysis. Many of the techniques presented here, especially the post-processing methods, are not limited to digital pathology but can be extended to almost any image analysis field

Computational Image Analysis For Axonal Transport, Phenotypic Profiling, And Digital Pathology

Recent advances in fluorescent probes, microscopy, and imaging platforms have revolutionized biology and medicine, generating multi-dimensional image datasets at unprecedented scales. Traditional, low-throughput methods of image analysis are inadequate to handle the increased “volume, velocity, and variety” that characterize the realm of big data. Thus, biomedical imaging requires a new set of tools, which include advanced computer vision and machine learning algorithms. In this work, we develop computational image analysis solutions to biological questions at the level of single-molecules, cells, and tissues. At the molecular level, we dissect the regulation of dynein-dynactin transport initiation using in vitro reconstitution, single-particle tracking, super-resolution microscopy, live-cell imaging in neurons, and computational modeling. We show that at least two mechanisms regulate dynein transport initiation neurons: (1) cytoplasmic linker proteins, which are regulated by phosphorylation, increase the capture radius around the microtubule, thus reducing the time cargo spends in a diffusive search; and (2) a spatial gradient of tyrosinated alpha-tubulin enriched in the distal axon increases the affinity of dynein-dynactin for microtubules. Together, these mechanisms support a multi-modal recruitment model where interacting layers of regulation provide efficient, robust, and spatiotemporal control of transport initiation. At the cellular level, we develop and train deep residual convolutional neural networks on a large and diverse set of cellular microscopy images. Then, we apply networks trained for one task as deep feature extractors for unsupervised phenotypic profiling in a different task. We show that neural networks trained on one dataset encode robust image phenotypes that are sufficient to cluster subcellular structures by type and separate drug compounds by the mechanism of action, without additional training, supporting the strength and flexibility of this approach. Future applications include phenotypic profiling in image-based screens, where clustering genetic or drug treatments by image phenotypes may reveal novel relationships among genetic or pharmacologic pathways. Finally, at the tissue level, we apply deep learning pipelines in digital pathology to segment cardiac tissue and classify clinical heart failure using whole-slide images of cardiac histopathology. Together, these results demonstrate the power and promise of computational image analysis, computer vision, and deep learning in biological image analysis