Iterative annotation to ease neural network training: Specialized machine learning in medical image analysis

Neural networks promise to bring robust, quantitative analysis to medical fields, but adoption is limited by the technicalities of training these networks. To address this translation gap between medical researchers and neural networks in the field of pathology, we have created an intuitive interface which utilizes the commonly used whole slide image (WSI) viewer, Aperio ImageScope (Leica Biosystems Imaging, Inc.), for the annotation and display of neural network predictions on WSIs. Leveraging this, we propose the use of a human-in-the-loop strategy to reduce the burden of WSI annotation. We track network performance improvements as a function of iteration and quantify the use of this pipeline for the segmentation of renal histologic findings on WSIs. More specifically, we present network performance when applied to segmentation of renal micro compartments, and demonstrate multi-class segmentation in human and mouse renal tissue slides. Finally, to show the adaptability of this technique to other medical imaging fields, we demonstrate its ability to iteratively segment human prostate glands from radiology imaging data.Comment: 15 pages, 7 figures, 2 supplemental figures (on the last page

Generative deep learning in digital pathology workflows

Funding: Supported by the Sir James Mackenzie Institute for Early Diagnosis, University of St Andrews and Industrial Centre for Artificial Intelligence Research in Digital Diagnostics (grant number TS/S013121/1).Many modern histopathology laboratories are in the process of digitising their workflows. Once images of the tissue exist as digital data, it becomes feasible to research the augmentation or automation of clinical reporting and diagnosis. The application of modern computer vision techniques, based on Deep Learning, promise systems that can identify pathologies in slide images with a high degree of accuracy. Generative modelling is an approach to machine learning and deep learning that can be used to transform and generate data. It can be applied to a broad range of tasks within digital pathology including the removal of color and intensity artefacts, the adaption of images in one domain into those of another, and the generation of synthetic digital tissue samples. This review provides an introduction to the topic, considers these applications, and discusses some future directions for generative models within histopathology.PostprintPeer reviewe

Deep-Learning based segmentation and quantification in experimental kidney histopathology

BACKGROUND: Nephropathologic analyses provide important outcomes-related data in experiments with the animal models that are essential for understanding kidney disease pathophysiology. Precision medicine increases the demand for quantitative, unbiased, reproducible, and efficient histopathologic analyses, which will require novel high-throughput tools. A deep learning technique, the convolutional neural network, is increasingly applied in pathology because of its high performance in tasks like histology segmentation. METHODS: We investigated use of a convolutional neural network architecture for accurate segmentation of periodic acid-Schiff-stained kidney tissue from healthy mice and five murine disease models and from other species used in preclinical research. We trained the convolutional neural network to segment six major renal structures: glomerular tuft, glomerulus including Bowman\u27s capsule, tubules, arteries, arterial lumina, and veins. To achieve high accuracy, we performed a large number of expert-based annotations, 72,722 in total. RESULTS: Multiclass segmentation performance was very high in all disease models. The convolutional neural network allowed high-throughput and large-scale, quantitative and comparative analyses of various models. In disease models, computational feature extraction revealed interstitial expansion, tubular dilation and atrophy, and glomerular size variability. Validation showed a high correlation of findings with current standard morphometric analysis. The convolutional neural network also showed high performance in other species used in research-including rats, pigs, bears, and marmosets-as well as in humans, providing a translational bridge between preclinical and clinical studies. CONCLUSIONS: We developed a deep learning algorithm for accurate multiclass segmentation of digital whole-slide images of periodic acid-Schiff-stained kidneys from various species and renal disease models. This enables reproducible quantitative histopathologic analyses in preclinical models that also might be applicable to clinical studies

Improving cancer subtype diagnosis and grading using clinical decision support system based on computer-aided tissue image analysis

This research focuses towards the development of a clinical decision support system (CDSS) based on cellular and tissue image analysis and classification system that improves consistency and facilitates the clinical decision making process. In a typical cancer examination, pathologists make diagnosis by manually reading morphological features in patient biopsy images, in which cancer biomarkers are highlighted by using different staining techniques. This process is subjected to pathologist's training and experience, especially when the same cancer has several subtypes (i.e. benign tumor subtype vs. malignant subtype) and the same cancer tissue biopsy contains heterogeneous morphologies in different locations. The variability in pathologist's manual reading may result in varying cancer diagnosis and treatment. This Ph.D. research aims to reduce the subjectivity and variation existing in traditional histo-pathological reading of patient tissue biopsy slides through Computer-Aided Diagnosis (CAD). Using the CAD, quantitative molecular profiling of cancer biomarkers of stained biopsy images are obtained by extracting and analyzing texture and cellular structure features. In addition, cancer sub-type classification and a semi-automatic grade scoring (i.e. clinical decision making) for improved consistency over a large number of cancer subtype images can be performed. The CAD tools do have their own limitations and in certain cases the clinicians, however, prefer systems which are flexible and take into account their individuality when necessary by providing some control rather than fully automated system. Therefore, to be able to introduce CDSS in health care, we need to understand users' perspectives and preferences on the new information technology. This forms as the basis for this research where we target to present the quantitative information acquired through the image analysis, annotate the images and provide suitable visualization which can facilitate the process of decision making in a clinical setting.PhDCommittee Chair: Dr. May D. Wang; Committee Member: Dr. Andrew N. Young; Committee Member: Dr. Anthony J. Yezzi; Committee Member: Dr. Edward J. Coyle; Committee Member: Dr. Paul Benkese

Promoting Student Success in the Flipped Online Classroom: Learning and Accountability Through Homework Strategies

As online and hybrid classes have become increasingly more prevalent in higher education, the flipped classroom structure has emerged as a viable, evidence-based, option for healthcare programs. In a flipped classroom, students view pre-recorded video lectures and complete reading assignments before class, and synchronous class time can then be used for active learning activities. Class sessions offer opportunities for group work, review of complex content, and access to instructor assistance with assignments. To effectively implement a flipped classroom approach, students must prepare prior to class time. One method for encouraging student accountability is to assign preparatory homework. This experimental study compared two types of accountability homework on measures of achievement, satisfaction, ease of use, and perceived learning from two types of assignments: concept maps or question-and-answer homework. Study participants included 46 first year occupational therapy students attending an online foundational occupational therapy course. Treatment included weekly completion of either a concept map or a set of three question-and-answer homework assignments over a period of three weeks. Findings suggested that accountability homework assignments of either type were helpful in promoting achievement. Results further revealed that satisfaction and perceived learning were greater in the concept map group as compared to the question-and-answer group. It is recommended that occupational therapy and other allied health instructors use accountability homework to reinforce student learning in the flipped classroom. The use of concept map assignments in particular has the potential to improve schema acquisition, critical thinking, and deep learning, which in turn can support educational success

The effect of neural network architecture on virtual H&E staining : Systematic assessment of histological feasibility

Conventional histopathology has relied on chemical staining for over a century. The staining process makes tissue sections visible to the human eye through a tedious and labor-intensive procedure that alters the tissue irreversibly, preventing repeated use of the sample. Deep learning-based virtual staining can potentially alleviate these shortcomings. Here, we used standard brightfield microscopy on unstained tissue sections and studied the impact of increased network capacity on the resulting virtually stained H&E images. Using the generative adversarial neural network model pix2pix as a baseline, we observed that replacing simple convolutions with dense convolution units increased the structural similarity score, peak signal-to-noise ratio, and nuclei reproduction accuracy. We also demonstrated highly accurate reproduction of histology, especially with increased network capacity, and demonstrated applicability to several tissues. We show that network architecture optimization can improve the image translation accuracy of virtual H&E staining, highlighting the potential of virtual staining in streamlining histopathological analysis.publishedVersionPeer reviewe