Iterative annotation to ease neural network training: Specialized machine learning in medical image analysis

Neural networks promise to bring robust, quantitative analysis to medical fields, but adoption is limited by the technicalities of training these networks. To address this translation gap between medical researchers and neural networks in the field of pathology, we have created an intuitive interface which utilizes the commonly used whole slide image (WSI) viewer, Aperio ImageScope (Leica Biosystems Imaging, Inc.), for the annotation and display of neural network predictions on WSIs. Leveraging this, we propose the use of a human-in-the-loop strategy to reduce the burden of WSI annotation. We track network performance improvements as a function of iteration and quantify the use of this pipeline for the segmentation of renal histologic findings on WSIs. More specifically, we present network performance when applied to segmentation of renal micro compartments, and demonstrate multi-class segmentation in human and mouse renal tissue slides. Finally, to show the adaptability of this technique to other medical imaging fields, we demonstrate its ability to iteratively segment human prostate glands from radiology imaging data.Comment: 15 pages, 7 figures, 2 supplemental figures (on the last page

Generative modeling for label-free glomerular modeling and classification.

Generative modeling using GANs has gained traction in machine learning literature, as training does not require labeled datasets. This is perfect for applications in biological datasets, where large labeled datasets are often difficult and expensive to acquire. However, generative models offer no easy way to encode real images into feature-sets, something that is desirable for network explainability and may yield potentially informative image features. For this reason, we test a VAE-GAN architecture for label-free modeling of glomerular structural features. We show that this network can generate realistic looking synthetic images, and be used to interpolate between images. To prove the biological relevance of the network encodings, we classify small-labeled sets of encoded glomeruli by biopsy Tervaert class and for the presence of sclerosis, obtaining a Cohen's kappa values of 0.87 and 0.78 respectfully

Computational detection and quantification of human and mouse neutrophil extracellular traps in flow cytometry and confocal microscopy

Neutrophil extracellular traps (NETs) are extracellular defense mechanisms used by neutrophils, where chromatin is expelled together with histones and granular/cytoplasmic proteins. They have become an immunology hotspot, implicated in infections, but also in a diverse array of diseases such as systemic lupus erythematosus, diabetes, and cancer. However, the precise assessment of in vivo relevance in different disease settings has been hampered by limited tools to quantify occurrence of extracellular traps in experimental models and human samples. To expedite progress towards improved quantitative tools, we have developed computational pipelines to identify extracellular traps from an in vitro human samples visualized using the ImageStream® platform (Millipore Sigma, Darmstadt, Germany), and confocal images of an in vivo mouse disease model of aspergillus fumigatus pneumonia. Our two in vitro methods, tested on n = 363/n =145 images respectively, achieved holdout sensitivity/specificity 0.98/0.93 and 1/0.92. Our unsupervised method for thin lung tissue sections in murine fungal pneumonia achieved sensitivity/specificity 0.99/0.98 in n = 14 images. Our supervised method for thin lung tissue classified NETs with sensitivity/specificity 0.86/0.90. We expect that our approach will be of value for researchers, and have application in infectious and inflammatory diseases

Impact of Sr-Containing Secondary Phases on Oxide Conductivity in Solid-Oxide Electrolyzer Cells

Solid-oxide electrolyzer cells (SOECs) based on yttria-stabilized zirconia (YSZ) oxide electrolytes are devices capable of producing hydrogen with excess thermal energy. However, beginning with initial materials sintering and extending through electrochemical aging, Sr diffusion within the Gd-doped CeO2 (GDC) barrier layer has been observed to lead to the formation of unwanted secondary phases such as SrO and SrZrO3. To establish the impact of these phases on SOEC performance, we perform firstprinciples calculations to determine secondary phase bulk oxide conductivities and compared them to that of the YSZ electrolyte. We find that SrO has a low conductivity arising from poor mobility and a low concentration of oxygen vacancies (V_O^2+), and its presence in SOECs should therefore be avoided as much as possible. SrZrO3 also has a lower oxide conductivity than YSZ; however, this discrepancy is primarily due to lower V_O^2+ concentrations, not V_O^2+ mobility. We find Y-doping to be a viable strategy to increase V_O^2+ concentrations in SrZrO3, with 16% substitution of Y on the Zr site leading to an ionic conductivity on par with that of YSZ. Energy dispersive x-ray spectroscopy obtained using scanning transmission electron microscropy on cross-sections of SOECs indicates that Y is the most common minority element present in SrZrO3 forming near the GDC—YSZ interface. Thus, we expect SrZrO3 to be rich in V_O^2+ and not to hinder long-term device performance. These results from our combined computational–experimental analysis can inform future materials engineering strategies designed to limit the detrimental effects of Sr-induced secondary phase formation on SOEC performance

Automated Reference Kidney Histomorphometry using a Panoptic Segmentation Neural Network Correlates to Patient Demographics and Creatinine.

Reference histomorphometric data of healthy human kidneys are lacking due to laborious quantitation requirements. We leveraged deep learning to investigate the relationship of histomorphometry with patient age, sex, and serum creatinine in a multinational set of reference kidney tissue sections. A panoptic segmentation neural network was developed and used to segment viable and sclerotic glomeruli, cortical and medullary interstitia, tubules, and arteries/arterioles in digitized images of 79 periodic acid-Schiff (PAS)-stained human nephrectomy sections showing minimal pathologic changes. Simple morphometrics (e.g., area, radius, density) were measured from the segmented classes. Regression analysis was used to determine the relationship of histomorphometric parameters with age, sex, and serum creatinine. The model achieved high segmentation performance for all test compartments. We found that the size and density of nephrons, arteries/arterioles, and the baseline level of interstitium vary significantly among healthy humans, with potentially large differences between subjects from different geographic locations. Nephron size in any region of the kidney was significantly dependent on patient creatinine. Slight differences in renal vasculature and interstitium were observed between sexes. Finally, glomerulosclerosis percentage increased and cortical density of arteries/arterioles decreased as a function of age. We show that precise measurements of kidney histomorphometric parameters can be automated. Even in reference kidney tissue sections with minimal pathologic changes, several histomorphometric parameters demonstrated significant correlation to patient demographics and serum creatinine. These robust tools support the feasibility of deep learning to increase efficiency and rigor in histomorphometric analysis and pave the way for future large-scale studies

Computational Integration of Renal Histology and Urinary Proteomics using Neural Networks

Histological image data and molecular profiles provide context into renal condition. Often, a biopsy is drawn to diagnose or monitor a suspected kidney problem. However, molecular profiles can go beyond a pathologist's ability to see and diagnose. Using AI, we computationally incorporated urinary proteomic profiles with microstructural morphology from renal biopsy to investigate new and existing molecular links to image phenotypes. We studied whole slide images of periodic acid-Schiff stained renal biopsies from 56 DN patients matched with 2,038 proteins measured from each patient's urine. Using Seurat, we identified differentially expressed proteins in patients that developed end-stage renal disease within 2 years of biopsy. Glomeruli, globally sclerotic glomeruli, and tubules were segmented from WSI using our previously published HAIL pipeline. For each glomerulus, 315 handcrafted digital image features were measured, and for tubules, 207 features. We trained fully connected networks to predict urinary protein measurements that were differentially expressed between patients who did/ did not progress to ESRD within 2 years of biopsy. The input to this network was either glomerular or tubular histomorphological features in biopsy. Trained network weights were used as a proxy to rank which morphological features correlated most highly with specific urinary proteins. We identified significant image feature-protein pairs by ranking network weights by magnitude. We also looked at which features on average were most significant in predicting proteins. For both glomeruli and tubules, RGB color values and variance in PAS(+) areas (specifically basement membrane for tubules) were, on average, more predictive of molecular profiles than other features. There is a strong connection between molecular profile and image phenotype, which can be elucidated through computational methods. These discovered links can provide insight to disease pathways, and discover new factors contributing to incidence and progression.N

Data from: An integrated iterative annotation technique for easing neural network training in medical image analysis

Neural networks promise to bring robust, quantitative analysis to medical fields. However, their adoption is limited by the technicalities of training these networks and the required volume and quality of human-generated annotations. To address this gap in the field of pathology, we have created an intuitive interface for data annotation and the display of neural network predictions within a commonly used digital pathology whole-slide viewer. This strategy used a ‘human-in-the-loop’ to reduce the annotation burden. We demonstrate that segmentation of human and mouse renal micro compartments is repeatedly improved when humans interact with automatically generated annotations throughout the training process. Finally, to show the adaptability of this technique to other medical imaging fields, we demonstrate its ability to iteratively segment human prostate glands from radiology imaging data