The Intrinsic Manifolds of Radiological Images and their Role in Deep Learning

The manifold hypothesis is a core mechanism behind the success of deep learning, so understanding the intrinsic manifold structure of image data is central to studying how neural networks learn from the data. Intrinsic dataset manifolds and their relationship to learning difficulty have recently begun to be studied for the common domain of natural images, but little such research has been attempted for radiological images. We address this here. First, we compare the intrinsic manifold dimensionality of radiological and natural images. We also investigate the relationship between intrinsic dimensionality and generalization ability over a wide range of datasets. Our analysis shows that natural image datasets generally have a higher number of intrinsic dimensions than radiological images. However, the relationship between generalization ability and intrinsic dimensionality is much stronger for medical images, which could be explained as radiological images having intrinsic features that are more difficult to learn. These results give a more principled underpinning for the intuition that radiological images can be more challenging to apply deep learning to than natural image datasets common to machine learning research. We believe rather than directly applying models developed for natural images to the radiological imaging domain, more care should be taken to developing architectures and algorithms that are more tailored to the specific characteristics of this domain. The research shown in our paper, demonstrating these characteristics and the differences from natural images, is an important first step in this direction.Comment: preprint version, accepted for MICCAI 2022 (25th International Conference on Medical Image Computing and Computer Assisted Intervention). 8 pages (+ author names + references + supplementary), 4 figures. Code available at https://github.com/mazurowski-lab/radiologyintrinsicmanifold

A systematic study of the foreground-background imbalance problem in deep learning for object detection

The class imbalance problem in deep learning has been explored in several studies, but there has yet to be a systematic analysis of this phenomenon in object detection. Here, we present comprehensive analyses and experiments of the foreground-background (F-B) imbalance problem in object detection, which is very common and caused by small, infrequent objects of interest. We experimentally study the effects of different aspects of F-B imbalance (object size, number of objects, dataset size, object type) on detection performance. In addition, we also compare 9 leading methods for addressing this problem, including Faster-RCNN, SSD, OHEM, Libra-RCNN, Focal-Loss, GHM, PISA, YOLO-v3, and GFL with a range of datasets from different imaging domains. We conclude that (1) the F-B imbalance can indeed cause a significant drop in detection performance, (2) The detection performance is more affected by F-B imbalance when fewer training data are available, (3) in most cases, decreasing object size leads to larger performance drop than decreasing number of objects, given the same change in the ratio of object pixels to non-object pixels, (6) among all selected methods, Libra-RCNN and PISA demonstrate the best performance in addressing the issue of F-B imbalance. (7) When the training dataset size is large, the choice of method is not impactful (8) Soft-sampling methods, including focal-loss, GHM, and GFL, perform fairly well on average but are relatively unstable

Medical Image Segmentation with InTEnt: Integrated Entropy Weighting for Single Image Test-Time Adaptation

Test-time adaptation (TTA) refers to adapting a trained model to a new domain during testing. Existing TTA techniques rely on having multiple test images from the same domain, yet this may be impractical in real-world applications such as medical imaging, where data acquisition is expensive and imaging conditions vary frequently. Here, we approach such a task, of adapting a medical image segmentation model with only a single unlabeled test image. Most TTA approaches, which directly minimize the entropy of predictions, fail to improve performance significantly in this setting, in which we also observe the choice of batch normalization (BN) layer statistics to be a highly important yet unstable factor due to only having a single test domain example. To overcome this, we propose to instead integrate over predictions made with various estimates of target domain statistics between the training and test statistics, weighted based on their entropy statistics. Our method, validated on 24 source/target domain splits across 3 medical image datasets surpasses the leading method by 2.9% Dice coefficient on average.Comment: Code and pre-trained weights: https://github.com/mazurowski-lab/single-image-test-time-adaptatio

Segment Anything Model for Medical Image Analysis: an Experimental Study

Training segmentation models for medical images continues to be challenging due to the limited availability and acquisition expense of data annotations. Segment Anything Model (SAM) is a foundation model trained on over 1 billion annotations, predominantly for natural images, that is intended to be able to segment the user-defined object of interest in an interactive manner. Despite its impressive performance on natural images, it is unclear how the model is affected when shifting to medical image domains. Here, we perform an extensive evaluation of SAM's ability to segment medical images on a collection of 11 medical imaging datasets from various modalities and anatomies. In our experiments, we generated point prompts using a standard method that simulates interactive segmentation. Experimental results show that SAM's performance based on single prompts highly varies depending on the task and the dataset, i.e., from 0.1135 for a spine MRI dataset to 0.8650 for a hip x-ray dataset, evaluated by IoU. Performance appears to be high for tasks including well-circumscribed objects with unambiguous prompts and poorer in many other scenarios such as segmentation of tumors. When multiple prompts are provided, performance improves only slightly overall, but more so for datasets where the object is not contiguous. An additional comparison to RITM showed a much better performance of SAM for one prompt but a similar performance of the two methods for a larger number of prompts. We conclude that SAM shows impressive performance for some datasets given the zero-shot learning setup but poor to moderate performance for multiple other datasets. While SAM as a model and as a learning paradigm might be impactful in the medical imaging domain, extensive research is needed to identify the proper ways of adapting it in this domain.Comment: Link to our code: https://github.com/mazurowski-lab/segment-anything-medica

Automated Grading of Radiographic Knee Osteoarthritis Severity Combined with Joint Space Narrowing

The assessment of knee osteoarthritis (KOA) severity on knee X-rays is a central criteria for the use of total knee arthroplasty. However, this assessment suffers from imprecise standards and a remarkably high inter-reader variability. An algorithmic, automated assessment of KOA severity could improve overall outcomes of knee replacement procedures by increasing the appropriateness of its use. We propose a novel deep learning-based five-step algorithm to automatically grade KOA from posterior-anterior (PA) views of radiographs: (1) image preprocessing (2) localization of knees joints in the image using the YOLO v3-Tiny model, (3) initial assessment of the severity of osteoarthritis using a convolutional neural network-based classifier, (4) segmentation of the joints and calculation of the joint space narrowing (JSN), and (5), a combination of the JSN and the initial assessment to determine a final Kellgren-Lawrence (KL) score. Furthermore, by displaying the segmentation masks used to make the assessment, our algorithm demonstrates a higher degree of transparency compared to typical "black box" deep learning classifiers. We perform a comprehensive evaluation using two public datasets and one dataset from our institution, and show that our algorithm reaches state-of-the art performance. Moreover, we also collected ratings from multiple radiologists at our institution and showed that our algorithm performs at the radiologist level. The software has been made publicly available at https://github.com/MaciejMazurowski/osteoarthritis-classification

SegmentAnyBone: A Universal Model that Segments Any Bone at Any Location on MRI

Magnetic Resonance Imaging (MRI) is pivotal in radiology, offering non-invasive and high-quality insights into the human body. Precise segmentation of MRIs into different organs and tissues would be highly beneficial since it would allow for a higher level of understanding of the image content and enable important measurements, which are essential for accurate diagnosis and effective treatment planning. Specifically, segmenting bones in MRI would allow for more quantitative assessments of musculoskeletal conditions, while such assessments are largely absent in current radiological practice. The difficulty of bone MRI segmentation is illustrated by the fact that limited algorithms are publicly available for use, and those contained in the literature typically address a specific anatomic area. In our study, we propose a versatile, publicly available deep-learning model for bone segmentation in MRI across multiple standard MRI locations. The proposed model can operate in two modes: fully automated segmentation and prompt-based segmentation. Our contributions include (1) collecting and annotating a new MRI dataset across various MRI protocols, encompassing over 300 annotated volumes and 8485 annotated slices across diverse anatomic regions; (2) investigating several standard network architectures and strategies for automated segmentation; (3) introducing SegmentAnyBone, an innovative foundational model-based approach that extends Segment Anything Model (SAM); (4) comparative analysis of our algorithm and previous approaches; and (5) generalization analysis of our algorithm across different anatomical locations and MRI sequences, as well as an external dataset. We publicly release our model at https://github.com/mazurowski-lab/SegmentAnyBone.Comment: 15 pages, 15 figure

Effects of Compound Probiotics on Growth Performance, Serum Biochemical and Immune Indices, Antioxidant Capacity, and Intestinal Tissue Morphology of Shaoxing Duck

This experiment was conducted to investigate the effects of compound probiotics on growth performance, serum biochemical and immune indices, antioxidant capacity, and the intestinal tissue morphology of Shaoxing ducks. A total of 640 1-day-old healthy Shaoxing ducks of similar body weight were randomly divided into two treatment groups with eight replicates each and forty ducks per replicate. The ducks were fed a basal diet (Ctrl) or a basal diet supplemented with 0.15% compound probiotics (CP) for 125 d. The results revealed that the live body weight (BW; day 85 and 125) and the average daily gain (ADG; 28–85 and 85–125 d) of the CP group were significantly higher (p p p p p p p p p p p p p < 0.05) on day 28. In summary, the compound probiotics improved the growth performance, increased serum biochemical and immune indices, increased antioxidant capacity, and improved the intestinal tissue morphology of Shaoxing ducks

Integrated transcriptome and microbiome analyses of residual feed intake in ducks during high production period

ABSTRACT: Residual feed intake (RFI) is a crucial parameter for assessing the feeding efficiency of poultry. Minimizing RFI can enhance feed utilization and reduce costs. In this study, 315 healthy female ducks were individually housed in cages. Growth performance was monitored during the high laying period, from 290 to 325 d of age. The cecal transcriptome and microbiome of 12 ducks with high RFI and 12 with low residual feed intake (LRFI) were analyzed. Regarding growth performance, the LRFI group exhibited significantly lower RFI, feed conversion ratio (FCR), and feed intake (Fi) compared to the HRFI group (p 0.05). Microbiome analysis demonstrated that RFI impacted gut microbial abundance, particularly affecting metabolism and disease-related microorganisms such as Romboutsia, Enterococcus, and Megamonas funiformis. Transcriptome analysis revealed that varying RFI changed the expression of genes related to glucose metabolism and lipid metabolism, including APOA1, G6PC1, PCK1, and PLIN1. The integrated analysis indicated that host genes were closely linked to the microbiota and primarily function in lipid metabolism, which may enhance feeding efficiency by influencing metabolism and maintaining gut homeostasis

A publicly available deep learning model and dataset for segmentation of breast, fibroglandular tissue, and vessels in breast MRI

Abstract Breast density, or the amount of fibroglandular tissue (FGT) relative to the overall breast volume, increases the risk of developing breast cancer. Although previous studies have utilized deep learning to assess breast density, the limited public availability of data and quantitative tools hinders the development of better assessment tools. Our objective was to (1) create and share a large dataset of pixel-wise annotations according to well-defined criteria, and (2) develop, evaluate, and share an automated segmentation method for breast, FGT, and blood vessels using convolutional neural networks. We used the Duke Breast Cancer MRI dataset to randomly select 100 MRI studies and manually annotated the breast, FGT, and blood vessels for each study. Model performance was evaluated using the Dice similarity coefficient (DSC). The model achieved DSC values of 0.92 for breast, 0.86 for FGT, and 0.65 for blood vessels on the test set. The correlation between our model’s predicted breast density and the manually generated masks was 0.95. The correlation between the predicted breast density and qualitative radiologist assessment was 0.75. Our automated models can accurately segment breast, FGT, and blood vessels using pre-contrast breast MRI data. The data and the models were made publicly available

Comparative analysis of the hypothalamus transcriptome of laying ducks with different residual feeding intake

ABSTRACT: Feed costs account for approximately 60 to 70% of the cost of poultry farming, and feed utilization is closely related to the profitability of the poultry industry. To understand the causes of the differences in feeding in Shan Partridge ducks, we compared the hypothalamus transcriptome profiles of 2 groups of ducks using RNA-seq. The 2 groups were: 1) low-residual feed intake (LRFI) group with low feed intake but high feed efficiency, and 2) high-residual feed intake (HRFI) group with high feed intake but low feed efficiency. We found 78 DEGs were enriched in 9 differential Kyoto Encyclopedia of Genes and Genome (KEGG) pathways, including neuroactive ligand-receptor interaction, GABAergic synapse, nitrogen metabolism, cAMP signaling pathway, calcium signaling pathway, nitrogen metabolism, tyrosine metabolism, ovarian steroidogenesis, and gluconeogenesis. To further identify core genes among the 78 DEGs, we performed protein-protein interaction and coexpression network analyses. After comprehensive analysis and experimental validation, 4 core genes, namely, glucagon (GCG), cholecystokinin (CCK), gamma-aminobutyric acid type A receptor subunit delta (GABRD), and gamma-aminobutyric acid type A receptor subunit beta1 (GABRB1), were identified as potential core genes responsible for the difference in residual feeding intake between the 2 breeds. We also investigated the level of cholecystokinin (CCK), neuropeptide Y (NPY), peptide YY (PYY), ghrelin, and glucagon-like peptide1 (GLP-1) hormones in the sera of Shan Partridge ducks at different feeding levels and found that there was a difference between the 2 groups with respect to GLP-1 and NPY levels. The findings will serve as a reference for future research on the feeding efficiency of Shan Partridge ducks and assist in promoting their genetic breeding