66 research outputs found
Adapting Pretrained Vision-Language Foundational Models to Medical Imaging Domains
Multi-modal foundation models are typically trained on millions of pairs of
natural images and text captions, frequently obtained through web-crawling
approaches. Although such models depict excellent generative capabilities, they
do not typically generalize well to specific domains such as medical images
that have fundamentally shifted distributions compared to natural images.
Building generative models for medical images that faithfully depict clinical
context may help alleviate the paucity of healthcare datasets. Thus, in this
study, we seek to research and expand the representational capabilities of
large pretrained foundation models to medical concepts, specifically for
leveraging the Stable Diffusion model to generate domain specific images found
in medical imaging. We explore the sub-components of the Stable Diffusion
pipeline (the variational autoencoder, the U-Net and the text-encoder) to
fine-tune the model to generate medical images. We benchmark the efficacy of
these efforts using quantitative image quality metrics and qualitative
radiologist-driven evaluations that accurately represent the clinical content
of conditional text prompts. Our best-performing model improves upon the stable
diffusion baseline and can be conditioned to insert a realistic-looking
abnormality on a synthetic radiology image, while maintaining a 95% accuracy on
a classifier trained to detect the abnormality.Comment: 17 pages, 8 figure
Contrastive Learning of Medical Visual Representations from Paired Images and Text
Learning visual representations of medical images is core to medical image
understanding but its progress has been held back by the small size of
hand-labeled datasets. Existing work commonly relies on transferring weights
from ImageNet pretraining, which is suboptimal due to drastically different
image characteristics, or rule-based label extraction from the textual report
data paired with medical images, which is inaccurate and hard to generalize. We
propose an alternative unsupervised strategy to learn medical visual
representations directly from the naturally occurring pairing of images and
textual data. Our method of pretraining medical image encoders with the paired
text data via a bidirectional contrastive objective between the two modalities
is domain-agnostic, and requires no additional expert input. We test our method
by transferring our pretrained weights to 4 medical image classification tasks
and 2 zero-shot retrieval tasks, and show that our method leads to image
representations that considerably outperform strong baselines in most settings.
Notably, in all 4 classification tasks, our method requires only 10% as much
labeled training data as an ImageNet initialized counterpart to achieve better
or comparable performance, demonstrating superior data efficiency
The LOINC RSNA radiology playbook - a unified terminology for radiology procedures
Objective:
This paper describes the unified LOINC/RSNA Radiology Playbook and the process by which it was produced.
Methods:
The Regenstrief Institute and the Radiological Society of North America (RSNA) developed a unification plan consisting of six objectives 1) develop a unified model for radiology procedure names that represents the attributes with an extensible set of values, 2) transform existing LOINC procedure codes into the unified model representation, 3) create a mapping between all the attribute values used in the unified model as coded in LOINC (ie, LOINC Parts) and their equivalent concepts in RadLex, 4) create a mapping between the existing procedure codes in the RadLex Core Playbook and the corresponding codes in LOINC, 5) develop a single integrated governance process for managing the unified terminology, and 6) publicly distribute the terminology artifacts.
Results:
We developed a unified model and instantiated it in a new LOINC release artifact that contains the LOINC codes and display name (ie LONG_COMMON_NAME) for each procedure, mappings between LOINC and the RSNA Playbook at the procedure code level, and connections between procedure terms and their attribute values that are expressed as LOINC Parts and RadLex IDs. We transformed all the existing LOINC content into the new model and publicly distributed it in standard releases. The organizations have also developed a joint governance process for ongoing maintenance of the terminology.
Conclusions:
The LOINC/RSNA Radiology Playbook provides a universal terminology standard for radiology orders and results
- …