Weakly Supervised Learning with Automated Labels from Radiology Reports for Glioma Change Detection

Gliomas are the most frequent primary brain tumors in adults. Glioma change detection aims at finding the relevant parts of the image that change over time. Although Deep Learning (DL) shows promising performances in similar change detection tasks, the creation of large annotated datasets represents a major bottleneck for supervised DL applications in radiology. To overcome this, we propose a combined use of weak labels (imprecise, but fast-to-create annotations) and Transfer Learning (TL). Specifically, we explore inductive TL, where source and target domains are identical, but tasks are different due to a label shift: our target labels are created manually by three radiologists, whereas our source weak labels are generated automatically from radiology reports via NLP. We frame knowledge transfer as hyperparameter optimization, thus avoiding heuristic choices that are frequent in related works. We investigate the relationship between model size and TL, comparing a low-capacity VGG with a higher-capacity ResNeXt model. We evaluate our models on 1693 T2-weighted magnetic resonance imaging difference maps created from 183 patients, by classifying them into stable or unstable according to tumor evolution. The weak labels extracted from radiology reports allowed us to increase dataset size more than 3-fold, and improve VGG classification results from 75% to 82% AUC. Mixed training from scratch led to higher performance than fine-tuning or feature extraction. To assess generalizability, we ran inference on an open dataset (BraTS-2015: 15 patients, 51 difference maps), reaching up to 76% AUC. Overall, results suggest that medical imaging problems may benefit from smaller models and different TL strategies with respect to computer vision datasets, and that report-generated weak labels are effective in improving model performances. Code, in-house dataset and BraTS labels are released.Comment: This work has been submitted as Original Paper to a Journa

Development and validation of a deep learning algorithm for longitudinal change detection in sequential chest X-ray images

최근 그래픽 처리 장치 및 빅데이터가 발전하면서, 의료 영상처리 분야에 인공지능 알고리즘을 접목시켜 주요 질병을 진단 및 검출하는 연구가 활발히 진행되고 있다. 그러나 현재까지 제안된 인공지능 기반의 분류 모델들은 주어진 단일 영상만을 독립적으로 이용하여 결과를 도출한다. 즉, 현재 촬영된 영상은 이전 기록과 잠재적으로 관련이 있음에도 불구하고, 사전에 정의된 비정상 범주만을 예측하는 횡단면적 분석을 시행하는 것이다. 분류 성능은 영상의학 임상의의 수준에 근접하였지만, 병변의 구체적인 변화에 대응하지 못한다. 이는 환자가 이전에 촬영한 영상을 분류하더라도 단순 질병의 출현 유무로는 병변의 변화를 파악하기가 어렵기 때문이다. 특히 일부 주요 질병의 경우, 동일한 질병 내에서도 그 패턴의 종류가 다양할 뿐 만 아니라, 장기간 또는 급성 변화 등 변화 양상이 환자의 임상기록에 따라 매우 다르다. 따라서 횡단면적 분석만으로는 시간에 따른 특정 변화를 검출하는 것은 불가능하기 때문에 종단면적 분석이 함께 요구된다. 본 연구는 주어진 두 영상(전,후) 간의 병변의 특정 변화를 감지하는 새로운 인공지능 알고리즘을 제안한다. 본 알고리즘의 핵심 기법은 정합되지 않은 두 영상의 기하 상관관계도를 구하여 영상 간 변화 유무에 따른 기하 상관관계도 변화 패턴을 파악하고, 변화유무를 이진 분류하는 것이다. 현재까지 종단면적 분석을 위한 기계학습용 참조표준 데이터베이스가 공개된 것이 없기 때문에, 본 연구에서는 영상 판독문을 분석하여 병변의 변화기준을 확립하고, 질환의 종류, 경과시간, 변화 형태 등에 따른 데이터 분류 체계를 구축하여 순차적 흉부 X-선 영상에 대한 참조표준 데이터베이스를 자체적으로 확보하였다. 본 연구는 알고리즘 성능을 객관적으로 분석하기 위하여 수신자조작특성(ROC)의 하의 면적(AUC)을 산출하고, 기존 개발된 알고리즘 및 유사 연구와 정량적으로 비교하였다. 본 연구에서 제안하는 기하 상관관계도를 이용한 알고리즘이 AUC=0.89 (95% 신뢰구간, 0.86-0.92) 및 Youden's index에서의 민감도=0.83, 특이도=0.82으로 가장 뛰어난 성능을 보였다. 또한 주어진 두 영상에서 특정 병변의 변화에 따른 기하 상관관계도를 정성적으로 분석함으로써 실제로 해당 변화가 발생한 위치를 역추적 및 설명할 수 있는 가능성을 제시하였다.The diagnostic decision for chest X-ray image generally considers a probable change in a lesion, compared to the previous examination. We propose a novel algorithm to detect the change in longitudinal chest X-ray images. We extract feature maps from a pair of input images through two streams of convolutional neural networks. Next, we generate the geometric correlation map computing matching scores for every possible match of local descriptors in two feature maps. This correlation map is fed into a binary classifier to detect specific patterns of the map representing the change in the lesion. Since no public dataset offers proper information to train the proposed network, we also build our own dataset by analyzing reports in examinations at a tertiary hospital. Experimental results show our approach outperforms previous methods in quantitative comparison. We also provide various case examples visualizing the effect of the proposed geometric correlation map.1. 서론 7 1.1. 배경 7 1.2. 연구의 목적 9 2. 본론 12 2.1. 알고리즘 구조 12 2.2.1. 특징 추출 14 2.2.2. 기하 상관관계도 14 2.2.3. 이진 분류기 16 2.2. 참조표준 데이터베이스 17 3. 결과 및 분석 21 3.1. 판독문 가공 21 3.2. 알고리즘 성능 22 4. 고찰 28 4.1. 실험 결과 고찰 28 4.2. 알고리즘 고찰 28 5. 결론 30 참고 문헌 31 Abstract 33석

Quantitative Evaluation of Pulmonary Emphysema Using Magnetic Resonance Imaging and x-ray Computed Tomography

Chronic obstructive pulmonary disease (COPD) is a leading cause of morbidity and mortality affecting at least 600 million people worldwide. The most widely used clinical measurements of lung function such as spirometry and plethysmography are generally accepted for diagnosis and monitoring of the disease. However, these tests provide only global measures of lung function and they are insensitive to early disease changes. Imaging tools that are currently available have the potential to provide regional information about lung structure and function but at present are mainly used for qualitative assessment of disease and disease progression. In this thesis, we focused on the application of quantitative measurements of lung structure derived from 1H magnetic resonance imaging (MRI) and high resolution computed tomography (CT) in subjects diagnosed with COPD by a physician. Our results showed that significant and moderately strong relationship exists between 1H signal intensity (SI) and 3He apparent diffusion coefficient (ADC), as well as between 1H SI and CT measurements of emphysema. This suggests that these imaging methods may be quantifying the same tissue changes in COPD, and that pulmonary 1H SI may be used effectively to monitor emphysema as a complement to CT and noble gas MRI. Additionally, our results showed that objective multi-threshold analysis of CT images for emphysema scoring that takes into account the frequency distribution of each Hounsfield unit (HU) threshold was effective in correctly classifying the patient into COPD and healthy subgroups. Finally, we found a significant correlation between whole lung average subjective and objective emphysema scores with high inter-observer agreement. It is concluded that 1H MRI and high resolution CT can be used to quantitatively evaluate lung tissue alterations in COPD subjects

BiomedJourney: Counterfactual Biomedical Image Generation by Instruction-Learning from Multimodal Patient Journeys

Rapid progress has been made in instruction-learning for image editing with natural-language instruction, as exemplified by InstructPix2Pix. In biomedicine, such methods can be applied to counterfactual image generation, which helps differentiate causal structure from spurious correlation and facilitate robust image interpretation for disease progression modeling. However, generic image-editing models are ill-suited for the biomedical domain, and counterfactual biomedical image generation is largely underexplored. In this paper, we present BiomedJourney, a novel method for counterfactual biomedical image generation by instruction-learning from multimodal patient journeys. Given a patient with two biomedical images taken at different time points, we use GPT-4 to process the corresponding imaging reports and generate a natural language description of disease progression. The resulting triples (prior image, progression description, new image) are then used to train a latent diffusion model for counterfactual biomedical image generation. Given the relative scarcity of image time series data, we introduce a two-stage curriculum that first pretrains the denoising network using the much more abundant single image-report pairs (with dummy prior image), and then continues training using the counterfactual triples. Experiments using the standard MIMIC-CXR dataset demonstrate the promise of our method. In a comprehensive battery of tests on counterfactual medical image generation, BiomedJourney substantially outperforms prior state-of-the-art methods in instruction image editing and medical image generation such as InstructPix2Pix and RoentGen. To facilitate future study in counterfactual medical generation, we plan to release our instruction-learning code and pretrained models.Comment: Project page & demo: https://aka.ms/biomedjourne

Pulmonary Image Segmentation and Registration Algorithms: Towards Regional Evaluation of Obstructive Lung Disease

Pulmonary imaging, including pulmonary magnetic resonance imaging (MRI) and computed tomography (CT), provides a way to sensitively and regionally measure spatially heterogeneous lung structural-functional abnormalities. These unique imaging biomarkers offer the potential for better understanding pulmonary disease mechanisms, monitoring disease progression and response to therapy, and developing novel treatments for improved patient care. To generate these regional lung structure-function measurements and enable broad clinical applications of quantitative pulmonary MRI and CT biomarkers, as a first step, accurate, reproducible and rapid lung segmentation and registration methods are required. In this regard, we first developed a 1H MRI lung segmentation algorithm that employs complementary hyperpolarized 3He MRI functional information for improved lung segmentation. The 1H-3He MRI joint segmentation algorithm was formulated as a coupled continuous min-cut model and solved through convex relaxation, for which a dual coupled continuous max-flow model was proposed and a max-flow-based efficient numerical solver was developed. Experimental results on a clinical dataset of 25 chronic obstructive pulmonary disease (COPD) patients ranging in disease severity demonstrated that the algorithm provided rapid lung segmentation with high accuracy, reproducibility and diminished user interaction. We then developed a general 1H MRI left-right lung segmentation approach by exploring the left-to-right lung volume proportion prior. The challenging volume proportion-constrained multi-region segmentation problem was approximated through convex relaxation and equivalently represented by a max-flow model with bounded flow conservation conditions. This gave rise to a multiplier-based high performance numerical implementation based on convex optimization theories. In 20 patients with mild- to-moderate and severe asthma, the approach demonstrated high agreement with manual segmentation, excellent reproducibility and computational efficiency. Finally, we developed a CT-3He MRI deformable registration approach that coupled the complementary CT-1H MRI registration. The joint registration problem was solved by exploring optical-flow techniques, primal-dual analyses and convex optimization theories. In a diverse group of patients with asthma and COPD, the registration approach demonstrated lower target registration error than single registration and provided fast regional lung structure-function measurements that were strongly correlated with a reference method. Collectively, these lung segmentation and registration algorithms demonstrated accuracy, reproducibility and workflow efficiency that all may be clinically-acceptable. All of this is consistent with the need for broad and large-scale clinical applications of pulmonary MRI and CT

Correlation of Acute Radiation Dermatitis to Tissue Oxygenation in Radiation Therapy treated Breast Cancer Subjects

Over 95% of radiation therapy (RT) treated breast cancer subjects undergo an adverse skin reaction known as radiation dermatitis (RD). Assessment of severity or grading of RD is clinically visual and hence subjective. Our objective is to determine sub-clinical tissue oxygenation (StO2) changes in response to RT treatment in breast cancer subjects using near-infrared spectroscopic imaging and correlate these changes to RD grading. A WIRB approved 6-8 week longitudinal pilot study was carried out on 10 RT-treated subjects at Miami Cancer Institute. Significant changes (p \u3c 0.05) in StO2 of irradiated and contralateral chest wall and axilla regions with weeks of treatment were observed. The overall drop in StO2 was higher in irradiated regions compared to its contralateral region. This drop was negatively correlated to RD scaling. Pre-RT assessment of StO2 also related to severity in RD. The long-term goal is physiological based prediction of RD severity via tissue oxygenation measurements

A Survey of the Impact of Self-Supervised Pretraining for Diagnostic Tasks with Radiological Images

Self-supervised pretraining has been observed to be effective at improving feature representations for transfer learning, leveraging large amounts of unlabelled data. This review summarizes recent research into its usage in X-ray, computed tomography, magnetic resonance, and ultrasound imaging, concentrating on studies that compare self-supervised pretraining to fully supervised learning for diagnostic tasks such as classification and segmentation. The most pertinent finding is that self-supervised pretraining generally improves downstream task performance compared to full supervision, most prominently when unlabelled examples greatly outnumber labelled examples. Based on the aggregate evidence, recommendations are provided for practitioners considering using self-supervised learning. Motivated by limitations identified in current research, directions and practices for future study are suggested, such as integrating clinical knowledge with theoretically justified self-supervised learning methods, evaluating on public datasets, growing the modest body of evidence for ultrasound, and characterizing the impact of self-supervised pretraining on generalization.Comment: 32 pages, 6 figures, a literature survey submitted to BMC Medical Imagin