Deep Learning for Musculoskeletal Image Analysis

The diagnosis, prognosis, and treatment of patients with musculoskeletal (MSK) disorders require radiology imaging (using computed tomography, magnetic resonance imaging(MRI), and ultrasound) and their precise analysis by expert radiologists. Radiology scans can also help assessment of metabolic health, aging, and diabetes. This study presents how machinelearning, specifically deep learning methods, can be used for rapidand accurate image analysis of MRI scans, an unmet clinicalneed in MSK radiology. As a challenging example, we focus on automatic analysis of knee images from MRI scans and study machine learning classification of various abnormalities including meniscus and anterior cruciate ligament tears. Using widely used convolutional neural network (CNN) based architectures, we comparatively evaluated the knee abnormality classification performances of different neural network architectures under limited imaging data regime and compared single and multi-view imaging when classifying the abnormalities. Promising results indicated the potential use of multi-view deep learning based classification of MSK abnormalities in routine clinical assessment.Comment: Invited Paper, ASILOMAR 2019, TP4b: Machine Learning Advances in Computational Imagin

GazeGNN: A Gaze-Guided Graph Neural Network for Disease Classification

The application of eye-tracking techniques in medical image analysis has become increasingly popular in recent years. It collects the visual search patterns of the domain experts, containing much important information about health and disease. Therefore, how to efficiently integrate radiologists' gaze patterns into the diagnostic analysis turns into a critical question. Existing works usually transform gaze information into visual attention maps (VAMs) to supervise the learning process. However, this time-consuming procedure makes it difficult to develop end-to-end algorithms. In this work, we propose a novel gaze-guided graph neural network (GNN), GazeGNN, to perform disease classification from medical scans. In GazeGNN, we create a unified representation graph that models both the image and gaze pattern information. Hence, the eye-gaze information is directly utilized without being converted into VAMs. With this benefit, we develop a real-time, real-world, end-to-end disease classification algorithm for the first time and avoid the noise and time consumption introduced during the VAM preparation. To our best knowledge, GazeGNN is the first work that adopts GNN to integrate image and eye-gaze data. Our experiments on the public chest X-ray dataset show that our proposed method exhibits the best classification performance compared to existing methods

Semi-Supervised Deep Learning for Multi-Tissue Segmentation from Multi-Contrast MRI

Segmentation of thigh tissues (muscle, fat, inter-muscular adipose tissue (IMAT), bone, and bone marrow) from magnetic resonance imaging (MRI) scans is useful for clinical and research investigations in various conditions such as aging, diabetes mellitus, obesity, metabolic syndrome, and their associated comorbidities. Towards a fully automated, robust, and precise quantification of thigh tissues, herein we designed a novel semi-supervised segmentation algorithm based on deep network architectures. Built upon Tiramisu segmentation engine, our proposed deep networks use variational and specially designed targeted dropouts for faster and robust convergence, and utilize multi-contrast MRI scans as input data. In our experiments, we have used 150 scans from 50 distinct subjects from the Baltimore Longitudinal Study of Aging (BLSA). The proposed system made use of both labeled and unlabeled data with high efficacy for training, and outperformed the current state-of-the-art methods with dice scores of 97.52%, 94.61%, 80.14%, 95.93%, and 96.83% for muscle, fat, IMAT, bone, and bone marrow tissues, respectively. Our results indicate that the proposed system can be useful for clinical research studies where volumetric and distributional tissue quantification is pivotal and labeling is a significant issue. To the best of our knowledge, the proposed system is the first attempt at multi-tissue segmentation using a single end-to-end semi-supervised deep learning framework for multi-contrast thigh MRI scans.Comment: 20 pages, 9 figures, Journal of Signal Processing System

The International Workshop on Osteoarthritis Imaging Knee MRI Segmentation Challenge: A Multi-Institute Evaluation and Analysis Framework on a Standardized Dataset

Purpose: To organize a knee MRI segmentation challenge for characterizing the semantic and clinical efficacy of automatic segmentation methods relevant for monitoring osteoarthritis progression. Methods: A dataset partition consisting of 3D knee MRI from 88 subjects at two timepoints with ground-truth articular (femoral, tibial, patellar) cartilage and meniscus segmentations was standardized. Challenge submissions and a majority-vote ensemble were evaluated using Dice score, average symmetric surface distance, volumetric overlap error, and coefficient of variation on a hold-out test set. Similarities in network segmentations were evaluated using pairwise Dice correlations. Articular cartilage thickness was computed per-scan and longitudinally. Correlation between thickness error and segmentation metrics was measured using Pearson's coefficient. Two empirical upper bounds for ensemble performance were computed using combinations of model outputs that consolidated true positives and true negatives. Results: Six teams (T1-T6) submitted entries for the challenge. No significant differences were observed across all segmentation metrics for all tissues (p=1.0) among the four top-performing networks (T2, T3, T4, T6). Dice correlations between network pairs were high (>0.85). Per-scan thickness errors were negligible among T1-T4 (p=0.99) and longitudinal changes showed minimal bias (<0.03mm). Low correlations (<0.41) were observed between segmentation metrics and thickness error. The majority-vote ensemble was comparable to top performing networks (p=1.0). Empirical upper bound performances were similar for both combinations (p=1.0). Conclusion: Diverse networks learned to segment the knee similarly where high segmentation accuracy did not correlate to cartilage thickness accuracy. Voting ensembles did not outperform individual networks but may help regularize individual models.Comment: Submitted to Radiology: Artificial Intelligence; Fixed typo

Chest Fat Quantification via CT Based on Standardized Anatomy Space in Adult Lung Transplant Candidates

Purpose Overweight and underweight conditions are considered relative contraindications to lung transplantation due to their association with excess mortality. Yet, recent work suggests that body mass index (BMI) does not accurately reflect adipose tissue mass in adults with advanced lung diseases. Alternative and more accurate measures of adiposity are needed. Chest fat estimation by routine computed tomography (CT) imaging may therefore be important for identifying high-risk lung transplant candidates. In this paper, an approach to chest fat quantification and quality assessment based on a recently formulated concept of standardized anatomic space (SAS) is presented. The goal of the paper is to seek answers to several key questions related to chest fat quantity and quality assessment based on a single slice CT (whether in the chest, abdomen, or thigh) versus a volumetric CT, which have not been addressed in the literature. Methods Unenhanced chest CT image data sets from 40 adult lung transplant candidates (age 58 ± 12 yrs and BMI 26.4 ± 4.3 kg/m2), 16 with chronic obstructive pulmonary disease (COPD), 16 with idiopathic pulmonary fibrosis (IPF), and the remainder with other conditions were analyzed together with a single slice acquired for each patient at the L5 vertebral level and mid-thigh level. The thoracic body region and the interface between subcutaneous adipose tissue (SAT) and visceral adipose tissue (VAT) in the chest were consistently defined in all patients and delineated using Live Wire tools. The SAT and VAT components of chest were then segmented guided by this interface. The SAS approach was used to identify the corresponding anatomic slices in each chest CT study, and SAT and VAT areas in each slice as well as their whole volumes were quantified. Similarly, the SAT and VAT components were segmented in the abdomen and thigh slices. Key parameters of the attenuation (Hounsfield unit (HU) distributions) were determined from each chest slice and from the whole chest volume separately for SAT and VAT components. The same parameters were also computed from the single abdominal and thigh slices. The ability of the slice at each anatomic location in the chest (and abdomen and thigh) to act as a marker of the measures derived from the whole chest volume was assessed via Pearson correlation coefficient (PCC) analysis. Results The SAS approach correctly identified slice locations in different subjects in terms of vertebral levels. PCC between chest fat volume and chest slice fat area was maximal at the T8 level for SAT (0.97) and at the T7 level for VAT (0.86), and was modest between chest fat volume and abdominal slice fat area for SAT and VAT (0.73 and 0.75, respectively). However, correlation was weak for chest fat volume and thigh slice fat area for SAT and VAT (0.52 and 0.37, respectively), and for chest fat volume for SAT and VAT and BMI (0.65 and 0.28, respectively). These same single slice locations with maximal PCC were found for SAT and VAT within both COPD and IPF groups. Most of the attenuation properties derived from the whole chest volume and single best chest slice for VAT (but not for SAT) were significantly different between COPD and IPF groups. Conclusions This study demonstrates a new way of optimally selecting slices whose measurements may be used as markers of similar measurements made on the whole chest volume. The results suggest that one or two slices imaged at T7 and T8 vertebral levels may be enough to estimate reliably the total SAT and VAT components of chest fat and the quality of chest fat as determined by attenuation distributions in the entire chest volume