Deep Learning for Musculoskeletal Image Analysis

The diagnosis, prognosis, and treatment of patients with musculoskeletal (MSK) disorders require radiology imaging (using computed tomography, magnetic resonance imaging(MRI), and ultrasound) and their precise analysis by expert radiologists. Radiology scans can also help assessment of metabolic health, aging, and diabetes. This study presents how machinelearning, specifically deep learning methods, can be used for rapidand accurate image analysis of MRI scans, an unmet clinicalneed in MSK radiology. As a challenging example, we focus on automatic analysis of knee images from MRI scans and study machine learning classification of various abnormalities including meniscus and anterior cruciate ligament tears. Using widely used convolutional neural network (CNN) based architectures, we comparatively evaluated the knee abnormality classification performances of different neural network architectures under limited imaging data regime and compared single and multi-view imaging when classifying the abnormalities. Promising results indicated the potential use of multi-view deep learning based classification of MSK abnormalities in routine clinical assessment.Comment: Invited Paper, ASILOMAR 2019, TP4b: Machine Learning Advances in Computational Imagin

Semi-Supervised Deep Learning for Multi-Tissue Segmentation from Multi-Contrast MRI

Segmentation of thigh tissues (muscle, fat, inter-muscular adipose tissue (IMAT), bone, and bone marrow) from magnetic resonance imaging (MRI) scans is useful for clinical and research investigations in various conditions such as aging, diabetes mellitus, obesity, metabolic syndrome, and their associated comorbidities. Towards a fully automated, robust, and precise quantification of thigh tissues, herein we designed a novel semi-supervised segmentation algorithm based on deep network architectures. Built upon Tiramisu segmentation engine, our proposed deep networks use variational and specially designed targeted dropouts for faster and robust convergence, and utilize multi-contrast MRI scans as input data. In our experiments, we have used 150 scans from 50 distinct subjects from the Baltimore Longitudinal Study of Aging (BLSA). The proposed system made use of both labeled and unlabeled data with high efficacy for training, and outperformed the current state-of-the-art methods with dice scores of 97.52%, 94.61%, 80.14%, 95.93%, and 96.83% for muscle, fat, IMAT, bone, and bone marrow tissues, respectively. Our results indicate that the proposed system can be useful for clinical research studies where volumetric and distributional tissue quantification is pivotal and labeling is a significant issue. To the best of our knowledge, the proposed system is the first attempt at multi-tissue segmentation using a single end-to-end semi-supervised deep learning framework for multi-contrast thigh MRI scans.Comment: 20 pages, 9 figures, Journal of Signal Processing System

Neural Transformers for Intraductal Papillary Mucosal Neoplasms (IPMN) Classification in MRI images

Early detection of precancerous cysts or neoplasms, i.e., Intraductal Papillary Mucosal Neoplasms (IPMN), in pancreas is a challenging and complex task, and it may lead to a more favourable outcome. Once detected, grading IPMNs accurately is also necessary, since low-risk IPMNs can be under surveillance program, while high-risk IPMNs have to be surgically resected before they turn into cancer. Current standards (Fukuoka and others) for IPMN classification show significant intra- and inter-operator variability, beside being error-prone, making a proper diagnosis unreliable. The established progress in artificial intelligence, through the deep learning paradigm, may provide a key tool for an effective support to medical decision for pancreatic cancer. In this work, we follow this trend, by proposing a novel AI-based IPMN classifier that leverages the recent success of transformer networks in generalizing across a wide variety of tasks, including vision ones. We specifically show that our transformer-based model exploits pre-training better than standard convolutional neural networks, thus supporting the sought architectural universalism of transformers in vision, including the medical image domain and it allows for a better interpretation of the obtained results

Towards Automatic Cartilage Quantification in Clinical Trials - Continuing from the 2019 IWOAI Knee Segmentation Challenge.

OBJECTIVE: To evaluate whether the deep learning (DL) segmentation methods from the six teams that participated in the IWOAI 2019 Knee Cartilage Segmentation Challenge are appropriate for quantifying cartilage loss in longitudinal clinical trials. DESIGN: We included 556 subjects from the Osteoarthritis Initiative study with manually read cartilage volume scores for the baseline and 1-year visits. The teams used their methods originally trained for the IWOAI 2019 challenge to segment the 1130 knee MRIs. These scans were anonymized and the teams were blinded to any subject or visit identifiers. Two teams also submitted updated methods. The resulting 9,040 segmentations are available online.The segmentations included tibial, femoral, and patellar compartments. In post-processing, we extracted medial and lateral tibial compartments and geometrically defined central medial and lateral femoral sub-compartments. The primary study outcome was the sensitivity to measure cartilage loss as defined by the standardized response mean (SRM). RESULTS: For the tibial compartments, several of the DL segmentation methods had SRMs similar to the gold standard manual method. The highest DL SRM was for the lateral tibial compartment at 0.38 (the gold standard had 0.34). For the femoral compartments, the gold standard had higher SRMs than the automatic methods at 0.31/0.30 for medial/lateral compartments. CONCLUSION: The lower SRMs for the DL methods in the femoral compartments at 0.2 were possibly due to the simple sub-compartment extraction done during post-processing. The study demonstrated that state-of-the-art DL segmentation methods may be used in standardized longitudinal single-scanner clinical trials for well-defined cartilage compartments

A Novel Extension to Fuzzy Connectivity for Body Composition Analysis: Applications in Thigh, Brain, and Whole Body Tissue Segmentation

###EgeUn###Magnetic resonance imaging (MRI) is the non-invasive modality of choice for body tissue composition analysis due to its excellent soft-tissue contrast and lack of ionizing radiation. However, quantification of body composition requires an accurate segmentation of fat, muscle, and other tissues from MR images, which remains a challenging goal due to the intensity overlap between them. In this study, we propose a fully automated, data-driven image segmentation platform that addresses multiple difficulties in segmenting MR images such as varying inhomogeneity, non-standardness, and noise, while producing a high-quality definition of different tissues. In contrast to most approaches in the literature, we perform segmentation operation by combining three different MRI contrasts and a novel segmentation tool, which takes into account variability in the data. The proposed system, based on a novel affinity definition within the fuzzy connectivity image segmentation family, prevents the need for user intervention and reparametrization of the segmentation algorithms. In order to make the whole system fully automated, we adapt an affinity propagation clustering algorithm to roughly identify tissue regions and image background. We perform a thorough evaluation of the proposed algorithm's individual steps as well as comparison with several approaches from the literature for the main application of muscle/fat separation. Furthermore, whole-body tissue composition and brain tissue delineation were conducted to show the generalization ability of the proposed system. This new automated platform outperforms other state-of-the-art segmentation approaches both in accuracy and efficiency.Intramural Research Program of the National Institute on Aging of the National Institutes of HealthUnited States Department of Health & Human ServicesNational Institutes of Health (NIH) - USANIH National Institute on Aging (NIA); Scientific Council of Turkey [TUBITAK-BIDEB 2214/A]This work was supported by the Intramural Research Program of the National Institute on Aging of the National Institutes of Health. The work of I. Irmakci was supported by the Scientific Council of Turkey (TUBITAK-BIDEB 2214/A)