Automated classification of coronary plaque calcification in OCT pullbacks with 3D deep neural networks

Significance: Detection and characterization of coronary atherosclerotic plaques often need reviews of a large number of optical coherence tomography (OCT) imaging slices to make a clinical decision. However, it is a challenge to manually review all the slices and consider the interrelationship between adjacent slices. Approach: Inspired by the recent success of deep convolutional network on the classification of medical images, we proposed a ResNet-3D network for classification of coronary plaque calcification in OCT pullbacks. The ResNet-3D network was initialized with a trained ResNet-50 network and a three-dimensional convolution filter filled with zeros padding and non-zeros padding with a convolutional filter. To retrain ResNet-50, we used a dataset of ∼4860 OCT images, derived by 18 entire pullbacks from different patients. In addition, we investigated a two-phase training method to address the data imbalance. For an improved performance, we evaluated different input sizes for the ResNet-3D network, such as 3, 5, and 7 OCT slices. Furthermore, we integrated all ResNet-3D results by majority voting. Results: A comparative analysis proved the effectiveness of the proposed ResNet-3D networks against ResNet-2D network in the OCT dataset. The classification performance (F1-scores  =  94  %   for non-zeros padding and F1-score  =  96  %   for zeros padding) demonstrated the potential of convolutional neural networks (CNNs) in classifying plaque calcification. Conclusions: This work may provide a foundation for further work in extending the CNN to voxel segmentation, which may lead to a supportive diagnostic tool for assessment of coronary plaque vulnerability

Recent Trends in Artificial Intelligence-Assisted Coronary Atherosclerotic Plaque Characterization

Coronary artery disease is a major cause of morbidity and mortality worldwide. Its underlying histopathology is the atherosclerotic plaque, which comprises lipid, fibrous and—when chronic—calcium components. Intravascular ultrasound (IVUS) and intravascular optical coherence tomography (IVOCT) performed during invasive coronary angiography are reference standards for characterizing the atherosclerotic plaque. Fine image spatial resolution attainable with contemporary coronary computed tomographic angiography (CCTA) has enabled noninvasive plaque assessment, including identifying features associated with vulnerable plaques known to presage acute coronary events. Manual interpretation of IVUS, IVOCT and CCTA images demands scarce physician expertise and high time cost. This has motivated recent research into and development of artificial intelligence (AI)-assisted methods for image processing, feature extraction, plaque identification and characterization. We performed parallel searches of the medical and technical literature from 1995 to 2021 focusing respectively on human plaque characterization using various imaging modalities and the use of AI-assisted computer aided diagnosis (CAD) to detect and classify atherosclerotic plaques, including their composition and the presence of high-risk features denoting vulnerable plaques. A total of 122 publications were selected for evaluation and the analysis was summarized in terms of data sources, methods—machine versus deep learning—and performance metrics. Trends in AI-assisted plaque characterization are detailed and prospective research challenges discussed. Future directions for the development of accurate and efficient CAD systems to characterize plaque noninvasively using CCTA are proposed.</jats:p

Atherosclerotic Plaque Tissue Characterization: An OCT-Based Machine Learning Algorithm With ex vivo Validation

There is a need to develop a validated algorithm for plaque characterization which can help to facilitate the standardization of optical coherence tomography (OCT) image interpretation of plaque morphology, and improve the efficiency and accuracy in the application of OCT imaging for the quantitative assessment of plaque vulnerability. In this study, a machine learning algorithm was implemented for characterization of atherosclerotic plaque components by intravascular OCT using ex vivo carotid plaque tissue samples. A total of 31 patients underwent carotid endarterectomy and the ex vivo carotid plaques were imaged with OCT. Optical parameter, texture features and relative position of pixels were extracted within the region of interest and then used to quantify the tissue characterization of plaque components. The potential of individual and combined feature set to discriminate tissue components was quantified using sensitivity, specificity, accuracy. The results show there was a lower classification accuracy in the calcified tissue than the fibrous tissue and lipid tissue. The pixel-wise classification accuracy obtained by the developed method, to characterize the fibrous, calcified and lipid tissue by comparing with histology, were 80.0, 62.0, and 83.1, respectively. The developed algorithm was capable of characterizing plaque components with an excellent accuracy using the combined feature set

Deep learning segmentation of fibrous cap in intravascular optical coherence tomography images

Thin-cap fibroatheroma (TCFA) is a prominent risk factor for plaque rupture. Intravascular optical coherence tomography (IVOCT) enables identification of fibrous cap (FC), measurement of FC thicknesses, and assessment of plaque vulnerability. We developed a fully-automated deep learning method for FC segmentation. This study included 32,531 images across 227 pullbacks from two registries. Images were semi-automatically labeled using our OCTOPUS with expert editing using established guidelines. We employed preprocessing including guidewire shadow detection, lumen segmentation, pixel-shifting, and Gaussian filtering on raw IVOCT (r,theta) images. Data were augmented in a natural way by changing theta in spiral acquisitions and by changing intensity and noise values. We used a modified SegResNet and comparison networks to segment FCs. We employed transfer learning from our existing much larger, fully-labeled calcification IVOCT dataset to reduce deep-learning training. Overall, our method consistently delivered better FC segmentation results (Dice: 0.837+/-0.012) than other deep-learning methods. Transfer learning reduced training time by 84% and reduced the need for more training samples. Our method showed a high level of generalizability, evidenced by highly-consistent segmentations across five-fold cross-validation (sensitivity: 85.0+/-0.3%, Dice: 0.846+/-0.011) and the held-out test (sensitivity: 84.9%, Dice: 0.816) sets. In addition, we found excellent agreement of FC thickness with ground truth (2.95+/-20.73 um), giving clinically insignificant bias. There was excellent reproducibility in pre- and post-stenting pullbacks (average FC angle: 200.9+/-128.0 deg / 202.0+/-121.1 deg). Our method will be useful for multiple research purposes and potentially for planning stent deployments that avoid placing a stent edge over an FC.Comment: 24 pages, 9 figures, 2 tables, 2 supplementary figures, 3 supplementary table

Deep Learning in Cardiology

The medical field is creating large amount of data that physicians are unable to decipher and use efficiently. Moreover, rule-based expert systems are inefficient in solving complicated medical tasks or for creating insights using big data. Deep learning has emerged as a more accurate and effective technology in a wide range of medical problems such as diagnosis, prediction and intervention. Deep learning is a representation learning method that consists of layers that transform the data non-linearly, thus, revealing hierarchical relationships and structures. In this review we survey deep learning application papers that use structured data, signal and imaging modalities from cardiology. We discuss the advantages and limitations of applying deep learning in cardiology that also apply in medicine in general, while proposing certain directions as the most viable for clinical use.Comment: 27 pages, 2 figures, 10 table