Computational Tools for Image Processing, Integration, and Visualization of Simultaneous OCT-FLIM Images of Tissue

Multimodal imaging systems have emerged as robust methods for the characterization of atherosclerotic plaques and early diagnosis of oral cancer. Multispectral wide-field Fluorescence Lifetime Imaging Microscopy (FLIM) has been shown to be a capable optical imaging modality for biomedical diagnosis oral cancer. A fiber-based endoscope combined with an intensified charge-coupled device (ICCD) allows to collect and split the fluorescence emission into multiple bands, from which the fluorescence lifetime decay in each spectral channel can be calculated separately. However, for accurate calculations, it is necessary to gather multiple gates increasing the imaging time. Since this time is critical for real-time in vivo applications. This study presents a novel approach to using Rapid Lifetime Determination (RLD) methods to considerably shorten this time period. Moreover, the use of a dual-modality system, incorporating Optical Coherence Tomography (OCT) and FLIM, which simultaneously characterizes 3-D tissue morphology and biochemical composition of tissue, leads to the development of robust computational tools for image processing, integration, and visualization of these imaging techniques. OCTFLIM systems provide 3D structural and 2D biochemical tissue information, which the software tools developed in this work properly integrate to assist the image processing, characterization, and visualization of OCT-FLIM images of atherosclerotic plaques. Additionally, plaque characterization is performed by visual assessment and requires a trained expert for interpretation of the large data sets. Here, we present two novel computational methods for automated intravascular (IV) OCT plaque characterization. The first method is based on the modeling of each A-line of an IV-OCT data set as a linear combination of a number of depth profiles. After estimating these depth profiles by means of an alternating least square optimization strategy, they are automatically classified to predefined tissue types based on their morphological characteristics. The second method is intended to automatically identify macrophage/foam cell clusters in atherosclerotic plaques. Vulnerable plaques are characterized by presenting a necrotic core below a thin fibrous cap, and extensive infiltration of macrophages/foam cells. Thus, the degree of macrophage accumulation is an indicator in determining plaque progression and probability of rupture. In this work, two texture features are introduced, the normalized standard deviation ratio (NSDRatio) and the entropy ratio (ENTRatio), to effectively classify areas in the plaque with macrophage/foam cell infiltration. Since this methodology has low complexity and computational cost, it could be implemented for in vivo real time identification of macrophage/foam cell presence

Computational Tools for Image Processing, Integration, and Visualization of Simultaneous OCT-FLIM Images of Tissue

Multimodal imaging systems have emerged as robust methods for the characterization of atherosclerotic plaques and early diagnosis of oral cancer. Multispectral wide-field Fluorescence Lifetime Imaging Microscopy (FLIM) has been shown to be a capable optical imaging modality for biomedical diagnosis oral cancer. A fiber-based endoscope combined with an intensified charge-coupled device (ICCD) allows to collect and split the fluorescence emission into multiple bands, from which the fluorescence lifetime decay in each spectral channel can be calculated separately. However, for accurate calculations, it is necessary to gather multiple gates increasing the imaging time. Since this time is critical for real-time in vivo applications. This study presents a novel approach to using Rapid Lifetime Determination (RLD) methods to considerably shorten this time period. Moreover, the use of a dual-modality system, incorporating Optical Coherence Tomography (OCT) and FLIM, which simultaneously characterizes 3-D tissue morphology and biochemical composition of tissue, leads to the development of robust computational tools for image processing, integration, and visualization of these imaging techniques. OCTFLIM systems provide 3D structural and 2D biochemical tissue information, which the software tools developed in this work properly integrate to assist the image processing, characterization, and visualization of OCT-FLIM images of atherosclerotic plaques. Additionally, plaque characterization is performed by visual assessment and requires a trained expert for interpretation of the large data sets. Here, we present two novel computational methods for automated intravascular (IV) OCT plaque characterization. The first method is based on the modeling of each A-line of an IV-OCT data set as a linear combination of a number of depth profiles. After estimating these depth profiles by means of an alternating least square optimization strategy, they are automatically classified to predefined tissue types based on their morphological characteristics. The second method is intended to automatically identify macrophage/foam cell clusters in atherosclerotic plaques. Vulnerable plaques are characterized by presenting a necrotic core below a thin fibrous cap, and extensive infiltration of macrophages/foam cells. Thus, the degree of macrophage accumulation is an indicator in determining plaque progression and probability of rupture. In this work, two texture features are introduced, the normalized standard deviation ratio (NSDRatio) and the entropy ratio (ENTRatio), to effectively classify areas in the plaque with macrophage/foam cell infiltration. Since this methodology has low complexity and computational cost, it could be implemented for in vivo real time identification of macrophage/foam cell presence

Automated analysis of fibrous cap in intravascular optical coherence tomography images of coronary arteries

Thin-cap fibroatheroma (TCFA) and plaque rupture have been recognized as the most frequent risk factor for thrombosis and acute coronary syndrome. Intravascular optical coherence tomography (IVOCT) can identify TCFA and assess cap thickness, which provides an opportunity to assess plaque vulnerability. We developed an automated method that can detect lipidous plaque and assess fibrous cap thickness in IVOCT images. This study analyzed a total of 4,360 IVOCT image frames of 77 lesions among 41 patients. To improve segmentation performance, preprocessing included lumen segmentation, pixel-shifting, and noise filtering on the raw polar (r, theta) IVOCT images. We used the DeepLab-v3 plus deep learning model to classify lipidous plaque pixels. After lipid detection, we automatically detected the outer border of the fibrous cap using a special dynamic programming algorithm and assessed the cap thickness. Our method provided excellent discriminability of lipid plaque with a sensitivity of 85.8% and A-line Dice coefficient of 0.837. By comparing lipid angle measurements between two analysts following editing of our automated software, we found good agreement by Bland-Altman analysis (difference 6.7+/-17 degree; mean 196 degree). Our method accurately detected the fibrous cap from the detected lipid plaque. Automated analysis required a significant modification for only 5.5% frames. Furthermore, our method showed a good agreement of fibrous cap thickness between two analysts with Bland-Altman analysis (4.2+/-14.6 micron; mean 175 micron), indicating little bias between users and good reproducibility of the measurement. We developed a fully automated method for fibrous cap quantification in IVOCT images, resulting in good agreement with determinations by analysts. The method has great potential to enable highly automated, repeatable, and comprehensive evaluations of TCFAs.Comment: 18 pages, 9 figure

Optical Coherence Tomography and Fibrous Cap Characterization

The pathophysiology of acute coronary syndromes has long been associated with atherosclerotic plaque rupture. Inflammation, thinning, and disruption of the fibrous cap have been implicated with the final processes leading to plaque rupture, but confirmation of these mechanisms of coronary thrombosis in humans has been hampered by the lack of imaging methods with sufficient resolution to resolve fibrous cap characterization and thickness in vivo. Intravascular optical coherence tomography (OCT) provides images with micron-level axial and lateral resolution, enabling detailed visualization of micro-structural changes of the arterial wall. The present article provides an overview of the potential role of OCT in identifying and characterizing fibrous cap morphology, thickness, and inflammation in human coronary plaques

Segmentation of anatomical layers and imaging artifacts in intravascular polarization sensitive optical coherence tomography using attending physician and boundary cardinality losses

Intravascular ultrasound and optical coherence tomography are widely available for assessing coronary stenoses and provide critical information to optimize percutaneous coronary intervention. Intravascular polarization-sensitive optical coherence tomography (PS-OCT) measures the polarization state of the light scattered by the vessel wall in addition to conventional cross-sectional images of subsurface microstructure. This affords reconstruction of tissue polarization properties and reveals improved contrast between the layers of the vessel wall along with insight into collagen and smooth muscle content. Here, we propose a convolutional neural network model, optimized using two new loss terms (Boundary Cardinality and Attending Physician), that takes advantage of the additional polarization contrast and classifies the lumen, intima, and media layers in addition to guidewire and plaque shadows. Our model segments the media boundaries through fibrotic plaques and continues to estimate the outer media boundary behind shadows of lipid-rich plaques. We demonstrate that our multi-class classification model outperforms existing methods that exclusively use conventional OCT data, predominantly segment the lumen, and consider subsurface layers at most in regions of minimal disease. Segmentation of all anatomical layers throughout diseased vessels may facilitate stent sizing and will enable automated characterization of plaque polarization properties for investigation of the natural history and significance of coronary atheromas.</p

Segmentation of anatomical layers and imaging artifacts in intravascular polarization sensitive optical coherence tomography using attending physician and boundary cardinality losses

Intravascular ultrasound and optical coherence tomography are widely available for assessing coronary stenoses and provide critical information to optimize percutaneous coronary intervention. Intravascular polarization-sensitive optical coherence tomography (PS-OCT) measures the polarization state of the light scattered by the vessel wall in addition to conventional cross-sectional images of subsurface microstructure. This affords reconstruction of tissue polarization properties and reveals improved contrast between the layers of the vessel wall along with insight into collagen and smooth muscle content. Here, we propose a convolutional neural network model, optimized using two new loss terms (Boundary Cardinality and Attending Physician), that takes advantage of the additional polarization contrast and classifies the lumen, intima, and media layers in addition to guidewire and plaque shadows. Our model segments the media boundaries through fibrotic plaques and continues to estimate the outer media boundary behind shadows of lipid-rich plaques. We demonstrate that our multi-class classification model outperforms existing methods that exclusively use conventional OCT data, predominantly segment the lumen, and consider subsurface layers at most in regions of minimal disease. Segmentation of all anatomical layers throughout diseased vessels may facilitate stent sizing and will enable automated characterization of plaque polarization properties for investigation of the natural history and significance of coronary atheromas.</p

Deep learning segmentation of fibrous cap in intravascular optical coherence tomography images

Thin-cap fibroatheroma (TCFA) is a prominent risk factor for plaque rupture. Intravascular optical coherence tomography (IVOCT) enables identification of fibrous cap (FC), measurement of FC thicknesses, and assessment of plaque vulnerability. We developed a fully-automated deep learning method for FC segmentation. This study included 32,531 images across 227 pullbacks from two registries. Images were semi-automatically labeled using our OCTOPUS with expert editing using established guidelines. We employed preprocessing including guidewire shadow detection, lumen segmentation, pixel-shifting, and Gaussian filtering on raw IVOCT (r,theta) images. Data were augmented in a natural way by changing theta in spiral acquisitions and by changing intensity and noise values. We used a modified SegResNet and comparison networks to segment FCs. We employed transfer learning from our existing much larger, fully-labeled calcification IVOCT dataset to reduce deep-learning training. Overall, our method consistently delivered better FC segmentation results (Dice: 0.837+/-0.012) than other deep-learning methods. Transfer learning reduced training time by 84% and reduced the need for more training samples. Our method showed a high level of generalizability, evidenced by highly-consistent segmentations across five-fold cross-validation (sensitivity: 85.0+/-0.3%, Dice: 0.846+/-0.011) and the held-out test (sensitivity: 84.9%, Dice: 0.816) sets. In addition, we found excellent agreement of FC thickness with ground truth (2.95+/-20.73 um), giving clinically insignificant bias. There was excellent reproducibility in pre- and post-stenting pullbacks (average FC angle: 200.9+/-128.0 deg / 202.0+/-121.1 deg). Our method will be useful for multiple research purposes and potentially for planning stent deployments that avoid placing a stent edge over an FC.Comment: 24 pages, 9 figures, 2 tables, 2 supplementary figures, 3 supplementary table

Depth resolved detection of lipid using spectroscopic optical coherence tomography

Optical frequency domain imaging (OFDI) can identify key components related to plaque vulnerability but can suffer from artifacts that could prevent accurate identification of lipid rich regions. In this paper, we present a model of depth resolved spectral analysis of OFDI data for improved detection of lipid. A quadratic Discriminant analysis model was developed based on phantom compositions known chemical mixtures and applied to a tissue phantom of a lipid-rich plaque. We demonstrate that a combined spectral and attenuation model can be used to predict the presence of lipid in OFDI images

Automated classification of coronary plaque calcification in OCT pullbacks with 3D deep neural networks

Significance: Detection and characterization of coronary atherosclerotic plaques often need reviews of a large number of optical coherence tomography (OCT) imaging slices to make a clinical decision. However, it is a challenge to manually review all the slices and consider the interrelationship between adjacent slices. Approach: Inspired by the recent success of deep convolutional network on the classification of medical images, we proposed a ResNet-3D network for classification of coronary plaque calcification in OCT pullbacks. The ResNet-3D network was initialized with a trained ResNet-50 network and a three-dimensional convolution filter filled with zeros padding and non-zeros padding with a convolutional filter. To retrain ResNet-50, we used a dataset of ∼4860 OCT images, derived by 18 entire pullbacks from different patients. In addition, we investigated a two-phase training method to address the data imbalance. For an improved performance, we evaluated different input sizes for the ResNet-3D network, such as 3, 5, and 7 OCT slices. Furthermore, we integrated all ResNet-3D results by majority voting. Results: A comparative analysis proved the effectiveness of the proposed ResNet-3D networks against ResNet-2D network in the OCT dataset. The classification performance (F1-scores  =  94  %   for non-zeros padding and F1-score  =  96  %   for zeros padding) demonstrated the potential of convolutional neural networks (CNNs) in classifying plaque calcification. Conclusions: This work may provide a foundation for further work in extending the CNN to voxel segmentation, which may lead to a supportive diagnostic tool for assessment of coronary plaque vulnerability