Automating Carotid Intima-Media Thickness Video Interpretation with Convolutional Neural Networks

Cardiovascular disease (CVD) is the leading cause of mortality yet largely preventable, but the key to prevention is to identify at-risk individuals before adverse events. For predicting individual CVD risk, carotid intima-media thickness (CIMT), a noninvasive ultrasound method, has proven to be valuable, offering several advantages over CT coronary artery calcium score. However, each CIMT examination includes several ultrasound videos, and interpreting each of these CIMT videos involves three operations: (1) select three end-diastolic ultrasound frames (EUF) in the video, (2) localize a region of interest (ROI) in each selected frame, and (3) trace the lumen-intima interface and the media-adventitia interface in each ROI to measure CIMT. These operations are tedious, laborious, and time consuming, a serious limitation that hinders the widespread utilization of CIMT in clinical practice. To overcome this limitation, this paper presents a new system to automate CIMT video interpretation. Our extensive experiments demonstrate that the suggested system significantly outperforms the state-of-the-art methods. The superior performance is attributable to our unified framework based on convolutional neural networks (CNNs) coupled with our informative image representation and effective post-processing of the CNN outputs, which are uniquely designed for each of the above three operations.Comment: J. Y. Shin, N. Tajbakhsh, R. T. Hurst, C. B. Kendall, and J. Liang. Automating carotid intima-media thickness video interpretation with convolutional neural networks. CVPR 2016, pp 2526-2535; N. Tajbakhsh, J. Y. Shin, R. T. Hurst, C. B. Kendall, and J. Liang. Automatic interpretation of CIMT videos using convolutional neural networks. Deep Learning for Medical Image Analysis, Academic Press, 201

An end-to-end framework for intima media measurement and atherosclerotic plaque detection in the carotid artery

Background and objectives: The detection and delineation of atherosclerotic plaque are usually manually performed by medical experts on the carotid artery. Evidence suggests that this manual process is subject to errors and has a large variability between experts, equipment, and datasets. This paper proposes a robust end-to-end framework for automatic atherosclerotic plaque detection. Methods: The proposed framework is composed of: (1) a semantic segmentation model based on U-Net, with EfficientNet as the backbone, that obtains a segmentation mask with the carotid intima-media region; and (2) a convolutional neural network designed using Bayesian optimization that simultaneously performs a regression to get the average and maximum carotid intima media thickness, and a classification to determine the presence of plaque. Results: Our approach improves the state-of-the-art in both co and bulb territories in the REGICOR database, with more than 8000 images, while providing predictions in real-time. The correlation coefficient was 0.89 in the common carotid artery and 0.74 for bulb region, and the F1 score for atherosclerotic plaque detecting was 0.60 and 0.59, respectively. The experimentation carried out includes a comparison with other fully automatic methods for carotid intima media thickness estimation found in the literature. Additionally, we present an extensive experimental study to evaluate the robustness of our proposal, as well as its suitability and efficiency compared to different versions of the framework. Conclusions: The proposed end-to-end framework significantly improves the automatic characterization of atherosclerotic plaque. The generation of the segmented mask can be helpful for practitioners since it allows them to evaluate and interpret the model's results by visual inspection. Furthermore, the proposed framework overcomes the limitations of previous research based on ad-hoc post-processing, which could lead to overestimations in the case of oblique forms of the carotid artery

Multimodality carotid plaque tissue characterization and classification in the artificial intelligence paradigm: a narrative review for stroke application

Cardiovascular disease (CVD) is one of the leading causes of morbidity and mortality in the United States of America and globally. Carotid arterial plaque, a cause and also a marker of such CVD, can be detected by various non-invasive imaging modalities such as magnetic resonance imaging (MRI), computer tomography (CT), and ultrasound (US). Characterization and classification of carotid plaque-type in these imaging modalities, especially into symptomatic and asymptomatic plaque, helps in the planning of carotid endarterectomy or stenting. It can be challenging to characterize plaque components due to (I) partial volume effect in magnetic resonance imaging (MRI) or (II) varying Hausdorff values in plaque regions in CT, and (III) attenuation of echoes reflected by the plaque during US causing acoustic shadowing. Artificial intelligence (AI) methods have become an indispensable part of healthcare and their applications to the non-invasive imaging technologies such as MRI, CT, and the US. In this narrative review, three main types of AI models (machine learning, deep learning, and transfer learning) are analyzed when applied to MRI, CT, and the US. A link between carotid plaque characteristics and the risk of coronary artery disease is presented. With regard to characterization, we review tools and techniques that use AI models to distinguish carotid plaque types based on signal processing and feature strengths. We conclude that AI-based solutions offer an accurate and robust path for tissue characterization and classification for carotid artery plaque imaging in all three imaging modalities. Due to cost, user-friendliness, and clinical effectiveness, AI in the US has dominated the most

Attention-Based UNet Deep Learning Model for Plaque Segmentation in Carotid Ultrasound for Stroke Risk Stratification: An Artificial Intelligence Paradigm

Stroke and cardiovascular diseases (CVD) significantly affect the world population. The early detection of such events may prevent the burden of death and costly surgery. Conventional methods are neither automated nor clinically accurate. Artificial Intelligence-based methods of automatically detecting and predicting the severity of CVD and stroke in their early stages are of prime importance. This study proposes an attention-channel-based UNet deep learning (DL) model that identifies the carotid plaques in the internal carotid artery (ICA) and common carotid artery (CCA) images. Our experiments consist of 970 ICA images from the UK, 379 CCA images from diabetic Japanese patients, and 300 CCA images from post-menopausal women from Hong Kong. We combined both CCA images to form an integrated database of 679 images. A rotation transformation technique was applied to 679 CCA images, doubling the database for the experiments. The cross-validation K5 (80% training: 20% testing) protocol was applied for accuracy determination. The results of the Attention-UNet model are benchmarked against UNet, UNet++, and UNet3P models. Visual plaque segmentation showed improvement in the Attention-UNet results compared to the other three models. The correlation coefficient (CC) value for Attention-UNet is 0.96, compared to 0.93, 0.96, and 0.92 for UNet, UNet++, and UNet3P models. Similarly, the AUC value for Attention-UNet is 0.97, compared to 0.964, 0.966, and 0.965 for other models. Conclusively, the Attention-UNet model is beneficial in segmenting very bright and fuzzy plaque images that are hard to diagnose using other methods. Further, we present a multi-ethnic, multi-center, racial bias-free study of stroke risk assessment

A Tool for Telediagnosis of Cardiovascular Diseases in a Collaborative and Adaptive Approach

International audienceIn this paper, we present a new telediagnosis environment for the detection of cardiovascular problems. This tool, called VACODIS (VAscular COllaborative teleDIagnosiS), allows practitioners to semi-automatically identify and quantify a patient's potential cardiovascular complications. The system generates first-time automatic detection of cardiovascular abnormalities using Doppler ultrasound images. The system then provides remote collaborative sharing of this information among different doctors to allow distance telediagnostics. With this new system, different actors in the field of medicine (nurses, practitioners, etc.) will be able to contribute to a more reliable diagnosis in the cardiovascular domain

Deep learning-based carotid media-adventitia and lumen-intima boundary segmentation from three-dimensional ultrasound images

Purpose: Quantification of carotid plaques has been shown to be important for assessing as well as monitoring the progression and regression of carotid atherosclerosis. Various metrics have been proposed and methods of measurements ranging from manual tracing to automated segmentations have also been investigated. Of those metrics, quantification of carotid plaques by measuring vessel-wall-volume (VWV) using the segmented media-adventitia (MAB) and lumen-intima (LIB) boundaries has been shown to be sensitive to temporal changes in carotid plaque burden. Thus, semi-automatic MAB and LIB segmentation methods are required to help generate VWV measurements with high accuracy and less user interaction. Methods: In this paper, we propose a semiautomatic segmentation method based on deep learning to segment the MAB and LIB from carotid three-dimensional ultrasound (3DUS) images. For the MAB segmentation, we convert the segmentation problem to a pixel-by-pixel classification problem. A dynamic convolutional neural network (Dynamic CNN) is proposed to classify the patches generated by sliding a window along the norm line of the initial contour where the CNN model is fine-tuned dynamically in each test task. The LIB is segmented by applying a region-of-interest of carotid images to a U-Net model, which allows the network to be trained end-to-end for pixel-wise classification. Results: A total of 144 3DUS images were used in this development, and a threefold cross-validation technique was used for evaluation of the proposed algorithm. The proposed algorithm-generated accuracy was significantly higher than the previous methods but with less user interactions. Comparing the algorithm segmentation results with manual segmentations by an expert showed that the average Dice similarity coefficients (DSC) were 96.46 ± 2.22% and 92.84 ± 4.46% for the MAB and LIB, respectively, while only an average of 34 s (vs 1.13, 2.8 and 4.4 min in previous methods) was required to segment a 3DUS image. The interobserver experiment indicated that the DSC was 96.14 ± 1.87% between algorithm-generated MAB contours of two observers\u27 initialization. Conclusions: Our results showed that the proposed carotid plaque segmentation method obtains high accuracy and repeatability with less user interactions, suggesting that the method could be used in clinical practice to measure VWV and monitor the progression and regression of carotid plaques

Bimodal automated carotid ultrasound segmentation using geometrically constrained deep neural networks

For asymptomatic patients suffering from carotid stenosis, the assessment of plaque morphology is an important clinical task which allows monitoring of the risk of plaque rupture and future incidents of stroke. Ultrasound Imaging provides a safe and non-invasive modality for this, and the segmentation of media-adventitia boundaries and lumen-intima boundaries of the Carotid artery form an essential part in this monitoring process. In this paper, we propose a novel Deep Neural Network as a fully automated segmentation tool, and its application in delineating both the media-adventitia boundary and the lumen-intima boundary. We develop a new geometrically constrained objective function as part of the Network's Stochastic Gradient Descent optimisation, thus tuning it to the problem at hand. Furthermore, we also apply a bimodal fusion of amplitude and phase congruency data proposed by us in previous work, as an input to the network, as the latter provides an intensity-invariant data source to the network. We finally report the segmentation performance of the network on transverse sections of the carotid. Tests are carried out on an augmented dataset of 81,000 images, and the results are compared to other studies by reporting the DICE coefficient of similarity, modified Hausdorff Distance, sensitivity and specificity. Our proposed modification is shown to yield improved results on the standard network over this larger dataset, with the advantage of it being fully automated. We conclude that Deep Neural Networks provide a reliable trained manner in which carotid ultrasound images may be automatically segmented, using amplitude data and intensity invariant phase congruency maps as a data source

The Ultrasound Window Into Vascular Ageing: A Technology Review by the VascAgeNet COST Action

Arteriosclerosis; Ultrasound; Vascular ageingArteriosclerosi; Ecografia; Envelliment vascularArteriosclerosis; Ecografía; Envejecimiento vascularNon-invasive ultrasound (US) imaging enables the assessment of the properties of superficial blood vessels. Various modes can be used for vascular characteristics analysis, ranging from radiofrequency (RF) data, Doppler- and standard B/M-mode imaging, to more recent ultra-high frequency and ultrafast techniques. The aim of the present work was to provide an overview of the current state-of-the-art non-invasive US technologies and corresponding vascular ageing characteristics from a technological perspective. Following an introduction about the basic concepts of the US technique, the characteristics considered in this review are clustered into: 1) vessel wall structure; 2) dynamic elastic properties, and 3) reactive vessel properties. The overview shows that ultrasound is a versatile, non-invasive, and safe imaging technique that can be adopted for obtaining information about function, structure, and reactivity in superficial arteries. The most suitable setting for a specific application must be selected according to spatial and temporal resolution requirements. The usefulness of standardization in the validation process and performance metric adoption emerges. Computer-based techniques should always be preferred to manual measures, as long as the algorithms and learning procedures are transparent and well described, and the performance leads to better results. Identification of a minimal clinically important difference is a crucial point for drawing conclusions regarding robustness of the techniques and for the translation into practice of any biomarker.This article is based upon work from COST Action CA18216 VascAgeNet, supported by COST (European Cooperation in Science and Technology, www.cost.eu). A.G. has received funding from “La Caixa” Foundation (LCF/BQ/PR22/11920008). R.E.C is supported by the National Health and Medical Research Council of Australia (reference: 2009005) and by a National Heart Foundation Future Leader Fellowship (reference: 105636). J.A. acknowledges support from the British Heart Foundation [PG/15/104/31913], the Wellcome EPSRC Centre for Medical Engineering at King's College London [WT 203148/Z/16/Z], and the Cardiovascular MedTech Co-operative at Guy's and St Thomas' NHS Foundation Trust [MIC-2016-019]